Automatic segmentation, aggregation and indexing of multimodal news information from television and the Internet - Montagnuolo, M.; Messina, A.; Borgotallo, R. (Journal of Digital Information Management, 2010)
The global diffusion of the Internet has enabled the distribution of informative content through dynamic media such as RSS feeds and video blogs. At the same time, the decreasing cost of electronic devices has increased the pervasive availability of the same informative content in the form of digital audiovisual data. This article presents a system for the large-scale unsupervised acquisition, segmentation and indexing of TV newscasts. In particular, it discusses the principles and performance of the parts of the system dedicated to the detection and segmentation of programmes from the acquired stream. In addition to the core technology, we also introduce and discuss a novel method for assessing the results of story boundaries segmentation algorithms, based on a user-validated measurement. Due to the heterogeneity of current news distribution channels, a further innovative aspect of this article is the description of a framework for multimodal information aggregation. The core of this framework is a cross-modal clustering process for which a novel, asymmetric similarity measure is provided. The implemented prototype uses online news articles and TV news programmes as information sources, and provides a multimodal service integrating both contributions. Experimental evaluation of the system proves the effectiveness of the method in the studied case.
Keywords: Content segmentation; Multimedia annotation; Multimodal aggregation; Semantic enrichment

Multimedia knowledge representation for automatic annotation of broadcast TV archives - Montagnuolo, M.; Messina, A. (Journal of Digital Information Management, 2007)
Multimedia content classification and retrieval are indispensable tools in the current convergence of audiovisual entertainment and information media. Thanks to the development of broadband networks, every consumer will have digital video programmes available on-line as well as through the traditional distribution channels. In this scenario, since the early '90s, the most important TV broadcasters in Europe have started projects whose aim was to achieve preservation, restoration and automatic documentation of their audiovisual archives, In particular, the association of low-level multimedia features to knowledge and semantics for the purpose of automatic classification of multimedia archives is currently the target of many researchers in both academic and IT industrial communities, This paper describes our research direction, which is focusing on three points: (a) We first introduce a new taxonomy for classification of broadcast digital archives based on a novel theoretical approach. The advantage of this taxonomy is that it can provide an unambiguous representation of multi-media informative content from the relevant points of view to the broadcasters community. (b) We secondly present a multilayer multimedia database model to represent both structure and content of multimedia objects. (c) We further propose a framework architecture for building a Multimedia Fuzzy Annotation System (MFAS), and a description of our experimental plan.
Keywords: Knowledge-based multimedia systems; Semantic multimedia retrieval and annotation; Video taxonomy