Sequence Multi-Labeling: A Unified Video Annotation Scheme With Spatial   and Temporal Context

doi:10.1109/TMM.2010.2066960

CORC > 北京大学 > 信息科学技术学院

	Sequence Multi-Labeling: A Unified Video Annotation Scheme With Spatial and Temporal Context
	Li, Yuanning ; Tian, Yonghong ; Duan, Ling-Yu ; Yang, Jingjing ; Huang, Tiejun ; Gao, Wen
刊名	ieee transactions on multimedia
	2010
关键词	Sequence multi-labeling spatial correlation temporal correlation video annotation CONCEPT FUSION FRAMEWORK RETRIEVAL
DOI	10.1109/TMM.2010.2066960
英文摘要	Automatic video annotation is a challenging yet important problem for content-based video indexing and retrieval. In most existing works, annotation is formulated as a multi-labeling problem over individual shots. However, video is by nature informative in spatial and temporal context of semantic concepts. In this paper, we formulate video annotation as a sequence multi-labeling (SML) problem over a shot sequence. Different from many video annotation paradigms working on individual shots, SML aims to predict a multi-label sequence for consecutive shots in a global optimization manner by incorporating spatial and temporal context into a unified learning framework. A novel discriminative method, called sequence multi-label support vector machine (SVMSML), is accordingly proposed to infer the multi-label sequence for a given shot sequence. In (SVMSML), a joint kernel is employed to model the feature-level and concept-level context relationships (i.e., the dependencies of concepts on the low-level features, spatial and temporal correlations of concepts). A multiple-kernel learning (MKL) algorithm is developed to optimize the kernel weights of the joint kernel as well as the SML score function. To efficiently search the desirable multi-label sequence over the large output space in both training and test phases, we adopt an approximate method to maximize the energy of a binary Markov random field (BMRF). Extensive experiments on TRECVID'05 and TRECVID'07 datasets have shown that our proposed (SVMSML) gains superior performance over the state-of-the-art.; http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000284365100004&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=8e1609b174ce4e31116a60747a720701 ; Computer Science, Information Systems; Computer Science, Software Engineering; Telecommunications; SCI(E); EI; 14; ARTICLE; 8; 814-828; 12
语种	英语
内容类型	期刊论文
源URL	[http://ir.pku.edu.cn/handle/20.500.11897/241549]
专题	信息科学技术学院
推荐引用方式 GB/T 7714	Li, Yuanning,Tian, Yonghong,Duan, Ling-Yu,et al. Sequence Multi-Labeling: A Unified Video Annotation Scheme With Spatial and Temporal Context[J]. ieee transactions on multimedia,2010.
APA	Li, Yuanning,Tian, Yonghong,Duan, Ling-Yu,Yang, Jingjing,Huang, Tiejun,&Gao, Wen.(2010).Sequence Multi-Labeling: A Unified Video Annotation Scheme With Spatial and Temporal Context.ieee transactions on multimedia.
MLA	Li, Yuanning,et al."Sequence Multi-Labeling: A Unified Video Annotation Scheme With Spatial and Temporal Context".ieee transactions on multimedia (2010).