CLOSE: Coupled content-semantic embedding

doi:10.1007/s11760-019-01449-w

	CLOSE: Coupled content-semantic embedding
	Ren, Junhong; Zhang, Wensheng
刊名	SIGNAL IMAGE AND VIDEO PROCESSING
	2019-09-01
卷号	13 期号:6 页码:1087-1095
关键词	Video captioning Coupled content-semantic embedding Multi-content embedding
ISSN号	1863-1703
DOI	10.1007/s11760-019-01449-w
通讯作者	Ren, Junhong(junhong.ren@ia.ac.cn)
英文摘要	This paper proposes a novel coupled content semantic embedding (CLOSE) method with its application to video captioning. The motivation behind this design is to seek a consistent latent space between the content-semantic pair, in which the pair with same attribute is close to each other. Under the framework constructed on content-semantic embedding, CLOSE first learns two independent and reversible content-content and semantic-semantic embeddings, respectively, and then aggregates the two items via a coupled content-semantic embedding. Benefitting from the reversible property, our CLOSE can be pretrained with quantities of unlabeled data. In addition, casting on the work setting of feature embedding, a paradigm named multi-content embedding (MCE) is developed to describe the multi-focus information. Typically, MCE is capable of learning a feature embedding that can capture multiple discriminative contents. Extensive experiments compared with state-of-the-art methods on benchmark datasets, i.e., MSVD and MSR-VTT, demonstrate the effectiveness and superiority of the proposed CLOSE.
资助项目	National Natural Science Foundation of China[61403376]
WOS研究方向	Engineering ; Imaging Science & Photographic Technology
语种	英语
出版者	SPRINGER LONDON LTD
WOS记录号	WOS:000481886600006
资助机构	National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/27526]
专题	中国科学院自动化研究所
通讯作者	Ren, Junhong
作者单位	Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Ren, Junhong,Zhang, Wensheng. CLOSE: Coupled content-semantic embedding[J]. SIGNAL IMAGE AND VIDEO PROCESSING,2019,13(6):1087-1095.
APA	Ren, Junhong,&Zhang, Wensheng.(2019).CLOSE: Coupled content-semantic embedding.SIGNAL IMAGE AND VIDEO PROCESSING,13(6),1087-1095.
MLA	Ren, Junhong,et al."CLOSE: Coupled content-semantic embedding".SIGNAL IMAGE AND VIDEO PROCESSING 13.6(2019):1087-1095.