Temporal Hallucinating for Action Recognition with Few Still Images

	Temporal Hallucinating for Action Recognition with Few Still Images
	Yali Wang; Lei Zhou; Yu Qiao
	2018
会议日期	2018
会议地点	美国
英文摘要	Action recognition in still images has been recently pro- moted by deep learning. However, the success of these deep models heavily depends on huge amount of training images for various action categories, which may not be available in practice. Alternatively, humans can classify new action categories after seeing few images, since we may not only compare appearance similarities between images on hand, but also attempt to recall importance motion cues from rel- evant action videos in our memory. To mimic this capacity, we propose a novel Hybrid Video Memory (HVM) machine, which can hallucinate temporal features of still images from video memory, in order to boost action recognition with few still images. First, we design a temporal memory module consisting of temporal hallucinating and predicting. Tem- poral hallucinating can generate temporal features of still images in an unsupervised manner. Hence, it can be flexi- bly used in realistic scenarios, where image and video cat- egories may not be consistent. Temporal predicting can effectively infer action categories for query image, by in- tegrating temporal features of training images and videos within a domain-adaptation manner. Second, we design a spatial memory module for spatial predicting. As spatial and temporal features are complementary to represent dif- ferent actions, we apply spatial-temporal prediction fusion to further boost performance. Finally, we design a video selection module to select strongly-relevant videos as mem- ory. In this case, we can balance the number of images and videos to reduce prediction bias as well as preserve com- putation efficiency. To show the effectiveness, we conduct extensive experiments on three challenging data sets, where our HVM outperforms a number of recent approaches by temporal hallucinating from video memory.
URL标识	查看原文
内容类型	会议论文
源URL	[http://ir.siat.ac.cn:8080/handle/172644/13685]
专题	深圳先进技术研究院_集成所
推荐引用方式 GB/T 7714	Yali Wang,Lei Zhou,Yu Qiao. Temporal Hallucinating for Action Recognition with Few Still Images[C]. 见:. 美国. 2018.