A Unified Framework for Multi-Modal Isolated Gesture Recognition

doi:10.1145/3131343

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 生物识别与安全技术研究中心

	A Unified Framework for Multi-Modal Isolated Gesture Recognition
	Duan, Jiali 1,2; Wan, Jun1,2 ; Zhou, Shuai 3; Guo, Xiaoyuan 4; Li, Stan Z.1,2
刊名	ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
	2018-04-01
卷号	14 期号:1 页码:16
关键词	Multi-modal consensus-voting 3D convolution isolated gesture recognition
ISSN号	1551-6857
DOI	10.1145/3131343
通讯作者	Wan, Jun(jun.wan@ia.ac.cn)
英文摘要	In this article, we focus on isolated gesture recognition and explore different modalities by involving RGB stream, depth stream, and saliency stream for inspection. Our goal is to push the boundary of this realm even further by proposing a unified framework that exploits the advantages of multi-modality fusion. Specifically, a spatial-temporal network architecture based on consensus-voting has been proposed to explicitly model the long-term structure of the video sequence and to reduce estimation variance when confronted with comprehensive inter-class variations. In addition, a three-dimensional depth-saliency convolutional network is aggregated in parallel to capture subtle motion characteristics. Extensive experiments are done to analyze the performance of each component and our proposed approach achieves the best results on two public benchmarks, ChaLearn IsoGD and RGBD-HuDaAct, outperforming the closest competitor by a margin of over 10% and 15%, respectively. Our project and codes will be released at https:// davidsonic.github. io/index/acm_tomm_2017.html.
资助项目	National Key Research and Development Plan[2016YFC0801002] ; Chinese National Natural Science Foundation[61502491] ; Chinese National Natural Science Foundation[61473291] ; Chinese National Natural Science Foundation[61572501] ; Chinese National Natural Science Foundation[61572536] ; Chinese National Natural Science Foundation[61673052] ; Science and Technology Development Fund of Macau[112/2014/A3] ; NVIDIA GPU ; AuthenMetric RD Funds
WOS关键词	REAL-TIME
WOS研究方向	Computer Science
语种	英语
出版者	ASSOC COMPUTING MACHINERY
WOS记录号	WOS:000433517100007
资助机构	National Key Research and Development Plan ; Chinese National Natural Science Foundation ; Science and Technology Development Fund of Macau ; NVIDIA GPU ; AuthenMetric RD Funds
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/15302]
专题	自动化研究所_模式识别国家重点实验室_生物识别与安全技术研究中心
通讯作者	Wan, Jun
作者单位	1.Chinese Acad Sci, CBSR, Inst Automat, Beijing, Peoples R China 2.Chinese Acad Sci, NLPR, Inst Automat, Beijing, Peoples R China 3.Macau Univ Sci & Technol, Taipa, Macao, Peoples R China 4.Univ Chinese Acad Sci, Sch Engn Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Duan, Jiali,Wan, Jun,Zhou, Shuai,et al. A Unified Framework for Multi-Modal Isolated Gesture Recognition[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2018,14(1):16.
APA	Duan, Jiali,Wan, Jun,Zhou, Shuai,Guo, Xiaoyuan,&Li, Stan Z..(2018).A Unified Framework for Multi-Modal Isolated Gesture Recognition.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,14(1),16.
MLA	Duan, Jiali,et al."A Unified Framework for Multi-Modal Isolated Gesture Recognition".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 14.1(2018):16.