Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition

doi:10.1109/TIP.2016.2552404

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition
	Du, Yong1 ; Fu, Yun 2,3; Wang, Liang1,4
刊名	IEEE TRANSACTIONS ON IMAGE PROCESSING
	2016-07-01
卷号	25 期号:7 页码:3010-3022
关键词	Action Recognition Hierarchical Recurrent Neural Network Random Scale & Rotation Transformations Skeleton
DOI	10.1109/TIP.2016.2552404
文献子类	Article
英文摘要	Motion characteristics of human actions can be represented by the position variation of skeleton joints. Traditional approaches generally extract the spatial-temporal representation of the skeleton sequences with well-designed hand-crafted features. In this paper, in order to recognize actions according to the relative motion between the limbs and the trunk, we propose an end-to-end hierarchical RNN for skeleton-based action recognition. We divide human skeleton into five main parts in terms of the human physical structure, and then feed them to five independent subnets for local feature extraction. After the following hierarchical feature fusion and extraction from local to global, dimensions of the final temporal dynamics representations are reduced to the same number of action categories in the corresponding data set through a single-layer perceptron. In addition, the output of the perceptron is temporally accumulated as the input of a softmax layer for classification. Random scale and rotation transformations are employed to improve the robustness during training. We compare with five other deep RNN variants derived from our model in order to verify the effectiveness of the proposed network. In addition, we compare with several other methods on motion capture and Kinect data sets. Furthermore, we evaluate the robustness of our model trained with random scale and rotation transformations for a multiview problem. Experimental results demonstrate that our model achieves the state-of-the-art performance with high computational efficiency.
WOS关键词	NETWORKS ; SEQUENCE
WOS研究方向	Computer Science ; Engineering
语种	英语
WOS记录号	WOS:000376087700006
资助机构	National Basic Research Program of China(2012CB316300) ; National Science Foundation(1314484) ; Strategic Priority Research Program within the Chinese Academy of Sciences(XDB02070100) ; National Natural Science Foundation of China(61525306 ; 61420106015)
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/11695]
专题	自动化研究所_智能感知与计算研究中心
通讯作者	Wang, Liang
作者单位	1.Chinese Acad Sci, Inst Automat, Ctr Res Intelligent Percept & Comp, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 2.Northeastern Univ, Dept Elect & Comp Engn, Coll Engn, Boston, MA 02115 USA 3.Northeastern Univ, Coll Comp & Informat Sci, Boston, MA 02115 USA 4.CAS Ctr Excellence Brain Sci & Intelligence Techn, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Du, Yong,Fu, Yun,Wang, Liang. Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2016,25(7):3010-3022.
APA	Du, Yong,Fu, Yun,&Wang, Liang.(2016).Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,25(7),3010-3022.
MLA	Du, Yong,et al."Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 25.7(2016):3010-3022.