A multimodal approach of generating 3D human-like talking agent

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 人机语音交互团队

	A multimodal approach of generating 3D human-like talking agent
	Yang, Minghao; Tao, Jianhua; Mu, Kaihui; Li, Ya; Che, Jianfeng
刊名	JOURNAL ON MULTIMODAL USER INTERFACES
	2012-03-01
卷号	5 期号:1-2 页码:61-68
关键词	Multimodal 3D talking agent Lip movement Head motion MFCC Facial expression Gesture animation
英文摘要	This paper introduces a multimodal framework of generating a 3D human-like talking agent which can communicate with user through speech, lip movement, head motion, facial expression and body animation. In this framework, lip movements are obtained by searching and matching acoustic features which are represented by Mel-frequency cepstral coefficients (MFCC) in audio-visual bimodal database. Head motion is synthesized by visual prosody which maps textual prosodic features into rotational and translational parameters. Facial expression and body animation are generated by transferring motion data to skeleton. A simplified high level Multimodal Marker Language (MML), in which only a few fields are used to coordinate the agent channels, is introduced to drive the agent. The experiments validate the effectiveness of the proposed multimodal framework.
WOS标题词	Science & Technology ; Technology
类目[WOS]	Computer Science, Artificial Intelligence ; Computer Science, Cybernetics
研究领域[WOS]	Computer Science
关键词[WOS]	CHARACTER ANIMATION ; FACE
收录类别	SCI
语种	英语
WOS记录号	WOS:000309997800008
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/3228]
专题	自动化研究所_模式识别国家重点实验室_人机语音交互团队
作者单位	Chinese Acad Sci, Inst Automat, NLPR, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Yang, Minghao,Tao, Jianhua,Mu, Kaihui,et al. A multimodal approach of generating 3D human-like talking agent[J]. JOURNAL ON MULTIMODAL USER INTERFACES,2012,5(1-2):61-68.
APA	Yang, Minghao,Tao, Jianhua,Mu, Kaihui,Li, Ya,&Che, Jianfeng.(2012).A multimodal approach of generating 3D human-like talking agent.JOURNAL ON MULTIMODAL USER INTERFACES,5(1-2),61-68.
MLA	Yang, Minghao,et al."A multimodal approach of generating 3D human-like talking agent".JOURNAL ON MULTIMODAL USER INTERFACES 5.1-2(2012):61-68.