ReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues

doi:10.1007/s11633-023-1478-9

CORC > 自动化研究所 > 中国科学院自动化研究所 > 学术期刊 > International Journal of Automation and Computing

	ReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues
	Ho Yin Au 2; Jie Chen 2; Junkun Jiang 2; Yike Guo 1
刊名	Machine Intelligence Research
	2024
卷号	21 期号:4 页码:771-781
关键词	Generative model cross-modality learning normalizing flow tempo synchronization style transfer
ISSN号	2731-538X
DOI	10.1007/s11633-023-1478-9
英文摘要	To generate dance that temporally and aesthetically matches the music is a challenging problem in three aspects. First, the generated motion should be beats-aligned to the local musical features. Second, the global aesthetic style should be matched between motion and music. And third, the generated motion should be diverse and non-self-repeating. To address these challenges, we propose ReChoreoNet, which re-choreographs high-quality dance motion for a given piece of music. A data-driven learning strategy is proposed to efficiently correlate the temporal connections between music and motion in a progressively learned cross-modality embedding space. The beats-aligned content motion will be subsequently used as autoregressive context and control signal to control a normalizing-flow model, which transfers the style of a prototype motion to the final generated dance. In addition, we present an aesthetically labelled music-dance repertoire (MDR) for both efficient learning of the cross-modality embedding, and understanding of the aesthetic connections between music and motion. We demonstrate that our repertoire-based framework is robustly extensible in both content and style. Both quantitative and qualitative experiments have been carried out to validate the efficiency of our proposed model.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/58571]
专题	自动化研究所_学术期刊_International Journal of Automation and Computing
作者单位	1.Department of Computer Science and Engineering, The Hong Kong University of Science and Engineering, Hong Kong 999077, China 2.Department of Computer Science, Hong Kong Baptist University, Hong Kong 999077, China
推荐引用方式 GB/T 7714	Ho Yin Au,Jie Chen,Junkun Jiang,et al. ReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues[J]. Machine Intelligence Research,2024,21(4):771-781.
APA	Ho Yin Au,Jie Chen,Junkun Jiang,&Yike Guo.(2024).ReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues.Machine Intelligence Research,21(4),771-781.
MLA	Ho Yin Au,et al."ReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues".Machine Intelligence Research 21.4(2024):771-781.