Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction | |
Niu, Mingyue7; Zhao, Ziping7; Tao, Jianhua4,5,6; Li, Ya3; Schuller, Bjorn1,2 | |
刊名 | IEEE TRANSACTIONS ON AFFECTIVE COMPUTING |
2023-07-01 | |
卷号 | 14期号:3页码:1954-1965 |
关键词 | Depression Feature extraction Tensors Databases Spatiotemporal phenomena Convolutional neural networks Physiology DAER network depression level prediction dual attention block element recalibration block facial differences |
ISSN号 | 1949-3045 |
DOI | 10.1109/TAFFC.2022.3177737 |
通讯作者 | Zhao, Ziping(ztianjin@126.com) ; Tao, Jianhua(jhtao@nlpr.ia.ac.cn) |
英文摘要 | Physiological studies have identified that facial dynamics can be considered as biomarkers to analyze depression severity. This paper accordingly develops a Dual Attention and Element Recalibration (DAER) network to extract facial changes to predict the depression level. In this model, we propose two blocks: a Dual Attention (DA) block and Element Recalibration (ER) block. The DA block uses the self-attention to investigate the dynamic changes in the representation sequence of a facial video segment. It further examines the influence of feature components of the representation sequence on depression level prediction through bilinear-attention. Moreover, to improve the representation ability of network, the ER block is used to obtain the global information to recalibrate each element of the tensor. Adopting this approach, for the depression level prediction task, we first divide the long-term video into fixed-length segments and use the trained ResNet50 to encode each frame to generate the representation sequences of video segments. Second, the representation sequences are input into DAER network to obtain the depression level scores. Finally, the average of these scores yields the prediction result corresponding to the long-term video. Experiments on publicly available AVEC 2013 and AVEC 2014 depression databases illustrate the effectiveness of our method. |
资助项目 | Open Project Program of the National Laboratory of Pattern Recognition (NLPR) ; National Natural Science Foundation of China[202200012] ; New Talent Project of Beijing University of Posts and Telecommunications[62071330] ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR) ; National Natural Science Foundation of China[202200012] ; New Talent Project of Beijing University of Posts and Telecommunications[62071330] ; [2021RC37] |
WOS关键词 | FACIAL APPEARANCE ; RECOGNITION |
WOS研究方向 | Computer Science |
语种 | 英语 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
WOS记录号 | WOS:001075041900019 |
资助机构 | Open Project Program of the National Laboratory of Pattern Recognition (NLPR) ; National Natural Science Foundation of China ; New Talent Project of Beijing University of Posts and Telecommunications ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR) ; National Natural Science Foundation of China ; New Talent Project of Beijing University of Posts and Telecommunications |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/52996] |
专题 | 多模态人工智能系统全国重点实验室 |
通讯作者 | Zhao, Ziping; Tao, Jianhua |
作者单位 | 1.Imperial Coll London, GLAM Grp Language Audio & Mus, London SW7 2BX, England 2.Univ Augsburg, ZD B Chair Embedded Intelligence Hlth Care & Wellb, D-86159 Augsburg, Germany 3.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China 4.CAS Ctr Excellence Brain Sci & Intelligence Techno, Beijing 100190, Peoples R China 5.Univ Chinese Acad Sci UCAS, Sch Artificial Intelligence, Beijing 100049, Peoples R China 6.Inst Automat Chinese Acad Sci CASIA, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China 7.Tianjin Normal Univ TJNU, Sch Comp & Informat Engn, Tianjin 300387, Peoples R China |
推荐引用方式 GB/T 7714 | Niu, Mingyue,Zhao, Ziping,Tao, Jianhua,et al. Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction[J]. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING,2023,14(3):1954-1965. |
APA | Niu, Mingyue,Zhao, Ziping,Tao, Jianhua,Li, Ya,&Schuller, Bjorn.(2023).Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction.IEEE TRANSACTIONS ON AFFECTIVE COMPUTING,14(3),1954-1965. |
MLA | Niu, Mingyue,et al."Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction".IEEE TRANSACTIONS ON AFFECTIVE COMPUTING 14.3(2023):1954-1965. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论