Learning to Answer Complex Visual Questions from Multi-View Analysis | |
Zhu MJ(朱敏郡); Weng YX(翁诣轩); He SZ(何世柱); Liu K(刘康); Zhao J(赵军) | |
2022-08 | |
会议日期 | 2022 |
会议地点 | 中国秦皇岛 |
英文摘要 | Visual Question Answering (VQA) has received increasing attention in NLP research. Most VQA images focus on natural scenes. However, some images widely used in textbooks such as diagrams often contain complicated and abstract information (e.g. constructed graphs with logic and concepts). Therefore, Diagram Question answering (DQA) is a challenging but significant task, which is also helpful for machines to understand human cognitive behaviors and learning habits. On DQA task, we propose a multi-perspective understanding based visual question-answering method, which constructs a variety of different self-monitoring tasks in the form of prompts to help the model learn deeper information. For the first time, we propose a decoding method of "Cross Entropy constraint Decoding", which can effectively constrain the content generated by the text when performing multiple selection tasks. This method has obtained SOTA in the evaluation task of CCKS-2022, which fully proves the effectiveness of the method. |
会议录出版者 | IEEE |
内容类型 | 会议论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/52286] |
专题 | 模式识别国家重点实验室_自然语言处理 |
通讯作者 | Zhao J(赵军) |
作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhu MJ,Weng YX,He SZ,et al. Learning to Answer Complex Visual Questions from Multi-View Analysis[C]. 见:. 中国秦皇岛. 2022. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论