Learning Effective RGB-D Representations for Scene Recognition | |
Jiang, Shuqiang2,3; Song, Xinhang2,3; Herranz, Luis1; Chen, Chengpeng2,3 | |
刊名 | IEEE TRANSACTIONS ON IMAGE PROCESSING
![]() |
2019-02-01 | |
卷号 | 28期号:2页码:980-993 |
关键词 | Scene recognition deep learning multimodal RGB-D video CNN RNN |
ISSN号 | 1057-7149 |
DOI | 10.1109/TIP.2018.2872629 |
英文摘要 | Deep convolutional networks can achieve impressive results on RGB scene recognition thanks to large data sets such as places. In contrast, RGB-D scene recognition is still underdeveloped in comparison, due to two limitations of RGB-D data we address in this paper. The first limitation is the lack of depth data for training deep learning models. Rather than fine tuning or transferring RGB-specific features, we address this limitation by proposing an architecture and a two-step training approach that directly learns effective depth-specific features using weak supervision via patches. The resulting RGB-D model also benefits from more complementary multimodal features. Another limitation is the short range of depth sensors (typically 0.5 m to 5.5 m), resulting in depth images not capturing distant objects in the scenes that RGB images can. We show that this limitation can he addressed by using RGB-D videos, where more comprehensive depth information is accumulated as the camera travels across the scenes. Focusing on this scenario, we introduce the ISIA RGB-D video data set to evaluate RGB-D scene recognition with videos. Our video recognition architecture combines convolutional and recurrent neural networks that are trained in three steps with increasingly complex data to learn effective features (i.e., patches, frames, and sequences). Our approach obtains the state-of-the-art performances on RGB-D image (NYUD2 and SUN RGB-D) and video (ISIA RGB-D) scene recognition. |
资助项目 | National Natural Science Foundation of China[61532018] ; Lenovo Outstanding Young Scientists Program ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals ; National Postdoctoral Program for Innovative Talents[BX201700255] ; China Postdoctoral Science Foundation[2018M631583] ; European Union Research and Innovation Program under the Marie Sklodowska-Curie Grant[6655919] |
WOS研究方向 | Computer Science ; Engineering |
语种 | 英语 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
WOS记录号 | WOS:000448501800011 |
内容类型 | 期刊论文 |
源URL | [http://119.78.100.204/handle/2XEOYT63/3638] ![]() |
专题 | 中国科学院计算技术研究所期刊论文_英文 |
通讯作者 | Jiang, Shuqiang |
作者单位 | 1.Univ Autonoma Barcelona, Comp Vis Ctr, E-08193 Barcelona, Spain 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China |
推荐引用方式 GB/T 7714 | Jiang, Shuqiang,Song, Xinhang,Herranz, Luis,et al. Learning Effective RGB-D Representations for Scene Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2019,28(2):980-993. |
APA | Jiang, Shuqiang,Song, Xinhang,Herranz, Luis,&Chen, Chengpeng.(2019).Learning Effective RGB-D Representations for Scene Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,28(2),980-993. |
MLA | Jiang, Shuqiang,et al."Learning Effective RGB-D Representations for Scene Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 28.2(2019):980-993. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论