Learning Effective RGB-D Representations for Scene Recognition

doi:10.1109/TIP.2018.2872629

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Learning Effective RGB-D Representations for Scene Recognition
	Jiang, Shuqiang 2,3; Song, Xinhang 2,3; Herranz, Luis 1; Chen, Chengpeng 2,3
刊名	IEEE TRANSACTIONS ON IMAGE PROCESSING
	2019-02-01
卷号	28 期号:2 页码:980-993
关键词	Scene recognition deep learning multimodal RGB-D video CNN RNN
ISSN号	1057-7149
DOI	10.1109/TIP.2018.2872629
英文摘要	Deep convolutional networks can achieve impressive results on RGB scene recognition thanks to large data sets such as places. In contrast, RGB-D scene recognition is still underdeveloped in comparison, due to two limitations of RGB-D data we address in this paper. The first limitation is the lack of depth data for training deep learning models. Rather than fine tuning or transferring RGB-specific features, we address this limitation by proposing an architecture and a two-step training approach that directly learns effective depth-specific features using weak supervision via patches. The resulting RGB-D model also benefits from more complementary multimodal features. Another limitation is the short range of depth sensors (typically 0.5 m to 5.5 m), resulting in depth images not capturing distant objects in the scenes that RGB images can. We show that this limitation can he addressed by using RGB-D videos, where more comprehensive depth information is accumulated as the camera travels across the scenes. Focusing on this scenario, we introduce the ISIA RGB-D video data set to evaluate RGB-D scene recognition with videos. Our video recognition architecture combines convolutional and recurrent neural networks that are trained in three steps with increasingly complex data to learn effective features (i.e., patches, frames, and sequences). Our approach obtains the state-of-the-art performances on RGB-D image (NYUD2 and SUN RGB-D) and video (ISIA RGB-D) scene recognition.
资助项目	National Natural Science Foundation of China[61532018] ; Lenovo Outstanding Young Scientists Program ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals ; National Postdoctoral Program for Innovative Talents[BX201700255] ; China Postdoctoral Science Foundation[2018M631583] ; European Union Research and Innovation Program under the Marie Sklodowska-Curie Grant[6655919]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000448501800011
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/3638]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Univ Autonoma Barcelona, Comp Vis Ctr, E-08193 Barcelona, Spain 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Song, Xinhang,Herranz, Luis,et al. Learning Effective RGB-D Representations for Scene Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2019,28(2):980-993.
APA	Jiang, Shuqiang,Song, Xinhang,Herranz, Luis,&Chen, Chengpeng.(2019).Learning Effective RGB-D Representations for Scene Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,28(2),980-993.
MLA	Jiang, Shuqiang,et al."Learning Effective RGB-D Representations for Scene Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 28.2(2019):980-993.