Instance-Aware Monocular 3D Semantic Scene Completion

doi:10.1109/TITS.2023.3344806

CORC > 力学研究所 > 中国科学院力学研究所 > 流固耦合系统力学重点实验室(2012-)

	Instance-Aware Monocular 3D Semantic Scene Completion
	Xiao, Haihong 2; Xu, Hongbin 2; Kang, Wenxiong 2; Li, Yuqiong 1
刊名	IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
	2024-01-02
页码	12
关键词	3D scene understanding semantic scene completion 3D vision
ISSN号	1524-9050
DOI	10.1109/TITS.2023.3344806
通讯作者	Kang, Wenxiong(auwxkang@scut.edu.cn)
英文摘要	We study outdoor 3D scene understanding, a challenging task demanding the intelligent system to infer both geometry and semantics from a single-view image - a critical skill for autonomous vehicles to navigate in the real 3D world. Towards this end, we present an instance-aware monocular semantic scene completion framework. To the best of our knowledge, this is the first endeavor specifically targeting the challenge of instance perception in the camera-based semantic scene completion task. Our method consists of two stages. In stage I, we design a region-based VQ-VAE network, providing an effective solution for 3D occupancy prediction. In stage II, we first introduce an instance-aware attention module, explicitly incorporating instance-level cues captured from mask images to enhance the instance features in RGB images. Then we leverage the deformable cross-attention to aggregate image features corresponding to each voxel query and utilize the deformable self-attention to refine query proposals. We combine these key ingredients and evaluate our method on two challenging datasets, namely SemanticKITTI and SSCBench-KITTI-360. The results unequivocally demonstrate the superiority of our proposed method over the state-of-the-art VoxFormer-S. Specifically, our method surpasses VoxFormer-S by 0.22 IoU and 0.72 mIoU on the validation set and achieves an impressive improvement of 3.04 IoU and 1.06 mIoU on the SSCBench-KITTI-360 validation set. Meanwhile, our approach ensures accurate perception of critical instances, thereby exhibiting its exceptional performance and potential for practical deployment.
资助项目	National Natural Science Foundation of China
WOS研究方向	Engineering ; Transportation
语种	英语
WOS记录号	WOS:001167317900001
资助机构	National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://dspace.imech.ac.cn/handle/311007/94549]
专题	力学研究所_流固耦合系统力学重点实验室(2012-)
通讯作者	Kang, Wenxiong
作者单位	1.Chinese Acad Sci, Inst Mech, Key Lab Mech Fluid Solid Coupling Syst, Beijing 100190, Peoples R China 2.South China Univ Technol, Sch Automat Sci & Engn, Guangzhou 511442, Peoples R China
推荐引用方式 GB/T 7714	Xiao, Haihong,Xu, Hongbin,Kang, Wenxiong,et al. Instance-Aware Monocular 3D Semantic Scene Completion[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2024:12.
APA	Xiao, Haihong,Xu, Hongbin,Kang, Wenxiong,&Li, Yuqiong.(2024).Instance-Aware Monocular 3D Semantic Scene Completion.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,12.
MLA	Xiao, Haihong,et al."Instance-Aware Monocular 3D Semantic Scene Completion".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2024):12.