Depth assisted novel view synthesis using few images

doi:10.1016/j.imavis.2024.105079

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 机器人视觉团队

	Depth assisted novel view synthesis using few images
	Li, Qian 2; Fu, Rao 2; Tang, Fulin 1
刊名	IMAGE AND VISION COMPUTING
	2024-07-01
卷号	147 页码:9
关键词	Neural radiance fields View synthesis Image warping
ISSN号	0262-8856
DOI	10.1016/j.imavis.2024.105079
通讯作者	Fu, Rao(rao.fu@inria.fr)
英文摘要	In this paper, we introduce a novel approach to improve the performance of Neural Radiance Fields (NeRF) from limited input views. NeRF has exhibited impressive capabilities in producing photo-realistic renderings when trained on dense input views, but its performance degrades as the number of training views decreases. Our key insight is that the original NeRF lacks geometric regularization and appearance information due to limited inputs, resulting in an over-fitting issue. To address this challenge, we present a novel method: first, a global sampling method with geometric regularization is employed by utilizing warped images as additional pseudoviews, which optimizes the multi-view consistency during the training. Second, we introduce a local patch sampling technique with perceptual regularization to ensure pixel correspondence in appearance. Furthermore, we incorporate depth information for explicit geometry regularization. We evaluate our method on the DTU dataset and LLFF dataset from a different number of inputs. Extensive evaluations demonstrate that our approach outperforms existing benchmarks across various metrics, achieving state-of-the-art results.
WOS研究方向	Computer Science ; Engineering ; Optics
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:001244560700001
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/58701]
专题	自动化研究所_模式识别国家重点实验室_机器人视觉团队
通讯作者	Fu, Rao
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China 2.Inria, Le Chesnay Rocquencourt, France
推荐引用方式 GB/T 7714	Li, Qian,Fu, Rao,Tang, Fulin. Depth assisted novel view synthesis using few images[J]. IMAGE AND VISION COMPUTING,2024,147:9.
APA	Li, Qian,Fu, Rao,&Tang, Fulin.(2024).Depth assisted novel view synthesis using few images.IMAGE AND VISION COMPUTING,147,9.
MLA	Li, Qian,et al."Depth assisted novel view synthesis using few images".IMAGE AND VISION COMPUTING 147(2024):9.