Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction

doi:10.1016/j.image.2023.117031

CORC > 自动化研究所 > 中国科学院自动化研究所 > 多模态人工智能系统全国重点实验室

	Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction
	Liu, Deyang 1,2,3; Mao, Yifan 2; Huang, Yan 4; Cao, Liqun 2; Wang, Yuanzhi 2; Fang, Yuming 3
刊名	SIGNAL PROCESSING-IMAGE COMMUNICATION
	2023-11-01
卷号	119 页码:11
关键词	Light field image Angular reconstruction Optical flow Multi-level fusion network
ISSN号	0923-5965
DOI	10.1016/j.image.2023.117031
通讯作者	Fang, Yuming(fa0001ng@e.ntu.edu.sg)
英文摘要	Light Field (LF) imaging can record both the intensities and directions of light rays in a single exposure, which has received extensive attentions. However, the limited angular resolution becomes the primary bottleneck for the wide-spread applications of LF imaging. To this end, this paper proposes a novel optical flow-assisted multi-level fusion network for LF angular reconstruction. In our method, we propose to infer the multi-angular optical flows to explore long-range dependency of LF sub-aperture images (SAIs) for high-quality angular reconstruction. By aligning the SAIs in multi-angular directions, the geometric consistency of reconstructed dense LF can be preserved. Moreover, a multi-level fusion framework for LF angular reconstruction is introduced, which consists of two stages, namely texture-optical flow feature fusion and parallax structure-information fusion. The former firstly extracts the texture and optical flow features from the reconstructed coarse LF and then fuses these two features by using the proposed texture-optical flow fusion-block. The latter further blends the LF parallax structure information with the fused texture and optical flow features using the proposed parallax structure-information fusion network. Comprehensive experiments on both real-world and synthetic LF scenes demonstrate the superiority of the proposed method for reconstructing high-quality dense LF. Moreover, practical application on depth estimation also validates that our method can recover more texture details, particularly for some occlusion regions.
资助项目	National Natural Science Foundation of China[62171002] ; National Natural Science Foundation of China[62132006] ; Shenzhen Municipal Science and Technology Innovation Council[2021Szvup051] ; STCSM[SKLSFO2021-05] ; Anhui Provincial Key Laboratory of Network and Information Security[AHNIS2023002] ; University Discipline Top Talent Program of Anhui[gxbjZD2022034] ; Anhui Outstanding Youth Fund by Colleges and Universities[2022AH030106]
WOS关键词	SUPERRESOLUTION
WOS研究方向	Engineering
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:001069612700001
资助机构	National Natural Science Foundation of China ; Shenzhen Municipal Science and Technology Innovation Council ; STCSM ; Anhui Provincial Key Laboratory of Network and Information Security ; University Discipline Top Talent Program of Anhui ; Anhui Outstanding Youth Fund by Colleges and Universities
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/53128]
专题	多模态人工智能系统全国重点实验室
通讯作者	Fang, Yuming
作者单位	1.Anhui Normal Univ, Natl Anhui Prov Key Lab Network & Informat Secur, Wuhu 240002, Peoples R China 2.Anqing Normal Univ, Sch Comp & Informat, Anqing 246000, Peoples R China 3.Jiangxi Univ Finance & Econ, Sch Informat Management, Nanchang 330000, Jiangxi, Peoples R China 4.Chinese Acad Sci CASIA, Natl Lab Pattern Recognit NLPR, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Liu, Deyang,Mao, Yifan,Huang, Yan,et al. Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction[J]. SIGNAL PROCESSING-IMAGE COMMUNICATION,2023,119:11.
APA	Liu, Deyang,Mao, Yifan,Huang, Yan,Cao, Liqun,Wang, Yuanzhi,&Fang, Yuming.(2023).Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction.SIGNAL PROCESSING-IMAGE COMMUNICATION,119,11.
MLA	Liu, Deyang,et al."Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction".SIGNAL PROCESSING-IMAGE COMMUNICATION 119(2023):11.