Video Super-Resolution via Bidirectional Recurrent Convolutional Networks

doi:10.1109/TPAMI.2017.2701380

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	Video Super-Resolution via Bidirectional Recurrent Convolutional Networks
	Huang, Yan1,2 ; Wang, Wei1,2 ; Wang, Liang1,2,3
刊名	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
	2018-04-01
卷号	40 期号:4 页码:1015-1028
关键词	Deep Learning Recurrent Neural Networks 3d Convolution Video Super-resolution
DOI	10.1109/TPAMI.2017.2701380
文献子类	Article
英文摘要	Super resolving a low-resolution video, namely video super-resolution (SR), is usually handled by either single-image SR or multi-frame SR. Single-Image SR deals with each video frame independently, and ignores intrinsic temporal dependency of video frames which actually plays a very important role in video SR. Multi-Frame SR generally extracts motion information, e.g., optical flow, to model the temporal dependency, but often shows high computational cost. Considering that recurrent neural networks (RNNs) can model long-term temporal dependency of video sequences well, we propose a fully convolutional RNN named bidirectional recurrent convolutional network for efficient multi-frame SR. Different from vanilla RNNs, 1) the commonly-used full feedforward and recurrent connections are replaced with weight-sharing convolutional connections. So they can greatly reduce the large number of network parameters and well model the temporal dependency in a finer level, i.e., patch-based rather than frame-based, and 2) connections from input layers at previous timesteps to the current hidden layer are added by 3D feedforward convolutions, which aim to capture discriminate spatio-temporal patterns for short-term fast-varying motions in local adjacent frames. Due to the cheap convolutional operations, our model has a low computational complexity and runs orders of magnitude faster than other multi-frame SR methods. With the powerful temporal dependency modeling, our model can super resolve videos with complex motions and achieve well performance.
WOS关键词	IMAGE SUPERRESOLUTION ; LEARNING ALGORITHM ; NEURAL-NETWORKS ; RESOLUTION ; REGISTRATION
WOS研究方向	Computer Science ; Engineering
语种	英语
WOS记录号	WOS:000426687100018
资助机构	National Natural Science Foundation of China(61572504 ; Strategic Priority Research Program of the CAS(XDB02070100) ; National Key Research and Development Program of China(2016YFB1001000) ; Beijing Natural Science Foundation(4162058) ; NVIDIA DGX-1 AI Supercomputer ; NVIDIA ; 61525306 ; 61420106015 ; 61633021)
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/14820]
专题	自动化研究所_智能感知与计算研究中心
作者单位	1.Chinese Acad Sci CASIA, Inst Automat, NLPR, Ctr Res Intelligent Percept & Comp CRIPAC, Beijing 100049, Peoples R China 2.UCAS, Beijing 100049, Peoples R China 3.Chinese Acad Sci CASIA, Inst Automat, CEBSIT, Beijing 100864, Peoples R China
推荐引用方式 GB/T 7714	Huang, Yan,Wang, Wei,Wang, Liang. Video Super-Resolution via Bidirectional Recurrent Convolutional Networks[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2018,40(4):1015-1028.
APA	Huang, Yan,Wang, Wei,&Wang, Liang.(2018).Video Super-Resolution via Bidirectional Recurrent Convolutional Networks.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,40(4),1015-1028.
MLA	Huang, Yan,et al."Video Super-Resolution via Bidirectional Recurrent Convolutional Networks".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 40.4(2018):1015-1028.