Video super-resolution (VSR) is a task that aims to reconstruct high-resolution (HR) frames from the low-resolution (LR) reference frame and multiple neighboring frames. The vital operation is to utilize the relative misaligned frames for the current frame reconstruction and preserve the consistency of the results. Existing methods generally explore information propagation and frame alignment to improve the performance of VSR. However, few studies focus on the temporal consistency of inter-frames. In this paper, we propose a Temporal Consistency learning Network (TCNet) for VSR in an end-to-end manner, to enhance the consistency of the reconstructed videos. A spatio-temporal stability module is designed to learn the self-alignment from inter-frames. Especially, the correlative matching is employed to exploit the spatial dependency from each frame to maintain structural stability. Moreover, a self-attention mechanism is utilized to learn the temporal correspondence to implement an adaptive warping operation for temporal consistency among multi-frames. Besides, a hybrid recurrent architecture is designed to leverage short-term and long-term information. We further present a progressive fusion module to perform a multistage fusion of spatio-temporal features. And the final reconstructed frames are refined by these fused features. Objective and subjective results of various experiments demonstrate that TCNet has superior performance on different benchmark datasets, compared to several state-of-the-art methods.
Layer Selection in Feature-Based Losses Affects Image Quality and Microstructural Consistency in Deep Learning Super-Resolution of Brain Diffusion MRI
Institue for Applied Medical Informatics, University Medical Center Hamburg-Eppendorf(应用医学信息学研究所,汉堡大学医学中心);Institute of Computational Neuroscience, University Medical Center Hamburg-Eppendorf(计算神经科学研究所,汉堡大学医学中心);Center for Biomedical Artificial Intelligence (bAIome), University Medical Center Hamburg-Eppendorf(生物医学人工智能中心(bAIome),汉堡大学医学中心)
Context Consistency Learning via Sentence Removal for Semi-Supervised Video Paragraph Grounding
School of Computer Science and Engineering, Sun Yat-sen University, China(中山大学计算机科学与工程学院);Guangdong Province Key Laboratory of Information Security Technology, China(广东省信息安全技术重点实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China(教育部机器智能与高级计算重点实验室);Guangdong University of Foreign Studies(广东外语外贸大学);Guangdong University of Technology(广东工业大学)
Uncertainty-Participation Context Consistency Learning for Semi-supervised Semantic Segmentation
School of Artificial Intelligence Nanjing Normal University Nanjing, China;School of Artificial Intelligence Nanjing Normal University Nanjing, China cs\;School of Artificial Intelligence Nanjing Normal University Nanjing, China zheng\
Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localization
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Macquarie University(麦考瑞大学);University of Chinese Academy of Sciences(中国科学院大学);University of California at Merced(加州大学默塞德分校);University of Adelaide(阿德莱德大学);University of Science and Technology of China(中国科学技术大学)
Spatio-Temporal Structure Consistency for Semi-supervised Medical Image Classification
Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)深圳大数据研究院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Boosting Semi-Supervised Medical Image Segmentation via Masked Image Consistency and Discrepancy Learning
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications(北京邮电大学信息与通信工程学院);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院)
An Intra- and Cross-frame Topological Consistency Scheme for Semi-supervised Atherosclerotic Coronary Plaque Segmentation
Xiamen University(厦门大学);University of Washington(华盛顿大学);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);National Institute for Data Science in Health and Medicine, Xiamen University(厦门大学健康与医学数据科学国家研究院)