Video frame interpolation (VFI) aims to improve the temporal resolution of a video sequence. Most of the existing deep learning based VFI methods adopt off-the-shelf optical flow algorithms to estimate the bidirectional flows and interpolate the missing frames accordingly. Though having achieved a great success, these methods require much human experience to tune the bidirectional flows and often generate unpleasant results when the estimated flows are not accurate. In this work, we rethink the VFI problem and formulate it as a continuous image transition (CIT) task, whose key issue is to transition an image from one space to another space continuously. More specifically, we learn to implicitly decouple the images into a translatable flow space and a non-translatable feature space. The former depicts the translatable states between the given images, while the later aims to reconstruct the intermediate features that cannot be directly translated. In this way, we can easily perform image interpolation in the flow space and intermediate image synthesis in the feature space, obtaining a CIT model. The proposed space decoupled learning (SDL) approach is simple to implement, while it provides an effective framework to a variety of CIT problems beyond VFI, such as style transfer and image morphing. Our extensive experiments on a variety of CIT tasks demonstrate the superiority of SDL to existing methods. The source code and models can be found at \url{https://github.com/yangxy/SDL}.
Decoupled Video Generation with Chain of Training-free Diffusion Model Experts
University of Sydney（悉尼大学）；Southeast University（东南大学）；Central South University（中南大学）；Shanghai Jiaotong University（上海交通大学）；Sensetime Research（商汤科技研究院）；Hong Kong University of Science and Technology（香港科技大学）
A General Decoupled Learning Framework for Parameterized Image Operators
Stanford University（斯坦福大学）；University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；Wormpex AI Research（Wormpex人工智能研究院）；Peking University（北京大学）
Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Transvascular Implantation Devices of the Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管植入设备国家重点实验室）；Transvascular Implantation Devices Research Institute（血管植入设备研究院）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）
Decoupled Spatial-Temporal Transformer for Video Inpainting
CUHK-SenseTime Joint Laboratory, The Chinese University of Hong Kong（香港中文大学-商汤科技联合实验室，香港中文大学）；SenseTime Research（商汤科技研究院）；Zhejiang University（浙江大学）；School of CST, Xidian University（西安电子科技大学计算机科学与技术学院）
D$^2$Turb: Depth-Aware Simulation and Decoupled Learning for Single-Frame Atmospheric Turbulence Mitigation
University of Electronic Science and Technology of China（电子科技大学）；Beijing Institute of Space Mechanics and Electricity（北京空间机械与电子研究所）；School of Physics, Northwest University Xi'an（西安西北大学物理学院）
