Interpolation of sparse pixel information towards a dense target resolution finds its application across multiple disciplines in computer vision. State-of-the-art interpolation of motion fields applies model-based interpolation that makes use of edge information extracted from the target image. For depth completion, data-driven learning approaches are widespread. Our work is inspired by latest trends in depth completion that tackle the problem of dense guidance for sparse information. We extend these ideas and create a generic cross-domain architecture that can be applied for a multitude of interpolation problems like optical flow, scene flow, or depth completion. In our experiments, we show that our proposed concept of Sparse Spatial Guided Propagation (SSGP) achieves improvements to robustness, accuracy, or speed compared to specialized algorithms.
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In this paper, we propose a novel joint deblurring and multi-frame interpolation (DeMFI) framework, called DeMFI-Net, which accurately converts blurry videos of lower-frame-rate to sharp videos at higher-frame-rate based on flow-guided attentive-correlation-based feature bolstering (FAC-FB) module and recursive boosting (RB), in terms of multi-frame interpolation (MFI). The DeMFI-Net jointly performs deblurring and MFI where its baseline version performs feature-flow-based warping with FAC-FB module to obtain a sharp-interpolated frame as well to deblur two center-input frames. Moreover, its e…
Guided depth super-resolution is a practical task where a low-resolution and noisy input depth map is restored to a high-resolution version, with the help of a high-resolution RGB guide image. Existing methods usually view this task as a generalized guided filtering problem that relies on designing explicit filters and objective functions, or a dense regression problem that directly predicts the target image via deep neural networks. These methods suffer from either model capability or interpretability. Inspired by the recent progress in implicit neural representation, we propose to formulate …
DNN-based frame interpolation, which generates intermediate frames from two consecutive frames, is often dependent on model architectures with a large number of features, preventing their deployment on systems with limited resources, such as mobile devices. We present a compression-driven network design for frame interpolation that leverages model pruning through sparsity-inducing optimization to greatly reduce the model size while attaining higher performance. Concretely, we begin by compressing the recently proposed AdaCoF model and demonstrating that a 10 times compressed AdaCoF performs si…
Shiraz University of Technology（设拉子理工大学）；Persian Gulf University（波斯湾大学）；Yazd Branch, Islamic Azad University（伊斯兰阿扎德大学亚兹德分校）；Najafabad Branch, Islamic Azad University（伊斯兰阿扎德大学纳杰法巴德分校）
In interpretation of remote sensing images, it is possible that some images which are supplied by different sensors become incomprehensible. For better visual perception of these images, it is essential to operate series of pre-processing and elementary corrections and then operate a series of main processing steps for more precise analysis on the images. There are several approaches for processing which are depended on the type of remote sensing images. The discussed approach in this article, i.e. image fusion, is the use of natural colors of an optical image for adding color to a grayscale s…
ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Upsampling videos of human activity is an interesting yet challenging task with many potential applications ranging from gaming to entertainment and sports broadcasting. The main difficulty in synthesizing video frames in this setting stems from the highly complex and non-linear nature of human motion and the complex appearance and texture of the body. We propose to address these issues in a motion-guided frame-upsampling framework that is capable of producing realistic human motion and appearance. A novel motion model is trained to inference the non-linear skeletal motion between frames by le…
FB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imaging
Computer Aided Medical Procedures, Technical University of Munich（技术大学慕尼黑计算机辅助医学程序）；Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医学机器人研究所）；Munich Center for Machine Learning, Munich（慕尼黑机器学习中心）；School of Computing, Macquarie University（麦考瑞大学计算机学院）
MCICSAM: Monte Carlo-guided Interpolation Consistency Segment Anything Model for Semi-Supervised Prostate Zone Segmentation
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；University of Chicago（芝加哥大学）；University of Dundee（邓迪大学）；School of Health Management, China Medical University（中国医科大学健康管理学院）
Learning Event-guided Exposure-agnostic Video Frame Interpolation via Adaptive Feature Blending
School of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea（计算机学院，韩国科学技术院（KAIST），大田，韩国）；School of Electrical and Electronic Engineering, Nanyang Technological University (NTU), Singapore（电子与电气工程学院，南洋理工大学（NTU），新加坡）
Deep Biomechanically-Guided Interpolation for Keypoint-Based Brain Shift Registration
LASIGE, Faculdade de Ciências da Universidade de Lisboa（里斯本大学科学学院LASIGE中心）；CRUK Cambridge Centre, University of Cambridge（剑桥大学CRUK剑桥中心）；Department of Oncology, University of Cambridge（剑桥大学肿瘤学系）；Intelligent System for Medicine Laboratory (ISML), School of Mechanical Engineering, The University of Western Australia（西澳大学机械工程学院医学智能系统实验室）；MIND Team, Inria Saclay, Université Paris-Saclay, Palaiseau, France（法国巴黎萨克利大学MIND团队）；Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（索邦大学巴黎脑研究所-ICM）
Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames
AI Thrust, HKUST(GZ)（香港科技大学（广州）人工智能方向）；school of software Shandong University（山东大学软件学院）；School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（新加坡南洋理工大学电子与电气工程学院）
Temporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditions
Center for Humanoid Research, Korea Institute of Science and Technology（韩国科学技术研究院仿人机器人研究中心）；School of Electrical and Electronic Engineering, Yonsei University（延世大学电气与电子工程学院）
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；DAMO Academy, Alibaba Group（阿里巴巴集团 DAMO 院）；Hupan Lab（虎斑实验室）；Zhejiang University（浙江大学）
Example-guided image synthesis has been recently attempted to synthesize an image from a semantic label map and an exemplary image. In the task, the additional exemplary image serves to provide style guidance that controls the appearance of the synthesized output. Despite the controllability advantage, the previous models are designed on datasets with specific and roughly aligned objects. In this paper, we tackle a more challenging and general task, where the exemplar is an arbitrary scene image that is semantically unaligned to the given label map. To this end, we first propose a new Masked S…
Example-guided image synthesis has recently been attempted to synthesize an image from a semantic label map and an exemplary image. In the task, the additional exemplar image provides the style guidance that controls the appearance of the synthesized output. Despite the controllability advantage, the existing models are designed on datasets with specific and roughly aligned objects. In this paper, we tackle a more challenging and general task, where the exemplar is an arbitrary scene image that is semantically different from the given label map. To this end, we first propose a Masked Spatial-C…
UC San Diego（加州大学圣地亚哥分校）；ShanghaiTech University（上海科技大学）；National University of Singapore（新加坡国立大学）；Shanghai AI Laboratory（上海人工智能实验室）；Technical University of Munich（慕尼黑工业大学）
Depth-Guided Bundle Sampling for Efficient Generalizable Neural Radiance Field Reconstruction
Key Laboratory of Media Audio and Video (Communication University of China), Ministry of Education（媒体音频与视频重点实验室（中国传媒大学），教育部）；School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）
针对可泛化神经辐射场渲染计算量大、采样冗余的问题，提出深度引导的束采样策略，通过分组采样和自适应分配采样点，在DTU数据集上实现1.27 dB PSNR提升和47% FPS增加，并达到2倍渲染加速。
Motion-guided sparse correction enables expert-quality point tracking across diverse microscopy regimes
Department of Computer Science, Old Dominion University（奥德赛大学计算机科学系）；Department of Biology, Massachusetts Institute of Technology（麻省理工学院生物学系）；The Picower Institute for Learning and Memory, Massachusetts Institute of Technology（麻省理工学院学习与记忆研究所）；Department of Physics and Technology, UiT--The Arctic University of Norway（挪威北极大学物理与技术系）；Department of Physics, University of Oslo（奥斯陆大学物理系）；School of Data Science, Old Dominion University（奥德赛大学数据科学学院）；Department of Physics, Old Dominion University（奥德赛大学物理系）
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Guangzhou University（广州大学）
Sparse Gain Radio Map Reconstruction With Geometry Priors and Uncertainty-Guided Measurement Selection
National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China (UESTC)（电子科技大学通信抗干扰全国重点实验室）；University of Electronic Science and Technology of China (UESTC)（电子科技大学）；Department of Information Science Technology, University of Houston（休斯顿大学信息科学与技术系）；School of Science, Technology and Engineering, University of the Sunshine Coast（阳光海岸大学科学、技术与工程学院）；ZGC Institute of Ubiquitous-X Innovation and Applications（中关村泛在融合创新研究院）
HKUST(GZ)（香港科技大学（广州））；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；HKUST（香港科技大学）
Event cameras sense the intensity changes asynchronously and produce event streams with high dynamic range and low latency. This has inspired research endeavors utilizing events to guide the challenging video superresolution (VSR) task. In this paper, we make the first attempt to address a novel problem of achieving VSR at random scales by taking advantages of the high temporal resolution property of events. This is hampered by the difficulties of representing the spatial-temporal information of events when guiding VSR. To this end, we propose a novel framework that incorporates the spatial-te…
