Slow motion videos are becoming increasingly popular, but capturing high-resolution videos at extremely high frame rates requires professional high-speed cameras. To mitigate this problem, current techniques increase the frame rate of standard videos through frame interpolation by assuming linear object motion which is not valid in challenging cases. In this paper, we address this problem using two video streams as input; an auxiliary video with high frame rate and low spatial resolution, providing temporal information, in addition to the standard main video with low frame rate and high spatial resolution. We propose a two-stage deep learning system consisting of alignment and appearance estimation that reconstructs high resolution slow motion video from the hybrid video input. For alignment, we propose to compute flows between the missing frame and two existing frames of the main video by utilizing the content of the auxiliary video frames. For appearance estimation, we propose to combine the warped and auxiliary frames using a context and occlusion aware network. We train our model on synthetically generated hybrid videos and show high-quality results on a variety of test scenes. To demonstrate practicality, we show the performance of our system on two real dual camera setups with small baseline.
The University of Tokyo（东京大学）；Data61-CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）；Australian National University（澳大利亚国立大学）
This paper tackles high-dynamic-range (HDR) image reconstruction given only a single low-dynamic-range (LDR) image as input. While the existing methods focus on minimizing the mean-squared-error (MSE) between the target and reconstructed images, we minimize a hybrid loss that consists of perceptual and adversarial losses in addition to HDR-reconstruction loss. The reconstruction loss instead of MSE is more suitable for HDR since it puts more weight on both over- and under- exposed areas. It makes the reconstruction faithful to the input. Perceptual loss enables the networks to utilize knowledg…
Hybrid adiabatic quantum computing for tomographic image reconstruction -- opportunities and limitations
Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthineers GmbH（西门子医疗有限公司）；Siemens Medical Solutions USA, Inc（美国西门子医疗解决方案公司）
We propose a novel method to reconstruct the 3D shapes of transparent objects using hand-held captured images under natural light conditions. It combines the advantage of explicit mesh and multi-layer perceptron (MLP) network, a hybrid representation, to simplify the capture setting used in recent contributions. After obtaining an initial shape through the multi-view silhouettes, we introduce surface-based local MLPs to encode the vertex displacement field (VDF) for the reconstruction of surface details. The design of local MLPs allows to represent the VDF in a piece-wise manner using two laye…
South China University of Technology（华南理工大学）；JD Finance America Corporation（京东金融美国公司）；Sun Yat-sen University（中山大学）
In this paper, we propose $\text{HF}^2$-VAD, a Hybrid framework that integrates Flow reconstruction and Frame prediction seamlessly to handle Video Anomaly Detection. Firstly, we design the network of ML-MemAE-SC (Multi-Level Memory modules in an Autoencoder with Skip Connections) to memorize normal patterns for optical flow reconstruction so that abnormal events can be sensitively identified with larger flow reconstruction errors. More importantly, conditioned on the reconstructed flows, we then employ a Conditional Variational Autoencoder (CVAE), which captures the high correlation between v…
A DeepONet joint Neural Tangent Kernel Hybrid Framework for Physics-Informed Inverse Source Problems and Robust Image Reconstruction
Shenzhen MSU-BIT University（深圳MSU-BIT大学）；China Media Group（中国传媒集团）；School of Mathematics and Statistics, Beijing Institute of Technology（北京理工大学数学与统计学院）
Hybrid Learning: A Novel Combination of Self-Supervised and Supervised Learning for Joint MRI Reconstruction and Denoising in Low-Field MRI
New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）；NYU Tandon School of Engineering（纽约大学坦登工程学院）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）
Event sensors output a stream of asynchronous brightness changes (called ``events'') at a very high temporal rate. Previous works on recovering the lost intensity information from the event sensor data have heavily relied on the event stream, which makes the reconstructed images non-photorealistic and also susceptible to noise in the event stream. We propose to reconstruct photorealistic intensity images from a hybrid sensor consisting of a low frame rate conventional camera, which has the scene texture information, along with the event sensor. To accomplish our task, we warp the low frame rat…
University of Southern Denmark（南丹麦大学）；Newtec Engineering A/S（Newtec工程公司）；Mads Clausen Institute（马德斯·克劳森研究所）
We present a simple but novel hybrid approach to hyperspectral data cube reconstruction from computed tomography imaging spectrometry (CTIS) images that sequentially combines neural networks and the iterative Expectation Maximization (EM) algorithm. We train and test the ability of the method to reconstruct data cubes of $100\times100\times25$ and $100\times100\times100$ voxels, corresponding to 25 and 100 spectral channels, from simulated CTIS images generated by our CTIS simulator. The hybrid approach utilizes the inherent strength of the Convolutional Neural Network (CNN) with regard to noi…
ETH Zurich（苏黎世联邦理工学院）；University of Haifa（海法大学）；Microsoft（微软）；University of Amsterdam（阿姆斯特丹大学）
Estimating 3D hand meshes from RGB images robustly is a highly desirable task, made challenging due to the numerous degrees of freedom, and issues such as self similarity and occlusions. Previous methods generally either use parametric 3D hand models or follow a model-free approach. While the former can be considered more robust, e.g. to occlusions, they are less expressive. We propose a hybrid approach, utilizing a deep neural network and differential rendering based optimization to demonstrably achieve the best of both worlds. In addition, we explore Virtual Reality (VR) as an application. M…
MCQ-Scan；IMT Lille Douai（IMT里尔-杜埃国立高等工程师学校）；Univ. Lille（里尔大学）；CNRS UMR 9189 CRIStAL（法国国家科研中心UMR 9189 CRIStAL）
Recovering the 3D geometric structure of a face from a single input image is a challenging active research area in computer vision. In this paper, we present a novel method for reconstructing 3D heads from a single or multiple image(s) using a hybrid approach based on deep learning and geometric techniques. We propose an encoder-decoder network based on the U-net architecture and trained on synthetic data only. It predicts both pixel-wise normal vectors and landmarks maps from a single input photo. Landmarks are used for the pose computation and the initialization of the optimization problem, …
Diffusion Low Rank Hybrid Reconstruction for Sparse View Medical Imaging
Shenzhen MSU-BIT University（深圳MSU-BIT大学）；China Media Group（中国媒体集团）；School of Mathematics and Statistics, Beijing Institute of Technology（北京理工大学数学与统计学院）
DuDoUniNeXt: Dual-domain unified hybrid model for single and multi-contrast undersampled MRI reconstruction
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（中国科学技术大学生命科学与医学部生物医学工程学院）；Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advance Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院医学影像、机器人、分析计算与学习中心）；School of Electrical Engineering and Computer Science, The University of Queensland（昆士兰大学电气工程与计算机科学学院）
HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes
Fudan University（复旦大学）；School of Future Technology, Harbin Institute of Technology（哈尔滨工业大学未来技术学院）；Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）
High-Fidelity 3D Geometric Reconstruction of Pelvic Organs from MRI: A Hybrid Deep Learning and Iterative Optimization Approach
Institute of Medical Technology, Peking University Health Science Center, Peking University（北京大学医学部医学技术研究院，北京大学）；Biomedical Engineering Department, Institute of Advanced Clinical Medicine, Peking University（北京大学先进临床医学研究院生物医学工程系）；Department of Obstetrics and Gynecology, Peking University People’s Hospital（北京大学人民医院妇产科部）
H3R: Hybrid Multi-view Correspondence for Generalizable 3D Reconstruction
ReLER Lab, CCAI, Zhejiang University（浙江大学ReLER实验室，中国计算机协会，浙江大学）；The State Key Lab of Brain-Machine Intelligence, Zhejiang University（浙江大学脑机智能国家重点实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses
City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Caritas Institute of Higher Education（明爱专上学院）；Shanghai Jiao Tong University（上海交通大学）
Combining Microscopy Data and Metadata for Reconstruction of Cellular Traction Forces Using a Hybrid Vision Transformer-U-Net
Fakultät Physik, Technische Universität Dortmund（物理系，德意志理工大学多特蒙德分校）；Faculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München（物理系和纳米科学中心，路德维希-马克西米利安大学慕尼黑分校）；Department of Veterinary Sciences, Ludwig-Maximilians-Universität München（兽医学系，路德维希-马克西米利安大学慕尼黑分校）
In accelerated MRI reconstruction, the anatomy of a patient is recovered from a set of under-sampled and noisy measurements. Deep learning approaches have been proven to be successful in solving this ill-posed inverse problem and are capable of producing very high quality reconstructions. However, current architectures heavily rely on convolutions, that are content-independent and have difficulties modeling long-range dependencies in images. Recently, Transformers, the workhorse of contemporary natural language processing, have emerged as powerful building blocks for a multitude of vision task…
ShanghaiTech University（上海科技大学）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Photoacoustic imaging (PAI) is an emerging non-invasive imaging modality combining the advantages of deep ultrasound penetration and high optical contrast. Image reconstruction is an essential topic in PAI, which is unfortunately an ill-posed problem due to the complex and unknown optical/acoustic parameters in tissue. Conventional algorithms used in PAI (e.g., delay-and-sum) provide a fast solution while many artifacts remain, especially for linear array probe with limited-view issue. Convolutional neural network (CNN) has shown state-of-the-art results in computer vision, and more and more w…
Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition
Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所语音声学与内容理解重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
ViT-NeBLa: A Hybrid Vision Transformer and Neural Beer-Lambert Framework for Single-View 3D Reconstruction of Oral Anatomy from Panoramic Radiographs
Artificial Intelligence and Image Processing Lab., Department of Information and Communication Engineering, Sun Moon University, South Korea（人工智能与图像处理实验室，信息与通信工程系，顺 Moon 大学，韩国）；Department of Physics and Engineering Physics, Tulane University, New Orleans, LA 70118, USA（物理系与工程物理系，路易斯安那大学，新奥尔良，路易斯安那州 70118，美国）
