Monocular metric depth estimation (MMDE) is a crucial task to solve for indoor scene reconstruction on edge devices. Despite this importance, existing models are sensitive to factors such as boundary frequency of objects in the scene and scene complexity, failing to fully capture many indoor scenes. In this work, we propose to close this gap through the task of monocular metric depth refinement (MMDR) by leveraging state-of-the-art MMDE models. MultiDepth proposes a solution by taking samples of the image along with the initial depth map prediction made by a pre-trained MMDE model. Compared to existing iterative depth refinement techniques, MultiDepth does not employ normal map prediction as part of its architecture, effectively lowering the model size and computation overhead while outputting impactful changes from refining iterations. MultiDepth implements a lightweight encoder-decoder architecture for the refinement network, processing multiple samples from the given image, including segmentation masking. We evaluate MultiDepth on four datasets and compare them to state-of-the-art methods to demonstrate its effective refinement with minimal overhead, displaying accuracy improvement upward of 45%.
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
Self-supervised monocular depth estimation has received much attention recently in computer vision. Most of the existing works in literature aggregate multi-scale features for depth prediction via either straightforward concatenation or element-wise addition, however, such feature aggregation operations generally neglect the contextual consistency between multi-scale features. Addressing this problem, we propose the Self-Distilled Feature Aggregation (SDFA) module for simultaneously aggregating a pair of low-scale and high-scale features and maintaining their contextual consistency. The SDFA e…
MonoMobility: Zero-Shot 3D Mobility Analysis from Monocular Videos
National University of Defense Technology（国防科技大学）；Sun Yat-sen University（中山大学）；College of Computer and Information Science, Southwest University（西南大学计算机与信息科学学院）
Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformer
School of Mechanical Engineering（机械工程学院）；State Key Laboratory of Intelligent Optimized Manufacturing in Mining & Metallurgy Process（矿冶过程智能优化制造国家重点实验室）；Hubei Key Laboratory of Modern Manufacturing Quality Engineering（现代制造质量工程湖北省重点实验室）；School of Mechanical Science and Engineering（机械科学与工程学院）；National Key Laboratory for Novel Software Technology（计算机软件新技术国家重点实验室）
University of Electronic Science and Technology of China（电子科技大学）
The technology for Visual Odometry (VO) that estimates the position and orientation of the moving object through analyzing the image sequences captured by on-board cameras, has been well investigated with the rising interest in autonomous driving. This paper studies monocular VO from the perspective of Deep Learning (DL). Unlike most current learning-based methods, our approach, called DeepAVO, is established on the intuition that features contribute discriminately to different motion patterns. Specifically, we present a novel four-branch network to learn the rotation and translation by levera…
VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction
Renmin University of China（中国人民大学）；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）
Baillie Gifford Pandemic Science Hub（巴利尔·吉福德疫情科学中心）；Institute of Regeneration and Repair（再生与修复研究所）；Heriot-Watt University（赫罗特-瓦特大学）；Institute of Photonics and Quantum Science（光子与量子科学研究所）
Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation
Nanjing University of Science and Technology（南京理工大学）；State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery（先进施工机械智能制造国家重点实验室）；Department of Electrical and Computer Engineering, Sungkyunkwan University（成均馆大学电子与计算机工程系）；University of Electronic Science and Technology of China（电子科技大学）；Zhejiang University（浙江大学）
Incremental Dense Reconstruction from Monocular Video with Guided Sparse Feature Volume Fusion
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Delaware（特拉华大学）；Imperial College London（帝国理工学院）；University of Toronto（多伦多大学）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器学习研究所）
MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction
ETH Zürich（苏黎世联邦理工学院）；University of Tübingen, Tübingen AI Center（图宾根大学图宾根人工智能中心）；Max Planck Institute for Intelligent Systems, Tübingen（智能系统马克斯·普朗克研究所）
Sparis: Neural Implicit Surface Reconstruction of Indoor Scenes from Sparse Views
Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University（北京信息科学与技术国家研究中心（BNRist），清华大学）；School of Software, Tsinghua University（清华大学软件学院）
Yale-NUS College（耶鲁-新加坡国大学院）；National University of Singapore（新加坡国立大学）；CtrsVision (USA)（CtrsVision（美国））
Monocular 3D human pose estimation has made progress in recent years. Most of the methods focus on single persons, which estimate the poses in the person-centric coordinates, i.e., the coordinates based on the center of the target person. Hence, these methods are inapplicable for multi-person 3D pose estimation, where the absolute coordinates (e.g., the camera coordinates) are required. Moreover, multi-person pose estimation is more challenging than single pose estimation, due to inter-person occlusion and close human interactions. Existing top-down multi-person methods rely on human detection…
Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of depth data. In this paper, we introduce a new method to fuse disparity maps from different sources, while incorporating supplementary information (intensity, gradient, etc.) into a refiner network to better refine raw disparity inputs. A discriminator network classifies disparities at diffe…
