While current high-resolution depth estimation methods achieve strong results, they often suffer from computational inefficiencies due to reliance on heavyweight models and multiple inference steps, increasing inference time. To address this, we introduce PatchRefiner V2 (PRV2), which replaces heavy refiner models with lightweight encoders. This reduces model size and inference time but introduces noisy features. To overcome this, we propose a Coarse-to-Fine (C2F) module with a Guided Denoising Unit for refining and denoising the refiner features and a Noisy Pretraining strategy to pretrain the refiner branch to fully exploit the potential of the lightweight refiner branch. Additionally, we introduce a Scale-and-Shift Invariant Gradient Matching (SSIGM) loss to enhance synthetic-to-real domain transfer. PRV2 outperforms state-of-the-art depth estimation methods on UnrealStereo4K in both accuracy and speed, using fewer parameters and faster inference. It also shows improved depth boundary delineation on real-world datasets like CityScape, ScanNet++, and KITTI, demonstrating its versatility across domains.
Monocular depth estimation can play an important role in addressing the issue of deriving scene geometry from 2D images. It has been used in a variety of industries, including robots, self-driving cars, scene comprehension, 3D reconstructions, and others. The goal of our method is to create a lightweight machine-learning model in order to predict the depth value of each pixel given only a single RGB image as input with the Unet structure of the image segmentation network. We use the NYU Depth V2 dataset to test the structure and compare the result with other methods. The proposed method achiev…
University of Oulu（奥卢大学）；Czech Technical University in Prague（布拉格捷克理工大学）；Tampere University（坦佩雷大学）
In this paper, we propose enhancing monocular depth estimation by adding 3D points as depth guidance. Unlike existing depth completion methods, our approach performs well on extremely sparse and unevenly distributed point clouds, which makes it agnostic to the source of the 3D points. We achieve this by introducing a novel multi-scale 3D point fusion network that is both lightweight and efficient. We demonstrate its versatility on two different depth estimation problems where the 3D points have been acquired with conventional structure-from-motion and LiDAR. In both cases, our network performs…
Self-supervised monocular depth estimation that does not require ground truth for training has attracted attention in recent years. It is of high interest to design lightweight but effective models so that they can be deployed on edge devices. Many existing architectures benefit from using heavier backbones at the expense of model sizes. This paper achieves comparable results with a lightweight architecture. Specifically, the efficient combination of CNNs and Transformers is investigated, and a hybrid architecture called Lite-Mono is presented. A Consecutive Dilated Convolutions (CDC) module a…
PuriLight: A Lightweight Shuffle and Purification Framework for Monocular Depth Estimation
Department of Mathematics, Hefei University of Technology, Hefei, China（数学系，合肥工业大学）；Department of Computer Science and Technology, University of Science and Technology of China, Hefei, China（计算机科学与技术系，中国科学技术大学）
Convolutional neural networks (CNNs) have emerged as the state-of-the-art in multiple vision tasks including depth estimation. However, memory and computing power requirements remain as challenges to be tackled in these models. Monocular depth estimation has significant use in robotics and virtual reality that requires deployment on low-end devices. Training a small model from scratch results in a significant drop in accuracy and it does not benefit from pre-trained large models. Motivated by the literature of model pruning, we propose a lightweight monocular depth model obtained from a large …
Shenzhen University（深圳大学）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人学会）
Predicting depth from a single image is an attractive research topic since it provides one more dimension of information to enable machines to better perceive the world. Recently, deep learning has emerged as an effective approach to monocular depth estimation. As obtaining labeled data is costly, there is a recent trend to move from supervised learning to unsupervised learning to obtain monocular depth. However, most unsupervised learning methods capable of achieving high depth prediction accuracy will require a deep network architecture which will be too heavy and complex to run on embedded …
We propose an accurate and lightweight convolutional neural network for stereo estimation with depth completion. We name this method fully-convolutional deformable similarity network with depth completion (FCDSN-DC). This method extends FC-DCNN by improving the feature extractor, adding a network structure for training highly accurate similarity functions and a network structure for filling inconsistent disparity estimates. The whole method consists of three parts. The first part consists of fully-convolutional densely connected layers that computes expressive features of rectified image pairs…
Boosting Light-Weight Depth Estimation Via Knowledge Distillation
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；South China Normal University（华南师范大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Zhejiang University（浙江大学）；Tohoku University（东北大学（日本））
CCDepth: A Lightweight Self-supervised Depth Estimation Network with Enhanced Interpretability
College of Information Science and Engineering, China University of Petroleum (Beijing)（中国石油大学（北京）信息科学与工程学院）；The University of Hong Kong（香港大学）；Tsinghua University（清华大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
University of New South Wales（新南威尔士大学）；Ocean University of China（中国海洋大学）
Monocular depth estimation is an important task that can be applied to many robotic applications. Existing methods focus on improving depth estimation accuracy via training increasingly deeper and wider networks, however these suffer from large computational complexity. Recent studies found that edge information are important cues for convolutional neural networks (CNNs) to estimate depth. Inspired by the above observations, we present a novel lightweight Edge Guided Depth Estimation Network (EGD-Net) in this study. In particular, we start out with a lightweight encoder-decoder architecture an…
University of Duisburg-Essen（杜伊斯堡-埃森大学）；Ulm University（乌尔姆大学）；Technical University of Denmark（丹麦技术大学）
We present a lightweight encoder-decoder architecture for monocular depth estimation, specifically designed for embedded platforms. Our main contribution is the Guided Upsampling Block (GUB) for building the decoder of our model. Motivated by the concept of guided image filtering, GUB relies on the image to guide the decoder on upsampling the feature representation and the depth map reconstruction, achieving high resolution results with fine-grained details. Based on multiple GUBs, our model outperforms the related methods on the NYU Depth V2 dataset in terms of accuracy while delivering up to…
Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model
School of AI, Shanghai Jiao Tong University（上海交通大学人工智能学院）；King Abdullah University of Science and Technology（卡塔尔国王 Abdullah 大学科学与技术大学）；Nanyang Technological University（南洋理工大学）；SJTU-Quic Robot Joint Lab（上海交通大学-Quick 机器人联合实验室）
This paper presents a novel method, MaskMVS, to solve depth estimation for unstructured multi-view image-pose pairs. In the plane-sweep procedure, the depth planes are sampled by histogram matching that ensures covering the depth range of interest. Unlike other plane-sweep methods, we do not rely on a cost metric to explicitly build the cost volume, but instead infer a multiplane mask representation which regularizes the learning. Compared to many previous approaches, we show that our method is lightweight and generalizes well without requiring excessive training. We outperform the current sta…
University of California, Irvine（加州大学欧文分校）；Trifo, Inc.（追觅科技（美国）有限公司）
In this paper, we propose an end-to-end deep learning network named 3dDepthNet, which produces an accurate dense depth image from a single pair of sparse LiDAR depth and color image for robotics and autonomous driving tasks. Based on the dimensional nature of depth images, our network offers a novel 3D-to-2D coarse-to-fine dual densification design that is both accurate and lightweight. Depth densification is first performed in 3D space via point cloud completion, followed by a specially designed encoder-decoder structure that utilizes the projected dense depth from 3D completion and the origi…
Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
School of Engineering Science, Simon Fraser University（西蒙弗雷泽大学工程科学学院）；School of Information Science and Technology, Eastern Institute of Technology（东方理工学院信息科学与技术学院）
RTS-Mono: A Real-Time Self-Supervised Monocular Depth Estimation Method for Real-World Deployment
Shaanxi Joint Laboratory of Artificial Intelligence and the School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology（陕西人工智能联合实验室和陕西科技大学电子信息与人工智能学院）；School of Electronics And Information, Northwestern Polytechnical University（电子信息学院，西北工业大学）
University of Oulu（奥卢大学）；Czech Technical University（捷克技术大学）；Tampere University（坦佩雷大学）
Deep neural networks have recently thrived on single image depth estimation. That being said, current developments on this topic highlight an apparent compromise between accuracy and network size. This work proposes an accurate and lightweight framework for monocular depth estimation based on a self-attention mechanism stemming from salient point detection. Specifically, we utilize a sparse set of keypoints to train a FuSaNet model that consists of two major components: Fusion-Net and Saliency-Net. In addition, we introduce a normalized Hessian loss term invariant to scaling and shear along th…
University of Alicante（阿利坎特大学）；ETH Zurich（苏黎世联邦理工学院）；Microsoft（微软）；University of Amsterdam（阿姆斯特丹大学）
Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation. In particular, we propose a novel training method split in three main steps. First, the prediction results of a monocular depth network are warped to an additional view point. Second, we apply an additional image synthesis network, which corrects and improves the quality of the warped RGB image. The output of this network is required to look as similar as possible to the ground-truth view by minimizing the pixel-wise RGB reconstruction error. Third, we reapply the same m…
Depth estimation from a single image is an important task that can be applied to various fields in computer vision, and has grown rapidly with the development of convolutional neural networks. In this paper, we propose a novel structure and training strategy for monocular depth estimation to further improve the prediction accuracy of the network. We deploy a hierarchical transformer encoder to capture and convey the global context, and design a lightweight yet powerful decoder to generate an estimated depth map while considering local connectivity. By constructing connected paths between multi…
College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究所）；National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University（西安交通大学人机混合增强智能国家重点实验室）
Monocular depth estimation is a challenging problem on which deep neural networks have demonstrated great potential. However, depth maps predicted by existing deep models usually lack fine-grained details due to the convolution operations and the down-samplings in networks. We find that increasing input resolution is helpful to preserve more local details while the estimation at low resolution is more accurate globally. Therefore, we propose a novel depth map fusion module to combine the advantages of estimations with multi-resolution inputs. Instead of merging the low- and high-resolution est…
Depth as Points: Center Point-based Depth Estimation
School of Mechanical Engineering, Nanjing University of Science and Technology（机械工程学院，南京理工大学）；School of Cyber Science and Engineering, Nanjing University of Science and Technology（网络科学与工程学院，南京理工大学）；School of Computer Science , Wuhan University（计算机科学学院，武汉大学）
ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera
State Key Laboratory of Industrial Control Technology, Institute of Cyber Systems and Control, Zhejiang University, Hangzhou, China（浙江大学工业控制技术状态重点实验室，系统与控制研究所，杭州，中国）；Huzhou Institute of Zhejiang University, Huzhou, China（浙江大学湖州研究院，湖州，中国）；The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳））
Autonomous navigation of drones using computer vision has achieved promising performance. Nano-sized drones based on edge computing platforms are lightweight, flexible, and cheap, thus suitable for exploring narrow spaces. However, due to their extremely limited computing power and storage, vision algorithms designed for high-performance GPU platforms cannot be used for nano drones. To address this issue this paper presents a lightweight CNN depth estimation network deployed on nano drones for obstacle avoidance. Inspired by Knowledge Distillation (KD), a Channel-Aware Distillation Transformer…
Zhejiang University（浙江大学）；Fuxi AI Lab, NetEase（网易伏羲人工智能实验室）
Self-supervised learning shows great potential in monoculardepth estimation, using image sequences as the only source ofsupervision. Although people try to use the high-resolutionimage for depth estimation, the accuracy of prediction hasnot been significantly improved. In this work, we find thecore reason comes from the inaccurate depth estimation inlarge gradient regions, making the bilinear interpolation er-ror gradually disappear as the resolution increases. To obtainmore accurate depth estimation in large gradient regions, itis necessary to obtain high-resolution features with spatialand s…
Thermal Image Refinement with Depth Estimation using Recurrent Networks for Monocular ORB-SLAM3
Automatic Control Group (RAT), Paderborn University（自动控制组（RAT），波恩大学）；Department of Electrical and Computer Engineering, Aarhus University（电气与计算机工程系，阿arhus大学）；Upteko ApS（Upteko公司）
Beihang University（北京航空航天大学）；Peking University（北京大学）；Institute for Infocomm Research（信息通信研究院）
Dense depth completion is essential for autonomous systems and 3D reconstruction. In this paper, a lightweight yet efficient network (S\&CNet) is proposed to obtain a good trade-off between efficiency and accuracy for the dense depth completion. A dual-stream attention module (S\&C enhancer) is introduced to measure both spatial-wise and the channel-wise global-range relationship of extracted features so as to improve the performance. A coarse-to-fine network is designed and the proposed S\&C enhancer is plugged into the coarse estimation network between its encoder and decoder network. Experi…
UNIST（蔚山科学技术院）；Seoul National University（首尔国立大学）；KAIST（韩国科学技术院）；ETRI（韩国电子通信研究院）
We introduce Repetition-Reduction network (RRNet) for resource-constrained depth estimation, offering significantly improved efficiency in terms of computation, memory and energy consumption. The proposed method is based on repetition-reduction (RR) blocks. The RR blocks consist of the set of repeated convolutions and the residual connection layer that take place of the pointwise reduction layer with linear connection to the decoder. The RRNet help reduce memory usage and power consumption in the residual connections to the decoder layers. RRNet consumes approximately 3.84 times less energy an…
PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；Zhejiang Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing（浙江省多模态通信网络与智能信息处理重点实验室）
BoRe-Depth: Self-supervised Monocular Depth Estimation with Boundary Refinement for Embedded Systems
School of Mechatronical Engineering, Beijing Institute of Technology（机械电子工程学院，北京理工大学）；Yangtze Delta Region Academy of Beijing Institute of Technology Jiaxing, China（北京理工大学嘉兴 Yangtze Delta Region 学院）
Depth prediction is fundamental for many useful applications on computer vision and robotic systems. On mobile phones, the performance of some useful applications such as augmented reality, autofocus and so on could be enhanced by accurate depth prediction. In this work, an efficient fully convolutional network architecture for depth prediction has been proposed, which uses RegNetY 06 as the encoder and split-concatenate shuffle blocks as decoder. At the same time, an appropriate combination of data augmentation, hyper-parameters and loss functions to efficiently train the lightweight network …
Peking University（北京大学）；Advanced Institute of Information Technology, Peking University（北京大学信息工程高等研究院）；Information Technology R&D Innovation Center of Peking University（北京大学信息技术研发创新中心）
We present two versatile methods to generally enhance self-supervised monocular depth estimation (MDE) models. The high generalizability of our methods is achieved by solving the fundamental and ubiquitous problems in photometric loss function. In particular, from the perspective of spatial frequency, we first propose Ambiguity-Masking to suppress the incorrect supervision under photometric loss at specific object boundaries, the cause of which could be traced to pixel-level ambiguity. Second, we present a novel frequency-adaptive Gaussian low-pass filter, designed to robustify the photometric…
