Monocular 3D object detection aims to locate objects in different scenes with just a single image. Due to the absence of depth information, several monocular 3D detection techniques have emerged that rely on auxiliary depth maps from the depth estimation task. There are multiple approaches to understanding the representation of depth maps, including treating them as pseudo-LiDAR point clouds, leveraging implicit end-to-end learning of depth information, or considering them as an image input. However, these methods have certain drawbacks, such as their reliance on the accuracy of estimated depth maps and suboptimal utilization of depth maps due to their image-based nature. While LiDAR-based methods and convolutional neural networks (CNNs) can be utilized for pseudo point clouds and depth maps, respectively, it is always an alternative. In this paper, we propose a framework named the Adaptive Distance Interval Separation Network (ADISN) that adopts a novel perspective on understanding depth maps, as a form that lies between LiDAR and images. We utilize an adaptive separation approach that partitions the depth map into various subgraphs based on distance and treats each of these subgraphs as an individual image for feature extraction. After adaptive separations, each subgraph solely contains pixels within a learned interval range. If there is a truncated object within this range, an evident curved edge will appear, which we can leverage for texture extraction using CNNs to obtain rich depth information in pixels. Meanwhile, to mitigate the inaccuracy of depth estimation, we designed an uncertainty module. To take advantage of both images and depth maps, we use different branches to learn localization detection tasks and appearance tasks separately.
Recognizing and localizing objects in the 3D space is a crucial ability for an AI agent to perceive its surrounding environment. While significant progress has been achieved with expensive LiDAR point clouds, it poses a great challenge for 3D object detection given only a monocular image. While there exist different alternatives for tackling this problem, it is found that they are either equipped with heavy networks to fuse RGB and depth information or empirically ineffective to process millions of pseudo-LiDAR points. With in-depth examination, we realize that these limitations are rooted in …
Monocular 3D object detection is a key problem for autonomous vehicles, as it provides a solution with simple configuration compared to typical multi-sensor systems. The main challenge in monocular 3D detection lies in accurately predicting object depth, which must be inferred from object and scene cues due to the lack of direct range measurement. Many methods attempt to directly estimate depth to assist in 3D detection, but show limited performance as a result of depth inaccuracy. Our proposed solution, Categorical Depth Distribution Network (CaDDN), uses a predicted categorical depth distrib…
SPAN: Spatial-Projection Alignment for Monocular 3D Object Detection
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；School of Mathematics and Statistics, University of Glasgow（格拉斯哥大学数学与统计学学院）；Basic Algorithm Center, PCG, Tencent（腾讯计算机系统有限公司基础算法中心）
We tackle the fundamentally ill-posed problem of 3D human localization from monocular RGB images. Driven by the limitation of neural networks outputting point estimates, we address the ambiguity in the task by predicting confidence intervals through a loss function based on the Laplace distribution. Our architecture is a light-weight feed-forward neural network that predicts 3D locations and corresponding confidence intervals given 2D human poses. The design is particularly well suited for small training data, cross-dataset generalization, and real-time applications. Our experiments show that …
University of Adelaide（阿德莱德大学）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；Monash University（莫纳什大学）
Semantic aware reconstruction is more advantageous than geometric-only reconstruction for future robotic and AR/VR applications because it represents not only where things are, but also what things are. Object-centric mapping is a task to build an object-level reconstruction where objects are separate and meaningful entities that convey both geometry and semantic information. In this paper, we present MOLTR, a solution to object-centric mapping using only monocular image sequences and camera poses. It is able to localise, track, and reconstruct multiple objects in an online fashion when an RGB…
The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）
3D interacting hand reconstruction is essential to facilitate human-machine interaction and human behaviors understanding. Previous works in this field either rely on auxiliary inputs such as depth images or they can only handle a single hand if monocular single RGB images are used. Single-hand methods tend to generate collided hand meshes, when applied to closely interacting hands, since they cannot model the interactions between two hands explicitly. In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images. Our method can generate 3D hand …
Nanjing University of Science and Technology（南京理工大学）；Renmin University of China（中国人民大学）；Nreal（Nreal（太若科技））
Monocular 3D object detection is a fundamental but very important task to many applications including autonomous driving, robotic grasping and augmented reality. Existing leading methods tend to estimate the depth of the input image first, and detect the 3D object based on point cloud. This routine suffers from the inherent gap between depth estimation and object detection. Besides, the prediction error accumulation would also affect the performance. In this paper, a novel method named MonoSIM is proposed. The insight behind introducing MonoSIM is that we propose to simulate the feature learni…
ARDepth: Auto-regressive Monocular Depth Estimation with Progressive Visual Conditioning
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Shenzhen Loop Area Institute（深圳河套学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；Baidu Inc.（百度公司）；LightSpeed Studios, Tencent America（美国光速工作室，腾讯）；School of Computer Science and Artificial Intelligence, Lanzhou University of Technology（兰州理工大学计算机科学与人工智能学院）
Neural networks have shown great success in extracting geometric information from color images. Especially, monocular depth estimation networks are increasingly reliable in real-world scenes. In this work we investigate the applicability of such monocular depth estimation networks to semi-transparent volume rendered images. As depth is notoriously difficult to define in a volumetric scene without clearly defined surfaces, we consider different depth computations that have emerged in practice, and compare state-of-the-art monocular depth estimation approaches for these different interpretations…
CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention
Harbin Engineering University（哈尔滨工程大学）；Key Laboratory of Advanced Marine Communication and Information Technology（先进海洋通信与信息技术重点实验室）；University of Toronto（多伦多大学）；Kanagawa University（神奈川大学）
Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Huawei European Research Center（华为欧洲研究中心）；Technische Universität München（慕尼黑工业大学）
Depth information is essential for on-board perception in autonomous driving and driver assistance. Monocular depth estimation (MDE) is very appealing since it allows for appearance and depth being on direct pixelwise correspondence without further calibration. Best MDE models are based on Convolutional Neural Networks (CNNs) trained in a supervised manner, i.e., assuming pixelwise ground truth (GT). Usually, this GT is acquired at training time through a calibrated multi-modal suite of sensors. However, also using only a monocular system at training time is cheaper and more scalable. This is …
National Engineering Laboratory for Brain-inspired Intelligence Technology and Application（类脑智能技术与应用国家工程实验室）；University of Science and Technology of China（中国科学技术大学）
Monocular depth estimation is an essential task for scene understanding. The underlying structure of objects and stuff in a complex scene is critical to recovering accurate and visually-pleasing depth maps. Global structure conveys scene layouts, while local structure reflects shape details. Recently developed approaches based on convolutional neural networks (CNNs) significantly improve the performance of depth estimation. However, few of them take into account multi-scale structures in complex scenes. In this paper, we propose a Structure-Aware Residual Pyramid Network (SARPN) to exploit mul…
Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation of Chinese Academy of Sciences（中国科学院自动化研究所）；School of Information Science and Technology, Shijiazhuang Tiedao University（石家庄铁道大学信息科学与技术学院）；D-Robotics（杭州宇树科技有限公司）
I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions
ShanghaiTech University（上海科技大学）；Shanghai Advanced Research Institute, Chinese Academy of Sciences（中国科学院上海高等研究院）；University of Chinese Academy of Sciences（中国科学院大学）
MonoRelief V2: Leveraging Real Data for High-Fidelity Monocular Relief Recovery
School of Mechanical Engineering, Qilu University of Technology (Shandong Academy of Sciences)（机械工程学院、齐鲁工业大学（山东科学院））；School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（计算机科学与技术学院、南京航空航天大学）；School of Computer Science and Technology, Shandong University of Finance and Economics（计算机科学与技术学院、山东财经大学）；School of Computer Science and Technology, Shandong University（计算机科学与技术学院、山东大学）
Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks. Classical cameras suffer from motion blur in such scenarios rendering them effete. On the contrary, event cameras, because of their high temporal resolution and lack of motion blur, are tailor-made for this problem. We present an approach for monocular multi-motion segmentation, which combines bottom-up feature tracking and top-down motion compensation into a unified pipeline, which is the first of its kind to our knowledge. Using the events within a time-interval, our method segments…
MapAnything: Evaluating Monocular Metric Depth Models for 3D Urban Asset Localization
ScaDS.AI (University of Leipzig)（ScaDS.AI（莱比锡大学））；University of Leipzig（莱比锡大学）；Kühne Logistics University（库赫内物流大学）；Wrocław University of Economics（沃拉夫经济大学）
Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimation
Princeton University（普林斯顿大学）；Springer Heidelberg（斯普林格海德堡）；ABC Institute（ABC研究所）；Aerospace Information Research Institute Chinese Academy of Sciences（中国科学院航天信息研究所）
RaGS: Unleashing 3D Gaussian Splatting from 4D Radar and Monocular Cues for 3D Object Detection
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Automotive Studies, Tongji University（同济大学汽车学院）；Momoni AI, Gothenburg, Sweden（Momoni AI（瑞典哥德堡））；College of Energy Engineering, Zhejiang University（浙江大学能源工程学院）
Google Research（谷歌研究院）；University of Maryland（马里兰大学）
Immersive maps such as Google Street View and Bing Streetside provide true-to-life views with a massive collection of panoramas. However, these panoramas are only available at sparse intervals along the path they are taken, resulting in visual discontinuities during navigation. Prior art in view synthesis is usually built upon a set of perspective images, a pair of stereoscopic images, or a monocular image, but barely examines wide-baseline panoramas, which are widely adopted in commercial platforms to optimize bandwidth and storage usage. In this paper, we leverage the unique characteristics …
Aerial Multi-View Stereo via Adaptive Depth Range Inference and Normal Cues
Department of Information Science and Technology, Ocean University of China（信息科学与技术学院，中国海洋大学）；School of Computing and Mathematical Sciences, University of Leicester（计算与数学科学学院，莱斯特大学）；Department of Electrical and Computer Engineering, Mississippi State University（电气与计算机工程学院，密苏里州立大学）
Université catholique de Louvain（鲁汶天主教大学）；Keemotion（基莫申公司）
This paper assumes prior detections of multiple targets at each time instant, and uses a graph-based approach to connect those detections across time, based on their position and appearance estimates. In contrast to most earlier works in the field, our framework has been designed to exploit the appearance features, even when they are only sporadically available, or affected by a non-stationary noise, along the sequence of detections. This is done by implementing an iterative hypothesis testing strategy to progressively aggregate the detections into short trajectories, named tracklets. Specific…
Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Renmin University of China（中国人民大学）；Chinese Academy of Sciences（中国科学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Peking University（北京大学）；Shanghai Jiaotong University（上海交通大学）
Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completion
Dalian University of Technology（大连理工大学）；Beihang University（北京航空航天大学）；College of Computer Science and Engineering, Shandong University of Science and Technology（山东科技大学计算机科学与工程学院）
University of California, San Diego（加州大学圣地亚哥分校）；Nuro Inc.（Nuro公司）；Columbia University（哥伦比亚大学）；Scale AI
We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geometry from multi-view images. Previous learning-based MVS methods estimate per-view depth using plane sweep volumes with a fixed depth hypothesis at each plane; this generally requires densely sampled planes for desired accuracy, and it is very hard to achieve high-resolution depth. In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to t…
