Real-time 3D reconstruction from RGB-D sensor data plays an important role in many robotic applications, such as object modeling and mapping. The popular method of fusing depth information into a truncated signed distance function (TSDF) and applying the marching cubes algorithm for mesh extraction has severe issues with thin structures: not only does it lead to loss of accuracy, but it can generate completely wrong surfaces. To address this, we propose the directional TSDF - a novel representation that stores opposite surfaces separate from each other. The marching cubes algorithm is modified accordingly to retrieve a coherent mesh representation. We further increase the accuracy by using surface gradient-based ray casting for fusing new measurements. We show that our method outperforms state-of-the-art TSDF reconstruction algorithms in mesh accuracy.
Dense real-time tracking and mapping from RGB-D images is an important tool for many robotic applications, such as navigation or grasping. The recently presented Directional Truncated Signed Distance Function (DTSDF) is an augmentation of the regular TSDF and shows potential for more coherent maps and improved tracking performance. In this work, we present methods for rendering depth- and color maps from the DTSDF, making it a true drop-in replacement for the regular TSDF in established trackers. We evaluate and show, that our method increases re-usability of mapped scenes. Furthermore, we add…
Dense real-time tracking and mapping from RGB-D images is an important tool for many robotic applications, such as navigation and manipulation. The recently presented Directional Truncated Signed Distance Function (DTSDF) is an augmentation of the regular TSDF that shows potential for more coherent maps and improved tracking performance. In this work, we present methods for rendering depth- and color images from the DTSDF, making it a true drop-in replacement for the regular TSDF in established trackers. We evaluate the algorithm on well-established datasets and observe that our method improve…
The University of Sydney（悉尼大学）；Massachusetts Institute of Technology（麻省理工学院）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Beijing Institute of Technology（北京理工大学）；The University of New South Wales（新南威尔士大学）
3D reconstruction from images has wide applications in Virtual Reality and Automatic Driving, where the precision requirement is very high. Ground-breaking research in the neural radiance field (NeRF) by utilizing Multi-Layer Perceptions has dramatically improved the representation quality of 3D objects. Some later studies improved NeRF by building truncated signed distance fields (TSDFs) but still suffer from the problem of blurred surfaces in 3D reconstruction. In this work, this surface ambiguity is addressed by proposing a novel way of 3D shape representation, OmniNeRF. It is based on trai…
DDM: A Metric for Comparing 3D Shapes Using Directional Distance Fields
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；School of Precision Instruments and Opto-Electronic Engineering, Tianjin University（天津大学精密仪器与光电工程学院）；Department of Electronic Engineering, Shanghai Jiao Tong University（上海交通大学电子工程系）；Department of Computer Science & Engineering, Texas A& M University（德克萨斯大学阿姆斯特朗分校计算机科学与工程系）
Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals
Department of Electrical and Computer Engineering, University of California San Diego（加州大学圣地亚哥分校电气与计算机工程系）；Brain Corporation（Brain公司）；Robotics Department, University of Michigan（密歇根大学机器人系）
ParkingTwin: Training-Free Streaming 3D Reconstruction for Parking-Lot Digital Twins
School of Information and Communication Engineering, University of Electronic Science and Technology of China（信息与通信工程学院，电子科学与技术大学）；Department of Communications and Networking, Xi'an Jiaotong-Liverpool University（通信与网络系，西安交通大学利物浦大学）；School of Electrical Engineering, Hebei University of Technology（电气工程学院，河北工业大学）；Advanced Networking Lab., Department of Electrical and Computer Engineering, New Jersey Institute of Technology（高级网络实验室，电气与计算机工程系，新泽西理工学院）
3D Gaussian Splatting against Moving Objects for High-Fidelity Street Scene Reconstruction
Ontario Knowledge Intelligence Centre, Canada（加拿大安大略知识智能中心）；Canadian Institute of Interscience, Canada（加拿大跨学科科学研究所）；DeepCox Intelligence, Canada（加拿大DeepCox智能科技公司）
Carnegie Mellon University（卡内基梅隆大学）；University of Toronto（多伦多大学）；LIX, École Polytechnique（巴黎综合理工学院计算机科学实验室）
We introduce a new general-purpose approach to deep learning on 3D surfaces, based on the insight that a simple diffusion layer is highly effective for spatial communication. The resulting networks are automatically robust to changes in resolution and sampling of a surface -- a basic property which is crucial for practical applications. Our networks can be discretized on various geometric representations such as triangle meshes or point clouds, and can even be trained on one representation then applied to another. We optimize the spatial support of diffusion as a continuous network parameter r…
ExMesh: Explicit Mesh Reconstruction with Topology Adaptation
University of Science and Technology of China（中国科学技术大学）；National Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory（国家空间科学探测重点实验室，深空探测实验室）
ShanghaiTech University（上海科技大学）；NeuDim Digital Technology (Shanghai) Co.,Ltd.（纽迪姆数字科技（上海）有限公司）；DGene Digital Technology Co., Ltd.（帝晶数字科技有限公司）；KU Leuven（鲁汶大学）
We have recently seen tremendous progress in the neural advances for photo-real human modeling and rendering. However, it's still challenging to integrate them into an existing mesh-based pipeline for downstream applications. In this paper, we present a comprehensive neural approach for high-quality reconstruction, compression, and rendering of human performances from dense multi-view videos. Our core intuition is to bridge the traditional animated mesh workflow with a new class of highly efficient neural techniques. We first introduce a neural surface reconstructor for high-quality surface ge…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）；Cardiff University（卡迪夫大学）
The freeform architectural modeling process often involves two important stages: concept design and digital modeling. In the first stage, architects usually sketch the overall 3D shape and the panel layout on a physical or digital paper briefly. In the second stage, a digital 3D model is created using the sketch as a reference. The digital model needs to incorporate geometric requirements for its components, such as the planarity of panels due to consideration of construction costs, which can make the modeling process more challenging. In this work, we present a novel sketch-based system to br…
