VRP-UDF: Towards Unbiased Learning of Unsigned Distance Functions from Multi-view Images with Volume Rendering Priors
School of Software, Tsinghua University（清华大学软件学院）；China Telecom Wanwei Information Technology Co., Ltd.（中国电信万维信息技术有限公司）；Kuaishou Technology（快手技术）；Department of Computer Science, Wayne State University（韦恩州立大学计算机科学系）
Wenyuan Zhang, Chunsheng Wang, Kanle Shi, Yu-Shen Liu, Zhizhong Han
Unsigned distance functions (UDFs) have been a vital representation for open surfaces. With different differentiable renderers, current methods are able to train neural networks to infer a UDF by minimizing the rendering errors with the UDF to the multi-view ground truth. However, these differentiable renderers are mainly handcrafted, which makes them either biased on ray-surface intersections, or sensitive to unsigned distance outliers, or not scalable to large scenes. To resolve these issues, we present a novel differentiable renderer to infer UDFs more accurately. Instead of using handcrafted equations, our differentiable renderer is a neural network which is pre-trained in a data-driven manner. It learns how to render unsigned distances into depth images, leading to a prior knowledge, dubbed volume rendering priors. To infer a UDF for an unseen scene from multiple RGB images, we generalize the learned volume rendering priors to map inferred unsigned distances in alpha blending for RGB image rendering. To reduce the bias of sampling in UDF inference, we utilize an auxiliary point sampling prior as an indicator of ray-surface intersection, and propose novel schemes towards more accurate and uniform sampling near the zero-level sets. We also propose a new strategy that leverages our pretrained volume rendering prior to serve as a general surface refiner, which can be integrated with various Gaussian reconstruction methods to optimize the Gaussian distributions and refine geometric details. Our results show that the learned volume rendering prior is unbiased, robust, scalable, 3D aware, and more importantly, easy to learn. Further experiments show that the volume rendering prior is also a general strategy to enhance other neural implicit representations such as signed distance function and occupancy.
Unsigned Distance Fields (UDFs) can be used to represent non-watertight surfaces. However, current approaches to converting them into explicit meshes tend to either be expensive or to degrade the accuracy. Here, we extend the marching cube algorithm to handle UDFs, both fast and accurately. Moreover, our approach to surface extraction is differentiable, which is key to using pretrained UDF networks to fit sparse data.
Scene Completion is the task of completing missing geometry from a partial scan of a scene. Most previous methods compute an implicit representation from range data using a Truncated Signed Distance Function (T-SDF) computed on a 3D grid as input to neural networks. The truncation decreases but does not remove the border errors introduced by the sign of SDF for open surfaces. As an alternative, we present an Unsigned Distance Function (UDF) as an input representation to scene completion neural networks. The proposed UDF is simple, and efficient as a geometry representation, and can be computed…
CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization
School of Software, Tsinghua University（清华大学软件学院）；Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（纽约大学阿布扎比分校人工智能与机器人中心）；Wayne State University（韦恩州立大学）
Details Enhancement in Unsigned Distance Field Learning for High-fidelity 3D Surface Reconstruction
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Stony Brook University（石溪大学）；InnoPeak Technology（InnoPeak科技公司）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
The University of Hong Kong（香港大学）；Tencent Games（腾讯游戏）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Texas A&M University（德克萨斯农工大学）
We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. Recent advances in neural rendering based reconstruction have achieved compelling results. However, these methods are limited to objects with closed surfaces since they adopt Signed Distance Function (SDF) as surface representation which requires the target shape to be divided into inside and outside. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF represe…
Voronoi-Assisted Optimization for Diffusing Unsigned Distance Fields from Unoriented Points
S-Lab Nanyang Technological University Singapore（南洋理工大学新加坡S-Lab）；College of Mathematics Nanjing University of Aeronautics and Astronautics China（南京航空航天大学数学学院）；College of Computing and Data Science Nanyang Technological University Singapore（南洋理工大学计算与数据科学学院）；Institute of Software Chinese Academy of Sciences China（中国科学院软件研究所）；School of Computer Science Shandong University China（山东大学计算机科学学院）；Department of Computer Science City University of Hong Kong China（香港城市大学计算机科学系）；SenseTime Research China（SenseTime研究院）
UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusion
School of Software Tsinghua University（清华大学软件学院）；BAAI（北京智源人工智能研究院）；Kuaishou Technology（快手科技）；Department of Computer Science Wayne State University（韦恩州立大学计算机科学系）
2S-UDF: A Novel Two-stage UDF Learning Method for Robust Non-watertight Model Reconstruction from Multi-view Images
State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所计算机科学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Unsigned Orthogonal Distance Fields: An Accurate Neural Implicit Representation for Diverse 3D Shapes
Donghua University（东华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
DCUDF2: Improving Efficiency and Accuracy in Extracting Zero Level Sets from Unsigned Distance Fields
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；OPPO Research Center（OPPO研究中心）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Robust Zero Level-Set Extraction from Unsigned Distance Fields Based on Double Covering
State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所计算机科学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Stony Brook University（石溪大学）；Nanyang Technological University（南洋理工大学）
High fidelity representation of shapes with arbitrary topology is an important problem for a variety of vision and graphics applications. Owing to their limited resolution, classical discrete shape representations using point clouds, voxels and meshes produce low quality results when used in these applications. Several implicit 3D shape representation approaches using deep neural networks have been proposed leading to significant improvements in both quality of representations as well as the impact on downstream applications. However, these methods can only be used to represent topologically c…
Inception Institute of Artificial Intelligence（人工智能初学者研究所）；ReLER, University of Technology Sydney（悉尼科技大学ReLER）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
While single-view 3D reconstruction has made significant progress benefiting from deep shape representations in recent years, garment reconstruction is still not solved well due to open surfaces, diverse topologies and complex geometric details. In this paper, we propose a novel learnable Anchored Unsigned Distance Function (AnchorUDF) representation for 3D garment reconstruction from a single image. AnchorUDF represents 3D shapes by predicting unsigned distance fields (UDFs) to enable open garment surface modeling at arbitrary resolution. To capture diverse garment topologies, AnchorUDF not o…
MIND: Material Interface Generation from UDFs for Non-Manifold Surface Reconstruction
Key Laboratory of System Software and State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所系统软件重点实验室、计算机科学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Computer Science, Stony Brook University（石溪大学计算机科学系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
A Lightweight UDF Learning Framework for 3D Reconstruction Based on Local Shape Functions
School of Software, Dalian University of Technology（大连理工大学软件学院）；CCDS, Nanyang Technological University（南洋理工大学CCDS）；Key Laboratory of System Software (CAS) and SKLCS, Institute of Software, CAS（中国科学院软件研究所系统软件重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Volume Rendering with Metric--Phase Fields for Thin Structures and Sign-Ambiguous Geometry
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；City University of Hong Kong（香港城市大学）；Texas A&M University（德克萨斯农工大学）
ETH Zurich（苏黎世联邦理工学院）；MPI for Intelligent Systems, Tübingen（蒂宾根智能系统MPI研究所）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）；Utopilot（优途驾驶）；Microsoft（微软公司）
Monge-Ampere Regularization for Learning Arbitrary Shapes from Point Clouds
School of Software, Shandong University（山东大学软件学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Department of Computer Science and Engineering, Texas A&M University（德克萨斯大学阿姆斯特朗分校计算机科学与工程系）
Carnegie Mellon University（卡内基梅隆大学）；Indian Institute of Science（印度科学学院）；Verisk Analytics（韦里斯克分析公司）
Deep neural representations of 3D shapes as implicit functions have been shown to produce high fidelity models surpassing the resolution-memory trade-off faced by the explicit representations using meshes and point clouds. However, most such approaches focus on representing closed shapes. Unsigned distance function (UDF) based approaches have been proposed recently as a promising alternative to represent both open and closed shapes. However, since the gradients of UDFs vanish on the surface, it is challenging to estimate local (differential) geometric properties like the normals and tangent pl…
Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models
Texas A&M University（得克萨斯农工大学）；The University of Hong Kong（香港大学）；TransGP；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Brown University（布朗大学）；Meta Reality Labs Zurich（元现实实验室苏黎世分部）
提出 Surf-D 方法，采用无符号距离场（UDF）作为表面表示，结合基于点的 AutoEncoder 和潜在扩散模型，生成具有任意拓扑结构的高质量 3D 表面，并在多种模态条件下的形状生成任务中表现卓越。
NeuralBoneReg: An Instance-Specific Label-Free Point Cloud-Based Method for Multi-Modal Bone Surface Registration
Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich（骨科计算机科学研究所，巴尔格里斯大学医院，苏黎世大学）；AI Center, ETH Zurich（人工智能中心，苏黎世联邦理工学院）
