We describe in this short note a technique to convert an implicit surface into a Signed Distance Function (SDF) while exactly preserving the zero level-set of the implicit. The proposed approach relies on embedding the input implicit in the final layer of a neural network, which is trained to minimize a loss function characterizing the SDF.
Microsoft Research Asia(微软亚洲研究院);Tsinghua University(清华大学)
Multilayer perceptrons (MLPs) have been successfully used to represent 3D shapes implicitly and compactly, by mapping 3D coordinates to the corresponding signed distance values or occupancy values. In this paper, we propose a novel positional encoding scheme, called Spline Positional Encoding, to map the input coordinates to a high dimensional space before passing them to MLPs, for helping to recover 3D signed distance fields with fine-scale geometric details from unorganized 3D point clouds. We verified the superiority of our approach over other positional encoding schemes on tasks of 3D shap…
ETH Zurich(苏黎世联邦理工学院);Tsinghua University(清华大学);Google(谷歌);Peking University(北京大学);Peng Cheng Laboratory(鹏城实验室);Microsoft(微软)
We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function. Due to the nature of the implicit function, the rendering process requires tremendous function queries, which is particularly problematic when the function is represented as a neural network. We optimize both the forward and backward passes of our rendering layer to make it run efficiently with affordable memory consumption on a commodity graphics card. Our rendering method is fully differentiable such that loss…
Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution. Generalizing across shapes with such neural implicit representations amounts to learning priors over the respective function space and enables geometry reconstruction from partial or noisy observations. Existing generalization methods rely on conditioning a neural network on a low-dimensional latent code that is either regressed by an encoder or jointly optimized in the auto-decoder framework. He…
National Technical University of Athens(雅典国立技术大学);Foundation for Research & Technology - Hellas (FORTH)(希腊研究与技术基金会(FORTH));University of Exeter(埃克塞特大学)
In recent years, implicit surface representations through neural networks that encode the signed distance have gained popularity and have achieved state-of-the-art results in various tasks (e.g. shape representation, shape reconstruction, and learning shape priors). However, in contrast to conventional shape representations such as polygon meshes, the implicit representations cannot be easily edited and existing works that attempt to address this problem are extremely limited. In this work, we propose the first method for efficient interactive editing of signed distance functions expressed thr…
Recent progress in neural implicit functions has set new state-of-the-art in reconstructing high-fidelity 3D shapes from a collection of images. However, these approaches are limited to closed surfaces as they require the surface to be represented by a signed distance field. In this paper, we propose NeAT, a new neural rendering framework that can learn implicit surfaces with arbitrary topologies from multi-view images. In particular, NeAT represents the 3D surface as a level set of a signed distance function (SDF) with a validity branch for estimating the surface existence probability at the …
Recently, neural networks have been used as implicit representations for surface reconstruction, modelling, learning, and generation. So far, training neural networks to be implicit representations of surfaces required training data sampled from a ground-truth signed implicit functions such as signed distance or occupancy functions, which are notoriously hard to compute. In this paper we introduce Sign Agnostic Learning (SAL), a deep learning approach for learning implicit shape representations directly from raw, unsigned geometric data, such as point clouds and triangle soups. We have tested …
Recent advances in learning 3D shapes using neural implicit functions have achieved impressive results by breaking the previous barrier of resolution and diversity for varying topologies. However, most of such approaches are limited to closed surfaces as they require the space to be divided into inside and outside. More recent works based on unsigned distance function have been proposed to handle complex geometry containing both the open and closed surfaces. Nonetheless, as their direct outputs are point clouds, robustly obtaining high-quality meshing results from discrete points remains an op…
We describe a two-level method for computing a function whose zero-level set is the surface reconstructed from given points scattered over the surface and associated with surface normal vectors. The function is defined as a linear combination of compactly supported radial basis functions (CSRBFs). The method preserves the simplicity and efficiency of implicit surface interpolation with CSRBFs and the reconstructed implicit surface owns the attributes, which are previously only associated with globally supported or globally regularized radial basis functions, such as exhibiting less extra zero-…
University of Maryland, College Park(马里兰大学帕克分校);Google Research(谷歌研究院)
We introduce a new implicit shape representation called Primary Ray-based Implicit Function (PRIF). In contrast to most existing approaches based on the signed distance function (SDF) which handles spatial locations, our representation operates on oriented rays. Specifically, PRIF is formulated to directly produce the surface hit point of a given input ray, without the expensive sphere-tracing operations, hence enabling efficient shape extraction and differentiable rendering. We demonstrate that neural networks trained to encode PRIF achieve successes in various tasks including single shape re…
UC San Diego(加州大学圣地亚哥分校);Massachusetts Institute of Technology(麻省理工学院)
Recent techniques have been successful in reconstructing surfaces as level sets of learned functions (such as signed distance fields) parameterized by deep neural networks. Many of these methods, however, learn only closed surfaces and are unable to reconstruct shapes with boundary curves. We propose a hybrid shape representation that combines explicit boundary curves with implicit learned interiors. Using machinery from geometric measure theory, we parameterize currents using deep networks and use stochastic gradient descent to solve a minimal surface problem. By modifying the metric accordin…
University of British Columbia(不列颠哥伦比亚大学);Simon Fraser University(西蒙菲莎大学);Google Research(谷歌研究院);Stanford University(斯坦福大学);University of Toronto(多伦多大学)
Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form. In this work, we introduce medial fields: a field function derived from the medial axis transform (MAT) that makes available information about the underlying 3D geometry that is immediately useful for a number of downstream tasks. In particular, the medial field encodes the local thickness of a 3D shape, and enables O(1) projection of a query point onto the medial axis. To construct the medial field we require nothin…
Tsinghua University(清华大学);Microsoft Research Asia(微软亚洲研究院)
We present a novel implicit representation -- neural halfspace representation (NH-Rep), to convert manifold B-Rep solids to implicit representations. NH-Rep is a Boolean tree built on a set of implicit functions represented by the neural network, and the composite Boolean function is capable of representing solid geometry while preserving sharp features. We propose an efficient algorithm to extract the Boolean tree from a manifold B-Rep solid and devise a neural network-based optimization approach to compute the implicit functions. We demonstrate the high quality offered by our conversion algo…
Inria Sophia Antipolis(索菲亚·安提波利斯法国国家信息与自动化研究所);Université Côte d’Azur(蔚蓝海岸大学);Head and Neck University Institute(头颈大学研究所);Oticon Medical(奥迪康医疗);Nice University Hospital(尼斯大学医院)
Signed distance map (SDM) is a common representation of surfaces in medical image analysis and machine learning. The computational complexity of SDM for 3D parametric shapes is often a bottleneck in many applications, thus limiting their interest. In this paper, we propose a learning based SDM generation neural network which is demonstrated on a tridimensional cochlea shape model parameterized by 4 shape parameters. The proposed SDM Neural Network generates a cochlea signed distance map depending on four input parameters and we show that the deep learning approach leads to a 60 fold improvemen…
Volumetrically Consistent Implicit Atlas Learning via Neural Diffeomorphic Flow for Placenta MRI
MIT Computer Science and Artificial Intelligence Laboratory(麻省理工学院计算机科学与人工智能实验室);Massachusetts General Hospital, Harvard Medical School(哈佛医学院麻省总医院);Boston Children’s Hospital, Harvard Medical School(哈佛医学院波士顿儿童医院)
Nanyang Technological University(南洋理工大学);A*STAR(新加坡科技研究局);Huazhong University of Science and Technology(华中科技大学)
Recently, neural implicit surfaces learning by volume rendering has become popular for multi-view reconstruction. However, one key challenge remains: existing approaches lack explicit multi-view geometry constraints, hence usually fail to generate geometry consistent surface reconstruction. To address this challenge, we propose geometry-consistent neural implicit surfaces learning for multi-view reconstruction. We theoretically analyze that there exists a gap between the volume rendering integral and point-based signed distance function (SDF) modeling. To bridge this gap, we directly locate th…
FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions
TU Munich(慕尼黑工业大学);MCML(慕尼黑计算数学与机器学习实验室);Massachusetts Institute of Technology(麻省理工学院);University of Bonn(波恩大学);MPI Informatics(马克斯·普朗克信息学研究所);SIC(萨尔信息中心)
Sapienza University of Rome(罗马第一大学);Hochschule Bonn-Rhein-Sieg(波恩-莱茵-锡格应用技术大学)
Representing 3D surfaces as level sets of continuous functions over $\mathbb{R}^3$ is the common denominator of neural implicit representations, which recently enabled remarkable progress in geometric deep learning and computer vision tasks. In order to represent 3D motion within this framework, it is often assumed (either explicitly or implicitly) that the transformations which a surface may undergo are homeomorphic: this is not necessarily true, for instance, in the case of fluid dynamics. In order to represent more general classes of deformations, we propose to apply this theoretical framew…
We introduce a novel approach for rendering static and dynamic 3D neural signed distance functions (SDF) in real-time. We rely on nested neighborhoods of zero-level sets of neural SDFs, and mappings between them. This framework supports animations and achieves real-time performance without the use of spatial data-structures. It consists of three uncoupled algorithms representing the rendering steps. The multiscale sphere tracing focuses on minimizing iteration time by using coarse approximations on earlier iterations. The neural normal mapping transfers details from a fine neural SDF to a surf…