We propose a novel, end-to-end trainable, deep network called ParSeNet that decomposes a 3D point cloud into parametric surface patches, including B-spline patches as well as basic geometric primitives. ParSeNet is trained on a large-scale dataset of man-made 3D shapes and captures high-level semantic priors for shape decomposition. It handles a much richer class of primitives than prior work, and allows us to represent surfaces with higher fidelity. It also produces repeatable and robust parametrizations of a surface compared to purely geometric approaches. We present extensive experiments to validate our approach against analytical and learning-based alternatives. Our source code is publicly available at: https://hippogriff.github.io/parsenet.
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Tsinghua University（清华大学）；Stanford University（斯坦福大学）；UC San Diego（加州大学圣地亚哥分校）
Reconstruction of geometry based on different input modes, such as images or point clouds, has been instrumental in the development of computer aided design and computer graphics. Optimal implementations of these applications have traditionally involved the use of spline-based representations at their core. Most such methods attempt to solve optimization problems that minimize an output-target mismatch. However, these optimization techniques require an initialization that is close enough, as they are local methods by nature. We propose a deep learning architecture that adapts to perform spline…
Technical University of Munich（慕尼黑工业大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Parametric 3D models have enabled a wide variety of tasks in computer graphics and vision, such as modeling human bodies, faces, and hands. However, the construction of these parametric models is often tedious, as it requires heavy manual tweaking, and they struggle to represent additional complexity and details such as wrinkles or clothing. To this end, we propose Neural Parametric Models (NPMs), a novel, learned alternative to traditional, parametric 3D models, which does not require hand-crafted, object-specific constraints. In particular, we learn to disentangle 4D dynamics into latent-spa…
We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance with a neural radiance field, namely Morphable Facial NeRF. Specifically, MoFaNeRF takes the coded facial shape, expression and appearance along with space coordinate and view direction as input to an MLP, and outputs the radiance of the space point for photo-realistic image synthesis. Compared with conventional 3D morphable models (3DMM), MoFaNeRF shows superiority in directly synthesizing photo-realistic facial details even for eyes, mouths, and beards. Also, continuo…
Fit4CAD: A point cloud benchmark for fitting simple geometric primitives in CAD objects
Istituto di Matematica Applicata e Tecnologie Informatiche “E. Magenes”, Consiglio Nazionale delle Ricerche（意大利国家研究委员会“E. Magenes”应用数学与信息技术研究所）；University of Oslo（奥斯陆大学）；ENS Paris-Saclay, Université Paris-Saclay（巴黎萨克雷大学巴黎萨克雷高等师范学院）
CUHKSZ（香港中文大学（深圳））；Tencent America（腾讯美国）；Technical University of Munich（慕尼黑工业大学）
Extracting parametric edge curves from point clouds is a fundamental problem in 3D vision and geometry processing. Existing approaches mainly rely on keypoint detection, a challenging procedure that tends to generate noisy output, making the subsequent edge extraction error-prone. To address this issue, we propose to directly detect structured edges to circumvent the limitations of the previous point-wise methods. We achieve this goal by presenting NerVE, a novel neural volumetric edge representation that can be easily learned through a volumetric learning framework. NerVE can be seamlessly co…
Taylor Series-Inspired Local Structure Fitting Network for Few-shot Point Cloud Semantic Segmentation
Cyber Security Research Center (CYSREN), Nanyang Technological University（南洋理工大学网络安全研究中心（CYSREN））；Shanghai University of Finance and Economics（上海财经大学）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Information Technology, Halmstad University（哈尔姆斯塔德大学信息技术学院）
Patlak Parametric Image Estimation from Dynamic PET Using Diffusion Model Prior
J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida（J. Crayton Pruitt家族生物医学工程系，佛罗里达大学）；Department of Radiology, University of Florida（放射学系，佛罗里达大学）；Interdisciplinary Program in Bioengineering, Seoul National University（生物工程跨学科项目，首尔国立大学）；Department of Nuclear Medicine, Seoul National University College of Medicine（核医学系，首尔国立大学医学院）；Department of Radiology, University of California Davis Health（放射学系，加州大学戴维斯医学中心）
University of North Carolina（北卡罗来纳大学）；University of Salzburg（萨尔茨堡大学）；University of California San Diego（加州大学圣迭戈分校）
We address the problem of fitting parametric curves on the Grassmann manifold for the purpose of intrinsic parametric regression. As customary in the literature, we start from the energy minimization formulation of linear least-squares in Euclidean spaces and generalize this concept to general nonflat Riemannian manifolds, following an optimal-control point of view. We then specialize this idea to the Grassmann manifold and demonstrate that it yields a simple, extensible and easy-to-implement solution to the parametric regression problem. In fact, it allows us to extend the basic geodesic mode…
Leibniz University Hannover（莱布尼茨汉诺威大学）；Heidelberg University（海德堡大学）；University of Twente（屯特大学）
We present a robust estimator for fitting multiple parametric models of the same form to noisy measurements. Applications include finding multiple vanishing points in man-made scenes, fitting planes to architectural imagery, or estimating multiple rigid motions within the same sequence. In contrast to previous works, which resorted to hand-crafted search strategies for multiple model detection, we learn the search strategy from data. A neural network conditioned on previously detected models guides a RANSAC estimator to different subsets of all measurements, thereby finding model instances one…
This paper presents a novel method for 3D human pose and shape estimation from images with sparse views, using joint points and silhouettes, based on a parametric model. Firstly, the parametric model is fitted to the joint points estimated by deep learning-based human pose estimation. Then, we extract the correspondence between the parametric model of pose fitting and silhouettes on 2D and 3D space. A novel energy function based on the correspondence is built and minimized to fit parametric model to the silhouettes. Our approach uses sufficient shape information because the energy function of …
In this article, we present a novel scheme for segmenting the image boundary (with the background) in optoacoustic small animal in vivo imaging systems. The method utilizes a multiscale edge detection algorithm to generate a binary edge map. A scale dependent morphological operation is employed to clean spurious edges. Thereafter, an ellipse is fitted to the edge map through constrained parametric transformations and iterative goodness of fit calculations. The method delimits the tissue edges through the curve fitting model, which has shown high levels of accuracy. Thus, this method enables se…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Microsoft（微软）；Meta Reality Labs Research（Meta现实实验室）
Fitting parametric models of human bodies, hands or faces to sparse input signals in an accurate, robust, and fast manner has the promise of significantly improving immersion in AR and VR scenarios. A common first step in systems that tackle these problems is to regress the parameters of the parametric model directly from the input data. This approach is fast, robust, and is a good starting point for an iterative minimization algorithm. The latter searches for the minimum of an energy function, typically composed of a data term and priors that encode our knowledge about the problem's structure…
