CADSplat：基于CAD模型辅助的稀疏视角3D高斯泼溅鲁棒逼真数字孪生重建

中文摘要 AI 辅助

我们提出CADSplat，一个通过使用显式CAD形状先验正则化3D高斯泼溅（3DGS），从物体的稀疏（少于15个视角）、宽基线有姿态图像中重建逼真、几何精确的数字孪生的框架。使用这样的先验需要找到一个形状与图像中物体相似的CAD模型，并确定每个相机相对于物体的姿态。我们通过将分割的物体轮廓与从CAD库渲染的轮廓进行匹配，并保留最佳匹配模型的相机到物体姿态来同时获得两者。然后，我们将3D高斯图元锚定到检索模型的表面，并联合优化3DGS参数、相机到物体的配准以及一个非刚性变形场，以考虑物理物体与CAD模型之间的形状差异。在两个真实世界数据集上，CADSplat优于无约束、少样本和网格纹理化基线，并且即使在只有3个视图的情况下也能优雅地退化。我们的实验表明，渲染质量的大部分提升来自于泼溅的约束方式——一组固定数量的泼溅绑定到表面并由单个平滑变形场驱动——而非CAD形状本身。CAD模型在视图最稀缺的地方（最稀疏的捕获和强烈自遮挡的物体上）增加了形状知识，并将每个相机置于物体自身的坐标系中。这使得超越新视图合成的应用成为可能，例如无标记增强现实配准、逐图像物体姿态估计、物理模拟以及将部件标签从设计转移到重建。

英文摘要

We present CADSplat, a framework that reconstructs photorealistic, geometrically accurate digital twins from sparse ($<15$ views), wide-baseline posed images of an object by regularizing 3D Gaussian Splatting (3DGS) with an explicit CAD shape prior. Using such a prior requires finding a CAD model whose shape resembles the object depicted in the images and determining the pose of each camera relative to the object. We obtain both by matching segmented object silhouettes against silhouettes rendered from a CAD library and keeping the camera-to-object poses of the best-matching model. We then anchor 3D Gaussian primitives to the surface of the retrieved model and jointly optimize the 3DGS parameters, the camera-to-object registration, and a non-rigid deformation field to account for shape differences between the physical object and the CAD model. Across two real-world datasets, CADSplat outperforms unconstrained, few-shot, and mesh-texturing baselines and degrades gracefully to as few as 3 views. Our experiments show that most of the gain in rendering quality comes from how the splats are constrained---a fixed set of splats tied to a surface and moved by a single smooth deformation field---rather than from the CAD shape itself. The CAD model adds shape knowledge where views are scarcest, in the sparsest captures and on strongly self-occluded objects, and it places every camera in the object's own frame. This enables applications beyond novel-view synthesis, such as markerless augmented reality registration, per-image object pose estimation, physical simulations, and the transfer of part labels from the design to the reconstruction.

发表机构

• Digital Future Lab, Flanders Make, Hasselt University（数字未来实验室，法兰德斯制造，哈瑟尔特大学）

机构由 AI 辅助整理，请以论文原文为准。