Thin-Shell-SfT：基于神经变形场的单目非刚性三维表面细粒度跟踪
Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields
AI总结：
针对单目视频中可变形表面重建缺乏细粒度细节的问题，提出Thin-Shell-SfT方法，采用连续时空神经场与薄壳物理先验，结合三维高斯泼溅可微渲染，实现高精度非刚性三维跟踪。
摘要：
从单目RGB视频中对高度可变形表面（如布料）进行三维重建是一个具有挑战性的问题，目前没有解决方案能够一致且准确地恢复细粒度的表面细节。为了应对该设置的不适定性，现有方法使用具有统计、神经或物理先验的变形模型。它们还主要依赖于非自适应的离散表面表示（如多边形网格），逐帧优化导致误差传播，并受到基于网格的可微渲染器梯度不佳的影响。因此，诸如布料褶皱等精细表面细节往往无法以期望的精度恢复。针对这些局限性，我们提出了ThinShell-SfT，一种新的非刚性三维跟踪方法，将表面表示为隐式且连续的时空神经场。我们引入了基于Kirchhoff-Love模型的连续薄壳物理先验用于空间正则化，这与早期工作的离散化替代方案形成鲜明对比。最后，我们利用三维高斯泼溅将表面可微地渲染到图像空间，并基于分析-综合原理优化变形。得益于我们的连续表面公式以及专门定制的模拟先验和表面诱导的三维高斯，我们的Thin-Shell-SfT在定性和定量上均优于先前的工作。请参阅我们的项目页面：https://4dqv.mpiinf.mpg.de/ThinShellSfT。
英文摘要：
3D reconstruction of highly deformable surfaces (e.g. cloths) from monocular RGB videos is a challenging problem, and no solution provides a consistent and accurate recovery of fine-grained surface details. To account for the ill-posed nature of the setting, existing methods use deformation models with statistical, neural, or physical priors. They also predominantly rely on nonadaptive discrete surface representations (e.g. polygonal meshes), perform frame-by-frame optimisation leading to error propagation, and suffer from poor gradients of the mesh-based differentiable renderers. Consequently, fine surface details such as cloth wrinkles are often not recovered with the desired accuracy. In response to these limitations, we propose ThinShell-SfT, a new method for non-rigid 3D tracking that represents a surface as an implicit and continuous spatiotemporal neural field. We incorporate continuous thin shell physics prior based on the Kirchhoff-Love model for spatial regularisation, which starkly contrasts the discretised alternatives of earlier works. Lastly, we leverage 3D Gaussian splatting to differentiably render the surface into image space and optimise the deformations based on analysis-bysynthesis principles. Our Thin-Shell-SfT outperforms prior works qualitatively and quantitatively thanks to our continuous surface formulation in conjunction with a specially tailored simulation prior and surface-induced 3D Gaussians. See our project page at https://4dqv.mpiinf.mpg.de/ThinShellSfT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
