arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2403.16292cs.CV

latentSplat:用于快速可泛化3D重建的自编码变分高斯

latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction

Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Saarland University(萨尔大学)

Christopher Wewer, Kevin Raj, Eddy Ilg, Bernt Schiele, Jan Eric Lenssen

AI总结:

latentSplat提出了一种在3D潜在空间预测变分语义高斯并结合轻量级2D生成解码器的方法,兼顾回归与生成优势,实现了快速、可扩展且高泛化性的3D重建。

摘要:

我们提出了latentSplat,一种在3D潜在空间中预测语义高斯的方法,这些高斯可被splat并由轻量级生成式2D架构解码。现有的可泛化3D重建方法要么无法扩展到大场景和高分辨率,要么仅限于对相近输入视图的插值。latentSplat结合了基于回归和生成式方法的优势,且完全基于现成的真实视频数据进行训练。该方法的核心是变分3D高斯,这是一种在由3D特征高斯组成的潜在空间中高效编码不同不确定性的表示。从这些高斯中,可采样特定实例,并通过高效splatting和快速生成式解码器进行渲染。我们证明latentSplat在重建质量和泛化能力上优于先前工作,同时具备快速且可扩展至高分辨率数据的特性。

英文摘要:

We present latentSplat, a method to predict semantic Gaussians in a 3D latent space that can be splatted and decoded by a light-weight generative 2D architecture. Existing methods for generalizable 3D reconstruction either do not scale to large scenes and resolutions, or are limited to interpolation of close input views. latentSplat combines the strengths of regression-based and generative approaches while being trained purely on readily available real video data. The core of our method are variational 3D Gaussians, a representation that efficiently encodes varying uncertainty within a latent space consisting of 3D feature Gaussians. From these Gaussians, specific instances can be sampled and rendered via efficient splatting and a fast, generative decoder. We show that latentSplat outperforms previous works in reconstruction quality and generalization, while being fast and scalable to high-resolution data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑