arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.04035cs.CVcs.LG
收藏

从多视角捕获中大规模高质量3D高斯头部重建

Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures

Apple(苹果公司)
Evangelos Ntavelis, Sean Wu, Mohamad Shahbazi, Fabio Maninchedda, Dmitry Kostiaev, Artem Sevastopolsky, Vittorio Megaro, Trevor Phillips, Alejandro Blumentals, … 展开作者

Evangelos Ntavelis, Sean Wu, Mohamad Shahbazi, Fabio Maninchedda, Dmitry Kostiaev, Artem Sevastopolsky, Vittorio Megaro, Trevor Phillips, Alejandro Blumentals, Shridhar Ravikumar, Mehak Gupta, Reinhard Knothe, Jeronimo Bayer, Matthias Vestner, Simon Schaefer, Thomas Etterlin, Christian Zimmermann, Alexey Artemov, Mathias Deschler, Peter Kaufmann, Stefan Brugger, Sebastian Martin, Brian Amberg, Tom Runia

AI总结:

HeadsUp通过高效编码器-解码器架构实现大规模多视角捕获下高质量3D高斯头部重建,利用UV参数化高斯体实现与输入图像数量和分辨率解耦,支持高分辨率输入图像训练,取得最佳重建质量并能泛化到新身份。

摘要:

我们提出HeadsUp,一种可扩展的前馈方法,用于从大规模多摄像头设置中重建高质量的3D高斯头部。我们的方法采用高效的编码器-解码器架构,将输入视角压缩成紧凑的潜在表示。该潜在表示随后被解码为一组UV参数化的3D高斯体,锚定在中性头部模板上。此UV表示将3D高斯体数量与输入图像的数量和分辨率解耦,使模型能够使用大量高分辨率输入视角进行训练。我们在一个内部数据集上训练和评估模型,该数据集包含超过10,000个主体,比现有多视角人类头部数据集大一个数量级。HeadsUp实现了最先进的重建质量和泛化能力,无需测试时优化即可处理新身份。我们广泛分析了模型在身份、视角和模型容量上的扩展行为,揭示了质量-计算量权衡的实用见解。最后,我们通过展示两个下游应用来突出潜在空间的优势:生成新的3D身份和使用表情混合形状动画3D头部。

英文摘要:

We propose HeadsUp, a scalable feed-forward method for reconstructing high-quality 3D Gaussian heads from large-scale multi-camera setups. Our method employs an efficient encoder-decoder architecture that compresses input views into a compact latent representation. This latent representation is then decoded into a set of UV-parameterized 3D Gaussians anchored to a neutral head template. This UV representation decouples the number of 3D Gaussians from the number and resolution of input images, enabling training with many high-resolution input views. We train and evaluate our model on an internal dataset with more than 10,000 subjects, which is an order of magnitude larger than existing multi-view human head datasets. HeadsUp achieves state-of-the-art reconstruction quality and generalizes to novel identities without test-time optimization. We extensively analyze the scaling behavior of our model across identities, views, and model capacity, revealing practical insights for quality-compute trade-offs. Finally, we highlight the strength of our latent space by showcasing two downstream applications: generating novel 3D identities and animating the 3D heads with expression blendshapes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑