arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.15239cs.CV
收藏

TokenGS: 通过可学习的标记解耦3D高斯预测与像素

TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens

NVIDIA

Jiawei Ren, Michal Jan Tyszkiewicz, Jiahui Huang, Zan Gojcic

AI总结:

TokenGS通过自监督渲染损失直接回归3D均值坐标,改进了3DGS预测的鲁棒性与效率,实现了更规则的几何和更平衡的3DGS分布,支持多视图不一致性和姿态噪声的鲁棒性。

摘要:

在本文中,我们重新审视了现代基于Transformer的方法在前馈3D高斯点扩散(3DGS)预测中的关键设计选择。我们认为将高斯均值作为沿相机光线的深度回归是次优的,转而提出仅使用自监督渲染损失直接回归3D均值坐标。这种形式使我们能够从标准的编码器-only设计转变为具有可学习高斯标记的编码器-解码器架构,从而将预测的原始数量与输入图像分辨率和视图数量解耦。我们的方法TokenGS在静态和动态场景中实现了最先进的前馈重建性能,产生更规则化的几何和更平衡的3DGS分布,同时无缝恢复出显式场景属性,如静态-动态分解和场景流。

英文摘要:

In this work, we revisit several key design choices of modern Transformer-based approaches for feed-forward 3D Gaussian Splatting (3DGS) prediction. We argue that the common practice of regressing Gaussian means as depths along camera rays is suboptimal, and instead propose to directly regress 3D mean coordinates using only a self-supervised rendering loss. This formulation allows us to move from the standard encoder-only design to an encoder-decoder architecture with learnable Gaussian tokens, thereby unbinding the number of predicted primitives from input image resolution and number of views. Our resulting method, TokenGS, demonstrates improved robustness to pose noise and multiview inconsistencies, while naturally supporting efficient test-time optimization in token space without degrading learned priors. TokenGS achieves state-of-the-art feed-forward reconstruction performance on both static and dynamic scenes, producing more regularized geometry and more balanced 3DGS distribution, while seamlessly recovering emergent scene attributes such as static-dynamic decomposition and scene flow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑