arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20160cs.CV
收藏

tttLRM:面向长上下文和自回归3D重建的测试时间训练

tttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstruction

University of Pennsylvania(宾夕法尼亚大学);Adobe Research(Adobe 研究);UCI(加州大学欧文分校)

Chen Wang, Hao Tan, Wang Yifan, Zhiqin Chen, Yuheng Liu, Kalyan Sunkavalli, Sai Bi, Lingjie Liu, Yiwei Hu

AI总结:

tttLRM通过测试时间训练实现长上下文自回归3D重建,提升重建质量和收敛速度。

摘要:

我们提出了tttLRM,一种新颖的大型3D重建模型,利用测试时间训练(TTT)层实现具有线性计算复杂度的长上下文、自回归3D重建,进一步扩展了模型的能力。我们的框架高效地将多个图像观测压缩到TTT层的快速权重中,形成隐空间中的隐式3D表示,可以解码为多种显式格式,如高斯溅射(GS)以供后续应用。我们模型的在线学习变体支持从流式观测中逐步进行3D重建和细化。我们证明了在新颖视角合成任务上预训练能够有效转移到显式3D建模,从而提升重建质量和加快收敛速度。大量实验表明,我们的方法在面向前馈3D高斯重建方面,在物体和场景上均优于最先进的方法。

英文摘要:

We propose tttLRM, a novel large 3D reconstruction model that leverages a Test-Time Training (TTT) layer to enable long-context, autoregressive 3D reconstruction with linear computational complexity, further scaling the model's capability. Our framework efficiently compresses multiple image observations into the fast weights of the TTT layer, forming an implicit 3D representation in the latent space that can be decoded into various explicit formats, such as Gaussian Splats (GS) for downstream applications. The online learning variant of our model supports progressive 3D reconstruction and refinement from streaming observations. We demonstrate that pretraining on novel view synthesis tasks effectively transfers to explicit 3D modeling, resulting in improved reconstruction quality and faster convergence. Extensive experiments show that our method achieves superior performance in feedforward 3D Gaussian reconstruction compared to state-of-the-art approaches on both objects and scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑