Mamba-VGGT: 通过外部滑动窗口Mamba内存实现持久长序列视频几何 grounded 变换器
Mamba-VGGT: Persistent Long-Sequence Video Geometry Grounded Transformer via External Sliding Window Mamba Memory
AI总结：
本文提出Mamba-VGGT框架，通过引入滑动窗口Mamba内存模块，解决传统VGGT在长序列视频中几何遗忘和累积漂移问题，提升3D场景重建的精度与稳定性。
摘要：
视觉几何 grounded 变换器（VGGT）在高保真3D场景重建中设立了新基准。然而，随着序列长度增加，这些模型因全局注意力的二次复杂度而出现灾难性几何遗忘和累积漂移，主要由于需要截断的时间窗口。为克服由此产生的几何漂移，我们提出了Mamba-VGGT，一种增强的VGGT框架，能够实现持久的长距离推理。我们的关键贡献是滑动窗口Mamba（SWM）内存模块，该模块在时间窗口间维护显式的外部记忆标记。该模块利用选择性状态空间建模来提炼和传播全局几何先验，有效绕过了传统变换器的记忆限制。为了在不破坏预训练VGGT高度优化的空间特征的情况下整合这些长期时间线索，我们提出了一种零初始化空间内存注入器。利用零卷积层，该注入器适应性地将持久记忆融合到patch token流中，确保结构稳定性和无缝特征对齐。广泛实验表明，我们的方法在维持空间一致性和减少轨迹累积误差方面显著优于现有VGGT方法。我们的工作为大规模3D环境中基于几何的世界建模提供了可扩展、线性复杂度的解决方案。
英文摘要：
Visual Geometry Grounded Transformers (VGGT) have set new benchmarks in high-fidelity 3D scene reconstruction. However, as the sequence length increases, these models suffer from catastrophic geometric forgetting and accumulation drift, primarily due to the quadratic complexity of global attention which necessitates truncated temporal windows. To overcome the resulting geometric drift, we present Mamba-VGGT, an enhanced VGGT framework capable of persistent long-range reasoning. Our key contribution is a Sliding Window Mamba (SWM) memory module that maintains an explicit external memory token across temporal windows. This module leverages selective state-space modeling to distill and propagate global geometric priors, effectively bypassing the memory constraints of traditional transformers. To integrate these long-term temporal cues without disrupting the highly optimized spatial features of the pre-trained VGGT, we propose a Zero-Init Spatial Memory Injector. Utilizing zero-convolutional layers, this injector adaptively fuses persistent memory into the patch token stream, ensuring structural stability and seamless feature alignment. Extensive experiments demonstrate that our approach significantly outperforms existing VGGT-based methods in maintaining spatial consistency and reducing trajectory accumulation errors. Our work provides a scalable, linear-complexity solution for geometry-grounded world modeling in extensive 3D environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
