StreamCacheVGGT:具有鲁棒评分和混合缓存压缩的流式视觉几何变换
StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression
AI总结:
本文提出StreamCacheVGGT框架,通过Cross-Layer Consistency-Enhanced Scoring和Hybrid Cache Compression模块,解决流式视频中稳定重建密集3D几何的问题,实现高精度和稳定性。
摘要:
从连续视频流中重建密集3D几何需要在恒定内存预算下稳定的推理。现有O(1)框架主要依赖于'纯驱逐'范式,由于二进制令牌删除和局部单层评分的噪声导致信息损失。为解决这些瓶颈,我们提出StreamCacheVGGT,一种无需训练的框架,通过两个协同模块:跨层一致性增强评分(CLCES)和混合缓存压缩(HCC)重新设计缓存管理。CLCES通过跟踪Transformer层次中的令牌重要性轨迹,利用顺序统计分析来识别持续的几何显著性,以减轻激活噪声。利用这些稳健的评分,HCC超越了简单的驱逐,引入了三级分拣策略,通过在键向量流形上进行最近邻分配,将中等重要的令牌合并到保留的锚点中。这种方法保留了否则会丢失的必要几何上下文。在五个基准(7-Scenes、NRGBD、ETH3D、Bonn和KITTI)上的广泛评估表明,StreamCacheVGGT实现了新的状态-of-the-art,提供优越的重建精度和长期稳定性,同时严格遵守恒定成本约束。
英文摘要:
Reconstructing dense 3D geometry from continuous video streams requires stable inference under a constant memory budget. Existing $O(1)$ frameworks primarily rely on a ``pure eviction'' paradigm, which suffers from significant information destruction due to binary token deletion and evaluation noise from localized, single-layer scoring. To address these bottlenecks, we propose StreamCacheVGGT, a training-free framework that reimagines cache management through two synergistic modules: Cross-Layer Consistency-Enhanced Scoring (CLCES) and Hybrid Cache Compression (HCC). CLCES mitigates activation noise by tracking token importance trajectories across the Transformer hierarchy, employing order-statistical analysis to identify sustained geometric salience. Leveraging these robust scores, HCC transcends simple eviction by introducing a three-tier triage strategy that merges moderately important tokens into retained anchors via nearest-neighbor assignment on the key-vector manifold. This approach preserves essential geometric context that would otherwise be lost. Extensive evaluations on five benchmarks (7-Scenes, NRGBD, ETH3D, Bonn, and KITTI) demonstrate that StreamCacheVGGT sets a new state-of-the-art, delivering superior reconstruction accuracy and long-term stability while strictly adhering to constant-cost constraints.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。