Keepsake：面向长时程视频生成的选择性空间记忆
Keepsake: Selective Spatial Memory for Long-Horizon Video Generation
- Institute of Artificial Intelligence, University of Central Florida（中佛罗里达大学人工智能研究所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对长时程视频生成中固定容量空间记忆的保留问题，提出Keepsake在线无训练控制模块，通过姿态-外观图动态评估观测可替代性，在MemCam和WorldMem上提升FVD和LPIPS，仅保留32/5397帧即降FVD 35.1%。
AI中文摘要：
长时程相机控制视频生成依赖于持久记忆来维持场景一致性。现有系统采用两种策略来实现这种一致性。全历史方法保留所有生成的观测，导致存储和检索成本无界增长。选择性构建方法减少了冗余，但做出一次性保留决策后便不再重新审视，即使观测的价值随记忆库的演变而变化。两种策略都留下一个共同的问题未解决：随着生成历史的演变，哪些存储的观测仍应保留在记忆中？我们的关键见解是，存储观测的价值并非固定不变，而是关系性的：它取决于记忆库中当前可用的替代项。一个由许多几何和视觉上相似的替代项支持的视图可以以较小的覆盖损失被放弃，而一个几乎没有可行替代项的观测无论新旧都应保留。我们引入了Keepsake，一种用于固定容量空间记忆的在线、无需训练的控制模块。在每次更新时，Keepsake在保留的和新生成的观测上构建一个姿态-外观图，结合相机姿态邻近性与视觉相似性。保留优先级共同捕捉强替代项的数量和最近替代项的相似度，使Keepsake能够持续重新评估记忆价值，保留替代支持较少的观测，并在固定预算下驱逐高度可替换的观测。该控制模块仅修改持久记忆更新；宿主生成器、去噪调度和检索规则保持不变。在MemCam和WorldMem上，Keepsake在固定记忆预算下改善了FVD和LPIPS。在180秒的MemCam轨迹上，它仅保留5397帧中的32帧，同时将FVD降低了35.1%。
英文摘要：
Long-horizon camera-controlled video generation relies on persistent memory to maintain scene consistency. Existing systems follow two strategies to achieve this consistency. Full-history approaches retain all generated observations, causing unbounded storage and retrieval costs. Selective-construction approaches reduce redundancy, but make one-time retention decisions that are never revisited, even as an observation's value changes with the evolving memory bank. Both strategies leave a shared question unresolved: as the generated history evolves, which stored observations should still remain in memory? Our key insight is that the value of a stored observation is not fixed, but relational: it depends on the alternatives currently available in the memory bank. A view supported by many geometrically and visually similar substitutes can be relinquished with little loss of coverage, whereas an observation with few viable alternatives should remain regardless of age. We introduce Keepsake, an online, training-free controller for fixed-capacity spatial memory. At each update, Keepsake constructs a pose-appearance graph over retained and newly generated observations, combining camera-pose proximity with visual similarity. A retention priority jointly captures the number of strong substitutes and the similarity of the closest alternative, allowing Keepsake to continually reassess memory value, preserve observations with little alternative support, and evict highly replaceable ones under a fixed budget. The controller modifies only the persistent-memory update; the host generator, denoising schedule, and retrieval rule remain unchanged. Across MemCam and WorldMem, Keepsake improves FVD and LPIPS under a fixed memory budget. On 180-second MemCam trajectories, it retains only 32 of 5,397 frames while reducing FVD by 35.1%.
