EGOSTREAM: 面向第一人称视角的流式情景记忆诊断基准
EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
AI总结：
提出EGOSTREAM基准，通过七种认知维度和答案有效期窗口，诊断流式视频中模型的情景记忆能力，并评估多种记忆管理机制。
摘要：
连续情景记忆是自主代理在动态真实环境中运行的核心能力，然而当前的流式视频基准为诊断模型记住什么以及记忆多久提供的工具有限。我们引入Egostream，一个面向第一人称视角的流式情景记忆评估诊断基准。Egostream沿七个认知维度组织了2250个精心设计的问题：细节、空间、时间、事件、社会、因果和前瞻记忆。我们引入了答案有效期窗口（AVW），它指定了随着观察场景演变答案保持有效的时间跨度。这使得我们将问题扩展为8528个回忆条件评估，从而能够控制从即时到超长期的回忆测试，同时将模型真正的遗忘与自然世界状态变化区分开来。我们通过一个统一的流式多模态大语言模型框架严格建立了基线性能，该框架比较了几种最先进的记忆管理机制，包括滑动窗口、注意力汇聚、KV缓存剪枝、合并和卸载。在统一的Qwen3-VL骨干网络上的实验表明，可比较的总体准确率掩盖了截然不同的记忆特征。例如，token剪枝在保留细粒度细节和时间结构方面显著优于token合并，而量化卸载则挽救了超长期回忆。最终，所有机制都远低于实时运行（>1秒每帧），且表现最好的方法准确率上限约为45%，揭示了当前架构中的关键差距。Egostream提供了弥合这些差距所需的诊断测试平台。项目网站、新闻和更新请访问：https://saroo25.github.io/Egostream/
英文摘要：
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce Egostream, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. \egostream organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, and offloading. Experiments within a unified Qwen3-VL backbone reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, while quantized offloading rescues ultra-long-term recall. Ultimately, all mechanisms operate well below real-time (>1s per frame), and top performing methods ceil at about 45% accuracy, exposing critical gaps in current architectures. Egostream provides the diagnostic testbed needed to close these gaps. Project website, news and updates at: https://saroo25.github.io/Egostream/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
