Light-Omni：基于长期记忆的智能视频理解中的反射优于推理
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
AI总结：
研究针对智能视频理解中先进智能体依赖迭代推理成本高的问题，提出Light-Omni框架，通过双重上下文状态实现反射式轻量级视频理解，经实验验证其有效性，性能优于M3-Agent且能增强现有MLLMs。
摘要：
智能视频理解使模型具备长期记忆，以自主处理和响应连续的长时多模态流。然而，先进的视频智能体在动作控制（如搜索）和证据聚合时，常依赖“侦探式”迭代推理，成本和延迟过高。本文介绍了Light-Omni，一个用于反射式和轻量级视频理解的多模态智能体框架。它通过双重上下文状态在单次前向传播中即时构建所需上下文。首先，维护一个全局状态，从情景记忆中持续整合的有限大小多模态脚本，作为全局上下文。其次，基于此全局上下文生成参数化潜在状态，直接驱动自主动作并产生检索嵌入，延迟极小。实验验证了Light-Omni在多个视频基准测试中的有效性，其性能优于M3-Agent，在准确率、速度和GPU内存效率上均有提升，还可增强现有MLLMs的性能和效率。
英文摘要：
Agentic video understanding equips models with long-term memory to autonomously process and respond to continuous, long-horizon multimodal streams. However, advanced video agents often rely on ``detective-style'' iterative reasoning for action control (e.g., $\mathtt{search}$) and evidence aggregation, incurring prohibitive costs and latency. We argue that such heavy reasoning primarily compensates for the lack of global context and semantic misalignment in retrieval. This paper introduces Light-Omni, a multimodal agent framework for reflexive and lightweight video understanding. It achieves this through dual contextual states that instantly build the required context in a single forward pass. First, we maintain a global state, a finite-sized multimodal script continuously consolidated from episodic memory, serving as the global context for Light-Omni. Through hierarchical merging, it preserves recent details while summarizing past events. Second, conditioned on this global context, we generate a parametric latent state that directly drives autonomous actions and produces retrieval embeddings, with minimal latency. Benefiting from this coupled design, Light-Omni achieves semantically aligned retrieval and reflexive responses while avoiding iterative reasoning. Extensive experiments validate the effectiveness of Light-Omni across multiple video benchmarks. Notably, it outperforms M3-Agent with an average 2.4% accuracy gain, a 12.1$\times$ speedup, and a 2.6$\times$ improvement in GPU memory efficiency. Furthermore, it serves as a memory system to enhance both the performance and efficiency of existing MLLMs. Project page: https://clare-nie.github.io/Light-Omni.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
