感知、反思并理解长视频：利用交互式智能体进行渐进式多粒度线索探索
Perceive, Reflect and Understand Long Video: Progressive Multi-Granular Clue Exploration with Interactive Agents
AI总结：
针对长视频中任务相关信息稀疏导致的完整性与效率难题，本文提出CogniGPT，通过MGPA与VERA的交互循环渐进筛选可靠线索，并在多个基准上以极少帧数取得优越性能。
摘要：
长视频具有时间复杂性和任务相关信息稀疏的特点，给AI系统带来了重大的推理挑战。尽管各种基于大语言模型（LLM）的方法推动了长视频理解的发展，但它们在捕获任务关键信息时仍难以同时兼顾完整性与效率。受人类渐进式视觉认知启发，我们提出CogniGPT框架，该框架利用多粒度感知智能体（MGPA）与验证增强反思智能体（VERA）之间的交互式循环，实现高效且可靠的长视频理解。具体而言，MGPA模拟人类视觉的发散注意和聚焦注意以捕获任务相关信息，而VERA则验证感知到的关键线索，以缓解幻觉并优化后续感知策略。通过这一交互过程，CogniGPT探索出一组最小规模的、信息丰富且可靠的任务相关线索。在EgoSchema、Video-MME、NExT-QA和MovieChat数据集上的大量实验证明了CogniGPT在准确性和效率两方面的优越性。值得注意的是，在EgoSchema上，它仅使用11.2帧便超越了现有的免训练方法，并取得了与Gemini 1.5-Pro相当的性能。
英文摘要：
Long videos, characterized by temporal complexity and sparse task-relevant information, pose significant reasoning challenges for AI systems. Although various Large Language Model (LLM)-based approaches have advanced long video understanding, they still struggle to achieve both completeness and efficiency in capturing task-critical information. Inspired by human progressive visual cognition, we propose CogniGPT, a framework that leverages an interactive loop between Multi-Granular Perception Agent (MGPA) and Verification-Enhanced Reflection Agent (VERA) for efficient and reliable long video understanding. Specifically, MGPA mimics human visual divergent and focused attention to capture task-related information, while VERA verifies perceived key clues to mitigate hallucination and optimize subsequent perception strategies. Through this interactive process, CogniGPT explores a minimal set of informative and reliable task-related clues. Extensive experiments on EgoSchema, Video-MME, NExT-QA, and MovieChat datasets demonstrate CogniGPT's superiority in both accuracy and efficiency. Notably, on EgoSchema, it surpasses existing training-free methods using only 11.2 frames and achieves performance comparable to Gemini 1.5-Pro.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
