arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.22918cs.CVcs.AIcs.CL
收藏

EVA: 为端到端视频代理的高效强化学习

EVA: Efficient Reinforcement Learning for End-to-End Video Agent

SenseTime Research(商汤科技研究院)

Yaolun Zhang, Ruohui Wang, Jiahao Wang, Yepeng Tang, Xuanyu Zheng, Haonan Duan, Hao Lu, Hanming Deng, Lewei Lu

AI总结:

EVA通过迭代总结-计划-行动-反思推理实现先规划后感知,提升长视频理解效率,采用三阶段学习流程和高质量数据集,在六个视频理解基准上取得显著提升。

摘要:

多模态大语言模型(MLLMs)在视频理解中面临挑战,因视频长token序列包含大量时间依赖和冗余帧。现有方法通常将MLLMs视为被动识别器,处理整个视频或均匀采样帧而不具备自适应推理能力。近期基于代理的方法引入外部工具,但仍依赖手动设计的工作流和感知优先策略,导致在长视频上效率低下。我们提出EVA,一种为端到端视频代理设计的高效强化学习框架,通过迭代总结-计划-行动-反思推理实现先规划后感知。EVA自主决定观看什么、何时观看以及如何观看,实现查询驱动和高效的视频理解。为训练此类代理,我们设计了一种简单而有效的三阶段学习流程——包括监督微调(SFT)、卡尼曼-特维斯基优化(KTO)和群体相对策略优化(GRPO)——连接监督模仿与强化学习。我们进一步构建了高质量的数据集以支持稳定且可重复的训练。我们在六个视频理解基准上评估了EVA,展示了其全面的能力。与现有基线相比,EVA在一般MLLM基线上实现了6-12%的显著提升,并在先前自适应代理方法上进一步获得1-3%的提升。

英文摘要:

Video understanding with multimodal large language models (MLLMs) remains challenging due to the long token sequences of videos, which contain extensive temporal dependencies and redundant frames. Existing approaches typically treat MLLMs as passive recognizers, processing entire videos or uniformly sampled frames without adaptive reasoning. Recent agent-based methods introduce external tools, yet still depend on manually designed workflows and perception-first strategies, resulting in inefficiency on long videos. We present EVA, an Efficient Reinforcement Learning framework for End-to-End Video Agent, which enables planning-before-perception through iterative summary-plan-action-reflection reasoning. EVA autonomously decides what to watch, when to watch, and how to watch, achieving query-driven and efficient video understanding. To train such agents, we design a simple yet effective three-stage learning pipeline - comprising supervised fine-tuning (SFT), Kahneman-Tversky Optimization (KTO), and Group Relative Policy Optimization (GRPO) - that bridges supervised imitation and reinforcement learning. We further construct high-quality datasets for each stage, supporting stable and reproducible training. We evaluate EVA on six video understanding benchmarks, demonstrating its comprehensive capabilities. Compared with existing baselines, EVA achieves a substantial improvement of 6-12% over general MLLM baselines and a further 1-3% gain over prior adaptive agent methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑