Video-RSI：通过工具进化实现视频理解代理的递归自我改进
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
- Tsinghua University（清华大学）
- Harbin Engineering University（哈尔滨工程大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
Video-RSI提出一种递归自我改进框架，让视频理解代理利用自身语言模型修订工具，通过主动视频调查验证失败假设，在成本感知下实现更少帧数下的更高准确率。
AI中文摘要：
视频理解代理通过一个可执行的工具来获取证据，该工具控制它们观察什么以及如何使用这些观察结果。然而，执行轨迹仅包含当前工具所获取的证据，使得对失败的竞争性解释无法解决，并限制了自我改进的基础。我们引入了Video-RSI，一个用于递归自我改进的框架，其中视频理解代理使用其自身的语言模型来修订其工具。通过主动的视频调查，模型重新访问原始训练视频，以额外的观察来测试竞争性的失败解释，将提议的更改基于超出现有轨迹的证据。成本感知的工具进化将这些诊断转化为可复用的修订，并通过考虑答案准确性和视觉成本来决定保留哪些修订。在我们对视频理解基准的评估设置中，进化后的代理在处理更少帧的同时提高了准确性，并实现了与现有视频理解代理相比具有竞争力的准确性-效率权衡。这些结果证明了视频理解代理通过工具进化来改进其自身证据获取和使用的潜力。代码可在该https URL获取。
英文摘要：
Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations. However, execution traces contain only the evidence acquired by the current harness, leaving competing explanations for failure unresolved and limiting the basis for self-improvement. We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness. Through active video investigation, the model revisits the original training videos to test competing failure explanations with additional observations, grounding proposed changes in evidence beyond the existing trace. Cost-aware harness evolution turns these diagnoses into reusable revisions and determines which revisions to retain by considering both answer accuracy and visual cost. Across our evaluation settings on video understanding benchmarks, the evolved agent improves accuracy while processing fewer frames and achieves competitive accuracy-efficiency trade-offs against existing video understanding agents. These results demonstrate the potential for video understanding agents to improve their own evidence acquisition and use through harness evolution. Code is available at https://github.com/bingjunluo/Video-RSI .
