VideoCuRL: 通过正交难度分解实现视频课程强化学习
VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition
AI总结：
VideoCuRL通过正交分解视频理解中的视觉复杂性和认知复杂性，提出了一种新的课程强化学习框架，提升了视频推理和感知任务的性能。
摘要：
强化学习（RL）对于赋能视频大语言模型（VideoLLMs）进行复杂的时空推理至关重要。然而，当前的RL范式主要依赖于随机数据洗牌或基于标量难度指标的简单课程策略。我们主张标量指标无法区分视频理解中的两个正交挑战：视觉时间感知负荷和认知推理深度。为此，我们提出了VideoCuRL，一种新的框架，将难度分解为这两个轴。我们采用高效且无需训练的代理，即光流和关键帧熵用于视觉复杂性，校准的惊奇度用于认知复杂性，将数据映射到二维课程网格上。随后，一种具有能力意识的对角波前策略则从基础对齐到复杂推理调度训练。此外，我们引入了动态稀疏KL和结构化重访以稳定训练以对抗奖励崩溃和灾难性遗忘。广泛的实验表明，VideoCuRL在推理任务（在VSI-Bench上+2.5）和感知任务（在VideoMME上+2.9）上超过了强大的RL基线。值得注意的是，VideoCuRL消除了基于生成的课程的不可接受的推理开销，为视频预训练后的稳健性提供了一种可扩展的解决方案。
英文摘要：
Reinforcement Learning (RL) is crucial for empowering VideoLLMs with complex spatiotemporal reasoning. However, current RL paradigms predominantly rely on random data shuffling or naive curriculum strategies based on scalar difficulty metrics. We argue that scalar metrics fail to disentangle two orthogonal challenges in video understanding: Visual Temporal Perception Load and Cognitive Reasoning Depth. To address this, we propose VideoCuRL, a novel framework that decomposes difficulty into these two axes. We employ efficient, training-free proxies, optical flow and keyframe entropy for visual complexity, Calibrated Surprisal for cognitive complexity, to map data onto a 2D curriculum grid. A competence aware Diagonal Wavefront strategy then schedules training from base alignment to complex reasoning. Furthermore, we introduce Dynamic Sparse KL and Structured Revisiting to stabilize training against reward collapse and catastrophic forgetting. Extensive experiments show that VideoCuRL surpasses strong RL baselines on reasoning (+2.5 on VSI-Bench) and perception (+2.9 on VideoMME) tasks. Notably, VideoCuRL eliminates the prohibitive inference overhead of generation-based curricula, offering a scalable solution for robust video post-training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
