arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.17045cs.CVcs.AIcs.LG
收藏

无需训练的视频推理

Video Reasoning without Training

Qualcomm AI Research(高通AI研究);University of California, San Diego(加州大学圣地亚哥分校)

Deepak Sridhar, Kartikeya Bhardwaj, Jeya Pradha Jeyaraj, Nuno Vasconcelos, Ankita Nayak, Harris Teague

AI总结:

提出V-Reason方法,利用输出分布熵作为信号,通过轻量级控制器在推理时自适应调整值缓存,无需强化学习或微调即可提升视频推理性能。

摘要:

使用大型多模态模型(LMM)进行视频推理依赖于昂贵的强化学习(RL)和冗长的思维链,导致训练和推理过程中产生大量计算开销。此外,这些推理模型中控制思维过程的机制非常有限。在本文中,我们利用模型输出分布的熵作为信号来研究和指导推理行为。我们发现高质量模型表现出微探索和微利用循环的特征模式,随后出现后期熵峰值(即更长的思考)和较低的最终熵,表明更谨慎的探索和自信的收敛(即当模型探索或思考答案时避免过度随机性)。然后,我们利用这些新颖的、有理论基础的见解,引入了V-Reason(Video-Reason),一种推理时优化方法,通过轻量级、可训练的控制器自适应调整LMM的值缓存。我们提出的控制器由基于熵的目标引导,直接在推理时调整模型行为,无需使用任何RL或监督微调。我们的实验表明,V-Reason在许多视频推理数据集上显著优于基础指令调优模型,将与RL模型的差距平均缩小到0.6%的准确率以内。我们在无需任何训练的情况下实现了这一点,同时提供了效率优势:V-Reason使用的token比RL模型少58.6%。项目页面:https://deepaksridhar.github.io/vreason.github.io/

英文摘要:

Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, we use the entropy of the model's output distribution as a signal to study and guide reasoning behavior. We discover that high-quality models exhibit a characteristic pattern of micro-exploration and micro-exploitation cycles, followed by a later entropy peak (i.e., longer thinking) and a lower final entropy, indicating more deliberate exploration and confident convergence (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We then use these novel, theoretically-grounded insights to introduce V-Reason (Video-Reason), an inference-time optimization method that adapts the value cache of the LMM through a lightweight, trainable controller. Our proposed controller is guided by an entropy-based objective, to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Our experiments show that V-Reason significantly outperforms the base instruction-tuned models on many video reasoning datasets, narrowing the gap with RL models to within 0.6% accuracy on average. We achieve this without any training, while offering efficiency benefits: V-Reason uses 58.6% fewer tokens than the RL model. Project Page https://deepaksridhar.github.io/vreason.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑