推理流是否重要？评估Gemini视觉-语言模型在视频场景理解中的推理能力
Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding
AI总结：
研究评估了Gemini视频语言模型中推理流对视频场景理解的影响，提出三个评估指标，并发现增加推理量对输出质量的提升迅速达到平台期，Flash Lite在质量和token使用之间取得最佳平衡。
摘要：
我们评估了内部推理轨迹（称为推理流）如何影响视觉-语言模型在视频场景理解中的表现。使用Google的Gemini 2.5 Flash和Flash Lite四种配置，从100小时视频中提取的场景进行测试，探讨了更多推理是否带来更好输出、收益何时停止以及模型实际思考内容。我们引入了三个评估指标：内容性衡量推理流中有用场景内容与元评论的比例；推理流覆盖度衡量推理流对最终输出的忠实程度；主导实体分析识别模型关注的主体、动作和场景。GPT-5作为独立评判者。研究发现，增加推理量对输出质量的提升迅速达到平台期，大部分改进发生在前几百个token中。Flash Lite在质量和token使用之间提供了最佳平衡。紧缩的推理预算导致模型在最终输出中添加其从未推理过的内容，这是一种压缩步骤幻觉。尽管是不同模型层级，Flash和Flash Lite产生相似的推理流，但风格不同：Flash讨论其推理过程，而Lite则专注于描述场景。
英文摘要：
We benchmark how internal reasoning traces, which we call thought streams, affect video scene understanding in vision-language models. Using four configurations of Google's Gemini 2.5 Flash and Flash Lite across scenes extracted from 100 hours of video, we ask three questions: does more thinking lead to better outputs, where do the gains stop, and what do these models actually think about? We introduce three evaluation metrics. Contentfulness measures how much of the thought stream is useful scene content versus meta-commentary. Thought-Final Coverage measures how faithfully the thought stream translates into the final output. Dominant Entity Analysis identifies which subjects, actions, and settings the model focuses on. GPT-5 serves as an independent judge. We find that quality gains from additional thinking plateau quickly, with most improvement occurring in the first few hundred tokens. Flash Lite offers the best balance between quality and token usage. Tight reasoning budgets cause the model to add content in the final output that it never reasoned about, a form of compression-step hallucination. Despite being different model tiers, Flash and Flash Lite produce similar thought streams, though they differ in style: Flash discusses its reasoning process, while Lite focuses on describing the scene.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
