关注何处至关重要：面向长视频理解的策略内自蒸馏

中文摘要 AI 辅助

视觉-语言模型（VLMs）在长视频理解领域已取得显著进展，标准骨干模型通常对从整个视频中采样的帧进行问题回答。然而，随着视频时长增加，全视频上下文不可避免地包含更多与问题无关的时间内容，这会干扰模型获取回答特定问题所需的证据。我们通过实验发现，与使用对应全视频相比，将视觉输入聚焦于包含问题相关证据的短标注线索区间，可在各类模型规模下持续提升预测准确率，同时所需输入帧更少。基于这一发现，我们提出Clue-OPSD，一种面向长视频理解的线索特权策略内自蒸馏框架。训练期间，全视频学生模型通过对齐自身生成轨迹上的下一个token分布，从对应线索区间条件下的自教师模型中学习。Clue-OPSD将线索区间作为特权监督，无需依赖真实答案标签，且推理时无需线索标注或额外模块。在多个长视频理解基准及Qwen3.5模型规模上开展的大量实验表明，其相较于对应骨干模型实现了持续提升，且相较于监督后训练基线展现出强劲性能。

英文摘要

Vision-language models (VLMs) have made substantial progress in long-video understanding, with standard backbone models typically answering questions from frames sampled across the full video. However, as videos become longer, the full-video context inevitably contains more question-irrelevant temporal content, which can distract the model from the evidence needed to answer a specific question. We empirically find that focusing the visual input on short annotated clue intervals containing question-relevant evidence consistently improves prediction accuracy across model scales compared with using the corresponding full videos, while requiring fewer input frames. Based on this finding, we introduce Clue-OPSD, a clue-privileged on-policy self-distillation framework for long-video understanding. During training, a full-video student learns from a self-teacher conditioned on the corresponding clue interval by aligning their next-token distributions along student-generated trajectories. Clue-OPSD thus uses clue intervals as privileged supervision without relying on ground-truth answer labels, while requiring no clue annotations or additional modules at inference time. Extensive experiments across multiple long-video understanding benchmarks and Qwen3.5 model scales demonstrate consistent improvements over the corresponding backbone models and strong performance against supervised post-training baselines.

发表机构

• University of Maryland（马里兰大学）
• Johns Hopkins University（约翰斯·霍普金斯大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

15 pages, 8 figures, 6 tables