arXiv 2605.310292026-08-20cs.CV

PEEK: 通过高效知识蒸馏提取关键帧

PEEK: Picking Essential frames via Efficient Knowledge distillation

Télécom SudParis — SAMOVAR（Telecom SudParis — SAMOVAR）；Institut Polytechnique de Paris（巴黎政治学院）；Moments Lab

Killian Steunou, Anas Filali Razzouki, Khalil Guetari, Mounîm A. El-Yacoubi, Yannis Tevissen

备注：: Accepted at BMVC 2026. Supplementary material at https://www.killian-steunou.com/peek/static/pdfs/peek_supplementary.pdf

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出PEEK方法，通过知识蒸馏将教师模型的帧相关性排名迁移至轻量级时序模型，实现高效动态帧采样，在低帧预算下显著提升视频字幕生成性能。

摘要：

视频语言模型只能处理有限数量的帧，使得帧选择成为高效视频字幕生成的关键瓶颈。大多数字幕生成流程仍依赖均匀采样，该方法计算成本低但忽略视觉内容。自适应帧采样最近成为从视频中选择最具信息量帧的有前景方法，但现有方法计算成本仍然高昂。我们提出PEEK，一种高效的动态帧采样方法，它将字幕条件帧相关性排名从更强的教师模型蒸馏到仅基于视觉内容运行的轻量级时序模型中。我们发现，总体而言，在ActivityNet Captions和MSR-VTT上，我们的方法在所有评估的下游视觉语言模型中优于最先进方法，特别是当仅选择一或两帧进行字幕生成时，在大多数帧预算下获得最佳CIDEr分数。在ActivityNet Captions上，PEEK尤其强大，在16个配置中赢得14个。在MSR-VTT上的零样本评估表明，我们的模型在低帧预算下迁移效果最佳，而在四帧和八帧时结果更为混合，因为时间覆盖和视觉多样性变得更具竞争力。与最近的自适应基线相比，PEEK在低预算场景下更准确且更高效：它仅增加5.2%的字幕生成时间，而CSTA增加65.4%，MaxInfo增加211.9%。我们在https://github.com/momentslab/peek发布代码和预训练检查点。

英文摘要：

Video-language models can process only a limited number of frames, making frame selection a key bottleneck for efficient video captioning. Most captioning pipelines still rely on uniform sampling, which is computationally cheap but agnostic to visual content. Adaptive frame sampling selects the most informative frames from a video, but existing methods remain computationally expensive. We introduce PEEK, an efficient dynamic frame sampling method that distills caption-conditioned frame relevance rankings from a stronger teacher model into a lightweight temporal model that operates only on visual content. On ActivityNet Captions and MSR-VTT, PEEK outperforms state-of-the-art methods across all evaluated downstream vision language models, obtaining the best CIDEr for most frame budgets, especially when only one or two frames are selected, winning 14 out of 16 configurations on ActivityNet Captions. Zero-shot on MSR-VTT, it transfers best at low frame budgets, while results at four and eight frames are more mixed as temporal coverage and visual diversity become increasingly competitive. Compared with recent adaptive baselines, PEEK is both more accurate in the low-budget regime and more efficient: it adds only $5.2\%$ to the captioning time, compared with $65.4\%$ for CSTA and $211.9\%$ for MaxInfo. We release our code and pre-trained checkpoint at https://github.com/momentslab/peek.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。