$R^2$-Tuning：面向视频时间定位的高效图像到视频迁移学习
$R^2$-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
AI总结：
提出$R^2$-Tuning，一种参数高效的图像到视频迁移学习方法，仅用1.5%参数实现从粗到细的时空建模，在六个基准上取得视频时间定位最先进性能。
摘要：
视频时间定位（VTG）是一个细粒度的视频理解问题，旨在根据自然语言查询在未修剪的视频中定位相关片段。大多数现有的VTG模型基于逐帧的最终层CLIP特征，并辅以额外的时间骨干网络（如SlowFast）和复杂的时间推理机制。在这项工作中，我们声称CLIP本身已经展现出在细粒度时空建模方面的巨大潜力，因为每一层在不同粒度级别下都提供了独特而有用的信息。基于此，我们提出了反向循环微调（$R^2$-Tuning），一种用于视频时间定位的参数和内存高效的迁移学习框架。我们的方法学习一个轻量级的$R^2$块，仅包含总参数的1.5%，以进行渐进式的时空建模。从CLIP的最后一层开始，$R^2$块循环地聚合来自更早层的空间特征，然后根据给定查询细化时间相关性，形成从粗到细的方案。$R^2$-Tuning在三个VTG任务（即时刻检索、高光检测和视频摘要）上的六个公共基准（即QVHighlights、Charades-STA、Ego4D-NLQ、TACoS、YouTube Highlights和TVSum）上取得了最先进的性能，即使没有额外的骨干网络，也证明了所提出方案的重要性和有效性。我们的代码可在https://github.com/yeliudev/R2-Tuning获取。
英文摘要：
Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries. Most existing VTG models are built upon frame-wise final-layer CLIP features, aided by additional temporal backbones (e.g., SlowFast) with sophisticated temporal reasoning mechanisms. In this work, we claim that CLIP itself already shows great potential for fine-grained spatial-temporal modeling, as each layer offers distinct yet useful information under different granularity levels. Motivated by this, we propose Reversed Recurrent Tuning ($R^2$-Tuning), a parameter- and memory-efficient transfer learning framework for video temporal grounding. Our method learns a lightweight $R^2$ Block containing only 1.5% of the total parameters to perform progressive spatial-temporal modeling. Starting from the last layer of CLIP, $R^2$ Block recurrently aggregates spatial features from earlier layers, then refines temporal correlation conditioning on the given query, resulting in a coarse-to-fine scheme. $R^2$-Tuning achieves state-of-the-art performance across three VTG tasks (i.e., moment retrieval, highlight detection, and video summarization) on six public benchmarks (i.e., QVHighlights, Charades-STA, Ego4D-NLQ, TACoS, YouTube Highlights, and TVSum) even without the additional backbone, demonstrating the significance and effectiveness of the proposed scheme. Our code is available at https://github.com/yeliudev/R2-Tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
