LoSA：用于端到端时间动作定位扩展的长短程适配器
LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization
AI总结：
针对大型视频基础模型适配TAL时GPU内存过大的问题，提出LoSA这一首个专为TAL设计的内存和参数高效骨干适配器，通过长短程适配器及门控融合提升特征，在标准基准上显著优于现有方法。
摘要：
时间动作定位（TAL）涉及在未修剪视频中定位和分类动作片段。大型视频基础模型的出现使仅RGB视频骨干网络优于之前需要RGB和光流两种模态的方法。由于调整视频骨干网络以适应TAL需要极大的GPU内存，利用这些大型模型通常仅限于训练TAL头部。为克服这一限制，我们引入LoSA，这是首个专为TAL设计的内存和参数高效骨干适配器，用于处理未修剪视频。LoSA通过引入长短程适配器来专门适配TAL，这些适配器在不同时间范围上调整视频骨干网络的中间层。这些适配器与视频骨干网络并行运行，以显著减少内存占用。LoSA还包括长短程门控融合，战略性地结合视频骨干网络层中这些适配器的输出，以增强提供给TAL头部的视频特征。实验表明，LoSA通过将端到端骨干适配扩展到VideoMAEv2（ViT-g）等十亿参数以上的模型，并在仅头部迁移学习之外利用它们，在标准TAL基准THUMOS-14和ActivityNet-v1.3上显著优于所有现有方法。
英文摘要：
Temporal Action Localization (TAL) involves localizing and classifying action snippets in an untrimmed video. The emergence of large video foundation models has led RGB-only video backbones to outperform previous methods needing both RGB and optical flow modalities. Leveraging these large models is often limited to training only the TAL head due to the prohibitively large GPU memory required to adapt the video backbone for TAL. To overcome this limitation, we introduce LoSA, the first memory-and-parameter-efficient backbone adapter designed specifically for TAL to handle untrimmed videos. LoSA specializes for TAL by introducing Long-Short-range Adapters that adapt the intermediate layers of the video backbone over different temporal ranges. These adapters run parallel to the video backbone to significantly reduce memory footprint. LoSA also includes Long-Short-range Gated Fusion that strategically combines the output of these adapters from the video backbone layers to enhance the video features provided to the TAL head. Experiments show that LoSA significantly outperforms all existing methods on standard TAL benchmarks, THUMOS-14 and ActivityNet-v1.3, by scaling end-to-end backbone adaptation to billion-parameter-plus models like VideoMAEv2~(ViT-g) and leveraging them beyond head-only transfer learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
