Q-Adapter：用于在视频字幕生成中提取文本相关特征的视觉查询适配器
Q-Adapter: Visual Query Adapter for Extracting Textually-related Features in Video Captioning
AI总结：
本文提出轻量级视觉适配器Q-Adapter，通过在Vision Encoder中引入可学习查询令牌和门控层，实现无需外部文本监督的高效微调，在MSR-VTT和MSVD数据集上仅用1.4%参数即达到SOTA性能。
摘要：
视频字幕生成的最新进展由大规模预训练模型驱动，这些模型遵循标准的“预训练后微调”范式，即对完整模型进行微调以适应下游任务。虽然有效，但随着模型规模增加，这种方法在计算上变得不可行。参数高效微调（PEFT）方法提供了一个有前景的替代方案，但主要集中在多模态大语言模型（MLLMs）的语言组件上。尽管最近取得了进展，PEFT在多模态任务中仍探索不足，且在微调模型时对视觉信息缺乏充分理解。为弥补这一差距，我们提出了Query-Adapter（Q-Adapter），这是一种轻量级视觉适配器模块，旨在通过为视频字幕生成任务实现高效微调来增强MLLMs。Q-Adapter在Vision Encoder中引入了可学习的查询令牌和门控层，能够在不依赖外部文本监督的情况下有效提取稀疏的、与字幕相关的特征。我们在两个知名的视频字幕生成数据集MSR-VTT和MSVD上评估了Q-Adapter，在采用PEFT方法的方法中，它在BLEU@4、METEOR、ROUGE-L和CIDEr指标上取得了最先进的性能。与采用完全微调方法相比，Q-Adapter也取得了具有竞争力的性能，同时仅需1.4%的参数。我们进一步分析了关键超参数和设计选择对微调有效性的影响，为基于适配器的学习提供了优化策略的见解。这些结果突显了Q-Adapter在平衡字幕质量和参数效率方面的强大潜力，证明了其在视频语言建模中的可扩展性。
英文摘要：
Recent advances in video captioning are driven by large-scale pretrained models, which follow the standard "pre-training followed by fine-tuning" paradigm, where the full model is fine-tuned for downstream tasks. Although effective, this approach becomes computationally prohibitive as the model size increases. The Parameter-Efficient Fine-Tuning (PEFT) approach offers a promising alternative, but primarily focuses on the language components of Multimodal Large Language Models (MLLMs). Despite recent progress, PEFT remains underexplored in multimodal tasks and lacks sufficient understanding of visual information during fine-tuning the model. To bridge this gap, we propose Query-Adapter (Q-Adapter), a lightweight visual adapter module designed to enhance MLLMs by enabling efficient fine-tuning for the video captioning task. Q-Adapter introduces learnable query tokens and a gating layer into Vision Encoder, enabling effective extraction of sparse, caption-relevant features without relying on external textual supervision. We evaluate Q-Adapter on two well-known video captioning datasets, MSR-VTT and MSVD, where it achieves state-of-the-art performance among the methods that take the PEFT approach across BLEU@4, METEOR, ROUGE-L, and CIDEr metrics. Q-Adapter also achieves competitive performance compared to methods that take the full fine-tuning approach while requiring only 1.4% of the parameters. We further analyze the impact of key hyperparameters and design choices on fine-tuning effectiveness, providing insights into optimization strategies for adapter-based learning. These results highlight the strong potential of Q-Adapter in balancing caption quality and parameter efficiency, demonstrating its scalability for video-language modeling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
