arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.13359cs.CVcs.CL
收藏

面向时序视频定位中预训练语言模型参数高效集成的研究

Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding

National Institute of Advanced Industrial Science Technology(国立先进工业科学技术研究所);Ochanomizu University(御茶水女子大学);The University of Tokyo(东京大学)

Erica K. Shimomoto, Edison Marrese-Taylor, Hiroya Takamura, Ichiro Kobayashi, Hideki Nakayama, Yusuke Miyao

AI总结:

本文研究预训练语言模型在时序视频定位中的作用,并验证NLP适配器作为参数高效替代方案的有效性,在不改变视觉输入的情况下显著提升模型性能。

摘要:

本文探讨了时序视频定位(Temporal Video Grounding, TVG)任务,即给定一段未剪辑视频和一个自然语言句子查询,目标是识别并确定视频中由查询所描述的动作实例的时间边界。近期的工作通过使用大型预训练语言模型(PLM)改进查询输入来解决这一任务,但代价是训练成本更高。然而,这种集成的效果尚不明确,因为这些工作同时还提出了视觉输入方面的改进。因此,本文研究了PLM在TVG中的效果,并评估了使用NLP适配器进行参数高效训练的适用性。我们将流行的PLM与一系列现有方法相结合,并测试了不同的适配器,以减少额外参数带来的影响。我们在三个具有挑战性的数据集上的结果表明,在不改变视觉输入的情况下,TVG模型从PLM集成和微调中获益显著,这强调了句子查询表示在该任务中的重要性。此外,尽管NLP适配器并非针对我们的任务定制,它们仍是完全微调的有效替代方案,使得PLM能够集成到更大的TVG模型中,并取得与SOTA模型相当的结果。最后,我们的结果揭示了在不同场景下哪些适配器效果最佳。

英文摘要:

This paper explores the task of Temporal Video Grounding (TVG) where, given an untrimmed video and a natural language sentence query, the goal is to recognize and determine temporal boundaries of action instances in the video described by the query. Recent works tackled this task by improving query inputs with large pre-trained language models (PLM) at the cost of more expensive training. However, the effects of this integration are unclear, as these works also propose improvements in the visual inputs. Therefore, this paper studies the effects of PLMs in TVG and assesses the applicability of parameter-efficient training with NLP adapters. We couple popular PLMs with a selection of existing approaches and test different adapters to reduce the impact of the additional parameters. Our results on three challenging datasets show that, without changing the visual inputs, TVG models greatly benefited from the PLM integration and fine-tuning, stressing the importance of sentence query representation in this task. Furthermore, NLP adapters were an effective alternative to full fine-tuning, even though they were not tailored to our task, allowing PLM integration in larger TVG models and delivering results comparable to SOTA models. Finally, our results shed light on which adapters work best in different scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑