用于视频检索的视频编辑
Video Editing for Video Retrieval
AI总结：
针对视频-文本检索中片段起止时间标注昂贵的问题，提出基于单一时间戳初始化并用学生-教师网络迭代编辑片段边界的模型无关方法，在三个模型和数据集上持续提升检索性能。
摘要：
尽管预训练视觉-语言模型已表明可从大规模网络视频中显著提升视频-文本检索性能，但使用带有开始和结束时间的人工标注片段进行微调仍发挥关键作用，而这需要大量人力。为解决该问题，我们探索一种成本更低的替代标注来源，即单一时间戳，用于视频-文本检索。我们以启发式方式从时间戳初始化片段，以预热检索模型。随后提出一种视频片段编辑方法，用于细化初始粗糙边界，从而提升检索性能。我们引入了一个用于视频片段编辑的学生-教师网络。教师模型用于编辑训练集中的片段，而学生模型在编辑后的片段上训练。当学生性能提升后，教师权重从学生权重更新。我们的方法与模型无关，可应用于任何检索模型。我们基于三种最先进的检索模型 COOT、VideoCLIP 和 CLIP4Clip 进行实验。在三个视频检索数据集 YouCook2、DiDeMo 和 ActivityNet-Captions 上开展的实验表明，在所有三种检索模型上，与初始片段相比，我们编辑后的片段均能持续提升检索性能。
英文摘要：
Though pre-training vision-language models have demonstrated significant benefits in boosting video-text retrieval performance from large-scale web videos, fine-tuning still plays a critical role with manually annotated clips with start and end times, which requires considerable human effort. To address this issue, we explore an alternative cheaper source of annotations, single timestamps, for video-text retrieval. We initialise clips from timestamps in a heuristic way to warm up a retrieval model. Then a video clip editing method is proposed to refine the initial rough boundaries to improve retrieval performance. A student-teacher network is introduced for video clip editing. The teacher model is employed to edit the clips in the training set whereas the student model trains on the edited clips. The teacher weights are updated from the student's after the student's performance increases. Our method is model agnostic and applicable to any retrieval models. We conduct experiments based on three state-of-the-art retrieval models, COOT, VideoCLIP and CLIP4Clip. Experiments conducted on three video retrieval datasets, YouCook2, DiDeMo and ActivityNet-Captions show that our edited clips consistently improve retrieval performance over initial clips across all the three retrieval models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
