提示大型语言模型重构时刻定位查询
Prompting Large Language Models to Reformulate Queries for Moment Localization
AI总结：
本文针对未裁剪视频时刻定位中查询难以被模型理解的问题，提出利用大型语言模型将原始查询重构为更友好的指令集。
摘要：
时刻定位任务旨在针对给定的自然语言查询，在未裁剪视频中定位一个时间时刻。由于未裁剪视频包含高度冗余的内容，查询质量对于准确定位时刻至关重要，即查询应提供有关目标时刻的精确信息，使定位模型能够理解要在视频中寻找什么。然而，当前数据集中的自然语言查询对于现有模型而言可能并不容易理解。例如，Ego4D数据集使用疑问句作为查询来描述相对复杂的时刻。尽管这些疑问句对人类来说自然且直白，但对于2D-TAN等主流时刻定位模型而言，理解此类疑问句颇具挑战。受大型语言模型近期成功的启发，尤其是其理解和生成复杂自然语言内容的能力，在这篇扩展摘要中，我们进行了早期尝试：利用大型语言模型将时刻查询重构为一组指令，使其对定位模型更加友好。
英文摘要：
The task of moment localization is to localize a temporal moment in an untrimmed video for a given natural language query. Since untrimmed video contains highly redundant contents, the quality of the query is crucial for accurately localizing moments, i.e., the query should provide precise information about the target moment so that the localization model can understand what to look for in the videos. However, the natural language queries in current datasets may not be easy to understand for existing models. For example, the Ego4D dataset uses question sentences as the query to describe relatively complex moments. While being natural and straightforward for humans, understanding such question sentences are challenging for mainstream moment localization models like 2D-TAN. Inspired by the recent success of large language models, especially their ability of understanding and generating complex natural language contents, in this extended abstract, we make early attempts at reformulating the moment queries into a set of instructions using large language models and making them more friendly to the localization models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
