长话短说：一种用于长视频问答的先总结后搜索方法
Long Story Short: a Summarize-then-Search Method for Long Video Question Answering
AI总结：
针对长视频叙事问答，提出Long Story Short框架，先总结情节再检索相关片段，并用CLIPCheck增强视觉匹配，在零样本设定下大幅超越监督模型。
摘要：
诸如GPT-3这样的大型语言模型已经展现出令人印象深刻的能力，能够在不需要任务特定训练数据的情况下适应新任务。这种能力在叙事问答等场景中尤为有效，因为这类场景中任务多样性极大，但可用的监督数据却很少。在这项工作中，我们研究这类语言模型能否将其零样本推理能力扩展到多媒体内容（如戏剧、电影和动画）中的长多模态叙事，在这些内容中故事起着至关重要的作用。我们提出Long Story Short，这是一个用于叙事视频问答的框架，该框架首先将视频的叙事总结为简短的情节，然后搜索与问题相关的视频片段。我们还提出利用CLIPCheck来增强视觉匹配。我们的模型大幅超越了最先进的监督模型，突显了零样本问答在长视频上的潜力。
英文摘要：
Large language models such as GPT-3 have demonstrated an impressive capability to adapt to new tasks without requiring task-specific training data. This capability has been particularly effective in settings such as narrative question answering, where the diversity of tasks is immense, but the available supervision data is small. In this work, we investigate if such language models can extend their zero-shot reasoning abilities to long multimodal narratives in multimedia content such as drama, movies, and animation, where the story plays an essential role. We propose Long Story Short, a framework for narrative video QA that first summarizes the narrative of the video to a short plot and then searches parts of the video relevant to the question. We also propose to enhance visual matching with CLIPCheck. Our model outperforms state-of-the-art supervised models by a large margin, highlighting the potential of zero-shot QA for long videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
