语言模型是零样本文本视频问答的因果知识提取器
Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering
AI总结：
针对现有QG模型难以生成因果问题导致CVidQA性能不佳的问题，提出CaKE-LM框架，利用语言模型提取因果常识知识生成问答对，在零样本准确率上提升4%至6%。
摘要：
因果视频问答（CVidQA）不仅查询视频中的关联或时序关系，还查询因果关系。现有的问题合成方法在带有文本描述输入的阅读理解数据集上预训练问题生成（QG）系统。然而，QG模型仅学会提出关联问题（例如，“某人在做什么……”），由于关联知识向专注于“为什么某人做……”等因果问题的CVidQA迁移效果差，导致性能不佳。观察到这一点，我们提出利用因果知识生成问答对，并提出了一种新颖的框架——从语言模型中提取因果知识（CaKE-LM），利用语言模型中的因果常识知识来解决CVidQA。为了从LM中提取知识，CaKE-LM通过提示LM动作（足球运动员踢球）来检索意图（进球得分），生成包含两个事件且一个触发另一个（例如，“进球得分”触发“足球运动员踢球”）的因果问题。在NExT-QA和Causal-VidQA数据集上，CaKE-LM在零样本CVidQA准确率上显著优于传统方法4%至6%。我们还进行了全面分析，并为未来的研究提供了关键发现。
英文摘要：
Causal Video Question Answering (CVidQA) queries not only association or temporal relations but also causal relations in a video. Existing question synthesis methods pre-trained question generation (QG) systems on reading comprehension datasets with text descriptions as inputs. However, QG models only learn to ask association questions (e.g., ``what is someone doing...'') and result in inferior performance due to the poor transfer of association knowledge to CVidQA, which focuses on causal questions like ``why is someone doing ...''. Observing this, we proposed to exploit causal knowledge to generate question-answer pairs, and proposed a novel framework, Causal Knowledge Extraction from Language Models (CaKE-LM), leveraging causal commonsense knowledge from language models to tackle CVidQA. To extract knowledge from LMs, CaKE-LM generates causal questions containing two events with one triggering another (e.g., ``score a goal'' triggers ``soccer player kicking ball'') by prompting LM with the action (soccer player kicking ball) to retrieve the intention (to score a goal). CaKE-LM significantly outperforms conventional methods by 4% to 6% of zero-shot CVidQA accuracy on NExT-QA and Causal-VidQA datasets. We also conduct comprehensive analyses and provide key findings for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
