arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.07402cs.MMcs.AI
收藏

BDIQA:一个通过心智理论探索认知推理的视频问答新数据集

BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mind

East China Normal University(华东师范大学);Shanghai International Studies University(上海外国语大学)

Yuanyuan Mao, Xin Lin, Qin Ni, Liang He

AI总结:

本文提出首个基于心智理论(ToM)的视频问答基准BDIQA,通过评估信念、欲望和意图(BDI)推理能力,揭示预训练模型在认知推理上的不足,并基于消融分析提出两条增强认知推理的指导原则。

摘要:

作为认知智能的基础组成部分,心智理论(ToM)能使AI更接近人类思维过程,从而增强其与人类的互动和协作。特别是,它能显著提升模型对复杂场景视频的理解能力。然而,当前的视频问答(VideoQA)数据集侧重于研究事件内部的因果推理,极少有真正融入人类ToM的。因此,VideoQA领域缺乏ToM推理任务的发展。本文提出了BDIQA,这是首个在ToM背景下探索VideoQA模型认知推理能力的基准。BDIQA受儿童ToM认知发展的启发,解决了当前数据集和任务中机器ToM的不足。具体而言,它提供了两个难度级别的任务,在简单和复杂场景中评估信念、欲望和意图(BDI)推理。我们在几种主流的VideoQA方法上进行了评估,并通过零样本、少样本和监督学习诊断了它们的能力。我们发现,预训练模型在认知推理任务上的表现仍不令人满意。为应对这一挑战,我们进行了深入的分析和实验,最终基于消融分析提出了两条增强认知推理的指导原则。

英文摘要:

As a foundational component of cognitive intelligence, theory of mind (ToM) can make AI more closely resemble human thought processes, thereby enhancing their interaction and collaboration with human. In particular, it can significantly improve a model's comprehension of videos in complex scenes. However, current video question answer (VideoQA) datasets focus on studying causal reasoning within events few of them genuinely incorporating human ToM. Consequently, there is a lack of development in ToM reasoning tasks within the area of VideoQA. This paper presents BDIQA, the first benchmark to explore the cognitive reasoning capabilities of VideoQA models in the context of ToM. BDIQA is inspired by the cognitive development of children's ToM and addresses the current deficiencies in machine ToM within datasets and tasks. Specifically, it offers tasks at two difficulty levels, assessing Belief, Desire and Intention (BDI) reasoning in both simple and complex scenarios. We conduct evaluations on several mainstream methods of VideoQA and diagnose their capabilities with zero shot, few shot and supervised learning. We find that the performance of pre-trained models on cognitive reasoning tasks remains unsatisfactory. To counter this challenge, we undertake thorough analysis and experimentation, ultimately presenting two guidelines to enhance cognitive reasoning derived from ablation analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑