FunQA：迈向令人惊喜的视频理解
FunQA: Towards Surprising Video Comprehension
AI总结：
本文提出了FunQA数据集，用于评估和增强模型对反直觉趣味视频的推理能力，并提出FunMentor智能体通过多轮对话提升视觉语言模型对反直觉性的理解。
摘要：
令人惊喜的视频，如搞笑片段、创意表演或视觉错觉，吸引了大量关注。对这些视频的欣赏并非仅仅是对视觉刺激的简单反应；相反，它取决于人类理解（并欣赏）这些视频中所描绘的常识违背的能力。我们引入了FunQA，一个具有挑战性的视频问答（QA）数据集，专门设计用于基于反直觉和趣味视频评估和增强视频推理的深度。与大多数关注较少惊喜情境（如烹饪或教学视频）的视频QA基准不同，FunQA涵盖了三种此前未被探索的令人惊喜的视频类型：1）HumorQA，2）CreativeQA，以及3）MagicQA。对于每个子集，我们建立了严格的QA任务，旨在评估模型在反直觉时间戳定位、详细视频描述以及围绕反直觉性进行推理方面的能力。我们还提出了更高层次的任务，例如为视频赋予贴切生动的标题以及对视频创意进行评分。FunQA基准总共包含源自4.3K个视频片段的312K个自由文本QA对，总时长达24小时。此外，我们提出了FunMentor，一种专为视觉语言模型（VLMs）设计的智能体，它使用多轮对话来增强模型对反直觉性的理解。对现有VLMs的大量实验证明了FunMentor的有效性，并揭示了在空间-时间推理、以视觉为中心的推理和自由文本生成方面，FunQA视频存在显著的性能差距。
英文摘要：
Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention. Enjoyment of these videos is not simply a response to visual stimuli; rather, it hinges on the human capacity to understand (and appreciate) commonsense violations depicted in these videos. We introduce FunQA, a challenging video question-answering (QA) dataset specifically designed to evaluate and enhance the depth of video reasoning based on counter-intuitive and fun videos. Unlike most video QA benchmarks which focus on less surprising contexts, e.g., cooking or instructional videos, FunQA covers three previously unexplored types of surprising videos: 1) HumorQA, 2) CreativeQA, and 3) MagicQA. For each subset, we establish rigorous QA tasks designed to assess the model's capability in counter-intuitive timestamp localization, detailed video description, and reasoning around counter-intuitiveness. We also pose higher-level tasks, such as attributing a fitting and vivid title to the video and scoring the video creativity. In total, the FunQA benchmark consists of 312K free-text QA pairs derived from 4.3K video clips, spanning a total of 24 video hours. Moreover, we propose FunMentor, an agent designed for Vision-Language Models (VLMs) that uses multi-turn dialogues to enhance models' understanding of counter-intuitiveness. Extensive experiments with existing VLMs demonstrate the effectiveness of FunMentor and reveal significant performance gaps for the FunQA videos across spatial-temporal reasoning, visual-centered reasoning, and free-text generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
