MLLMs能察言观色吗？一个用于验证多方社交互动中真实性的多模态基准
Can MLLMs Read the Room? A Multimodal Benchmark for Verifying Truthfulness in Multi-Party Social Interactions
AI总结：
本文提出了多方社交互动真实性评估任务MIVA及基于《狼人杀》的多模态数据集，评估发现GPT-4o等先进模型难以可靠辨别真伪，且在视觉线索对齐上存在不足。
摘要：
随着AI系统日益融入人类生活，赋予其强大的社会智能已成为关键前沿。这种智能的关键在于从欺骗中辨别真相，这是人类互动中普遍存在的元素，通过口头语言和非语言视觉线索的复杂交织来传递。然而，在动态的多方对话中进行自动欺骗检测仍是一项重大挑战。近期强大的多模态大语言模型（MLLMs）的兴起，凭借其在视觉和文本理解方面的出色能力，使其成为该任务的自然候选者。因此，它们在这一关键领域的能力大多未被量化。为填补这一空白，我们引入了一项新任务——多模态互动真实性评估（MIVA），并提出了一个源自社交推理游戏《狼人杀》的新型多模态数据集。该数据集提供了同步的视频、文本，以及每条陈述可验证的真实标签。我们建立了一个综合基准来评估最先进的MLLMs，揭示了显著的性能差距：即使是像GPT-4o这样强大的模型也难以可靠地辨别真伪。我们对失败模式的分析表明，这些模型未能有效地将语言与视觉社交线索联系起来，并且在其对齐上可能过于保守，凸显了构建更具感知力和可信赖的AI系统对新方法的迫切需求。
英文摘要：
As AI systems become increasingly integrated into human lives, endowing them with robust social intelligence has emerged as a critical frontier. A key aspect of this intelligence is discerning truth from deception, a ubiquitous element of human interaction that is conveyed through a complex interplay of verbal language and non-verbal visual cues. However, automatic deception detection in dynamic, multi-party conversations remains a significant challenge. The recent rise of powerful Multimodal Large Language Models (MLLMs), with their impressive abilities in visual and textual understanding, makes them natural candidates for this task. Consequently, their capabilities in this crucial domain are mostly unquantified. To address this gap, we introduce a new task, Multimodal Interactive Veracity Assessment (MIVA), and present a novel multimodal dataset derived from the social deduction game Werewolf. This dataset provides synchronized video, text, with verifiable ground-truth labels for every statement. We establish a comprehensive benchmark evaluating state-of-the-art MLLMs, revealing a significant performance gap: even powerful models like GPT-4o struggle to distinguish truth from falsehood reliably. Our analysis of failure modes indicates that these models fail to ground language in visual social cues effectively and may be overly conservative in their alignment, highlighting the urgent need for novel approaches to building more perceptive and trustworthy AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
