口语对话中的上下文感知多模态声明验证
Context-Aware Multimodal Claim Verification in Spoken Dialogues
AI总结：
提出MAD2基准和上下文感知多模态融合方法，验证对话音频中的声明，发现对话结构比虚假信息框架对验证更重要。
摘要：
每天，数百万人从播客和流媒体中吸收声明，而这些声明从未被事实核查员看到。口语错误信息是通过对话构建的，其中可信度不仅来自事实本身，还来自声明如何在对话轮次中被构建、强化或未被质疑。然而，事实核查一直专注于孤立的文本，对话音频研究不足。我们引入了MAD2，一个新的用于口语声明验证的多轮音频对话基准，包含1,000个双说话者对话，3,368个值得核查的声明和约10小时的音频，并提出了上下文感知音频编码器和对话感知文本模型的校准多模态融合。在各种设置下，添加对话上下文改善了验证，但收益取决于场景类型。仅使用前文上下文通常与离线性能相当，支持实时审核设置，而当基于转录的模型被额外上下文 destabilized 时，音频贡献最大。总体而言，对话结构对验证的影响比错误信息框架更大。
英文摘要：
Spoken factual claims often occur within multi-turn conversations, where surrounding dialogue can provide context unavailable from the claim alone. Yet most fact-checking research evaluates isolated text, leaving conversational audio under-studied. We introduce MAD2, a synthetic Multi-turn Audio Dialogues benchmark for spoken claim verification with 1,000 two-speaker dialogues, 1,230 sentence-level check-worthy candidate annotations, and approximately 10 hours of audio. We also propose calibrated multimodal fusion of a context-aware audio encoder and a dialogue-aware text model. Adding dialogue context improves verification across settings, although the gains vary by scenario. Past-only context often approaches local offline performance, suggesting its usefulness when future context is unavailable. Fusion achieves its highest mean advantage over text with full-dialogue context, but does not consistently or significantly outperform text across settings. Under full-dialogue context, exploratory subgroup results show greater performance variation across dialogue scenarios for text and fusion, but across spread styles for audio.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
