E-THER：用于共情AI的多模态数据集——迈向情感错位感知
E-THER: A Multimodal Dataset for Empathic AI -- Towards Emotional Mismatch Awareness
AI总结：
本文提出E-THER，首个基于人本主义疗法的多模态数据集，用于检测言语-视觉情绪错位，以训练真正共情的AI系统，实验表明错位训练模型在治疗对话质量上优于通用模型。
摘要：
当前共情AI系统的一个普遍缺陷是，它们无法识别言语表达可能并未完全反映潜在情绪状态的情况。这是因为用于训练这些系统的现有数据集侧重于表面层面的情绪识别，而未涉及对共情理解有用的复杂言语-视觉不一致（错位）模式。在本文中，我们提出了E-THER，这是首个以人本主义疗法（Person-Centered Therapy, PCT）为基础的多模态数据集，具有用于言语-视觉不一致检测的多维标注，使得训练具有真正而非表演性共情能力的AI系统成为可能。数据集中的标注源自人本主义方法，即识别来访者-咨询师互动中的言语-视觉情绪错位，从而形成一个用于训练和评估AI共情任务的框架。额外的参与度评分提供了用于研究应用的行为标注。在使用基于共情和治疗原则的评估指标时，最先进的视觉语言模型（VLMs），如IDEFICS和VideoLLAVA，在共情和治疗对话质量方面取得了显著提升。实证结果表明，我们经过错位训练的模型在关键特质上优于通用模型，例如维持治疗参与度、最小化人工或夸张的语言模式，以及保持对PCT理论框架的忠实度。
英文摘要：
A prevalent shortfall among current empathic AI systems is their inability to recognize when verbal expressions may not fully reflect underlying emotional states. This is because the existing datasets, used for the training of these systems, focus on surface-level emotion recognition without addressing the complex verbal-visual incongruence (mismatch) patterns useful for empathic understanding. In this paper, we present E-THER, the first Person-Centered Therapy-grounded multimodal dataset with multidimensional annotations for verbal-visual incongruence detection, enabling training of AI systems that develop genuine rather than performative empathic capabilities. The annotations included in the dataset are drawn from humanistic approach, i.e., identifying verbal-visual emotional misalignment in client-counsellor interactions - forming a framework for training and evaluating AI on empathy tasks. Additional engagement scores provide behavioral annotations for research applications. Notable gains in empathic and therapeutic conversational qualities are observed in state-of-the-art vision-language models (VLMs), such as IDEFICS and VideoLLAVA, using evaluation metrics grounded in empathic and therapeutic principles. Empirical findings indicate that our incongruence-trained models outperform general-purpose models in critical traits, such as sustaining therapeutic engagement, minimizing artificial or exaggerated linguistic patterns, and maintaining fidelity to PCT theoretical framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
