CAREBench: 通过评估认知评价推理来评估LLMs的情感理解
CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoning
AI总结：
本文提出CAREBench，首个全面标注认知评价推理、评价评分和多标签情感标注的基准，通过系统实验发现更强模型在某些任务上匹配或超越人类，但在评价推理和积极情绪识别上表现不足，揭示了当前模型未能内部化捕捉人类主观异质性的机制。
摘要：
情感理解是LLMs有效与人类交互的核心能力，但现有评估方法依赖离散情绪标签预测，无法捕捉情绪生成的认知过程。基于评价理论，我们引入CAREBench，首个包含从第一和第三人称视角对现实叙述的完整推断链标注的基准，涵盖评价推理、评价评分和多标签情感标注。我们提出一个过程级评估框架，并在六个LLMs上围绕四个研究问题开展系统实验。我们发现，更强的模型在某些任务上匹配或超越人类观察者，但在评价推理和积极情绪识别上表现不足；跨步骤性能和对评价干预的敏感性在不同模型间表现出差异；当前模型尚未内部化捕捉人类主观异质性的机制。这些发现表明，下游情绪预测指标可能高估LLMs的真实情感理解，而CAREBench为更具有诊断信息的LLMs情感认知能力评估提供了基础。
英文摘要：
Emotion understanding is a core capability for LLMs to interact effectively with humans, yet existing evaluation paradigms rely on discrete emotion label prediction and fail to capture the cognitive processes underlying emotion generation. Grounded in appraisal theory, we introduce CAREBench, the first benchmark with complete inferential chain annotations from both first- and third-person perspectives on real-world narratives, spanning appraisal reasoning, appraisal ratings, and multi-label emotion annotation. We propose a process-level evaluation framework and conduct systematic experiments across six LLMs organized around four research questions. We find that stronger models match or surpass human observers on certain tasks, yet fall short on appraisal reasoning and positive emotion recognition; performance across chain steps and sensitivity to appraisal interventions exhibit dissociations across models; and current models have not internalized the mechanisms needed to capture human subjective heterogeneity. These findings suggest that downstream emotion prediction metrics may overestimate LLMs' true emotion understanding, and CAREBench provides a foundation for more diagnostically informative evaluation of LLMs' affective cognitive capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
