HEART：一个评估人类和大语言模型在情感支持对话中能力的统一基准
HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue
AI总结：
HEART通过多轮情感支持对话评估人类与大语言模型的能力差异，揭示两者在共情、一致性等维度上的表现及趋同趋势。
摘要：
支持性对话依赖于超越语言流利度的技能，包括读取情绪、调整语气以及处理抵抗、沮丧或痛苦的时刻。尽管语言模型取得了快速进展，我们仍然缺乏清晰的方法来理解它们在这些人际领域的能力如何与人类相比。我们介绍了HEART，这是首个直接在多轮情感支持对话中比较人类和LLM的框架。对于每段对话历史，我们配对人类和模型的回应，并通过盲评人类评估者和一组LLM作为评判者进行评估。所有评估都基于人际沟通科学的五个维度：人类对齐、共情回应、同步、共鸣和任务跟进。HEART揭示了显著的行为模式。一些前沿模型在感知共情和一致性方面接近或超越了平均人类回应。同时，人类在适应性重构、紧张命名和细腻语气转变方面保持优势，特别是在对抗性回合中。人类和LLM作为评判者的偏好在约80%的配对比较中一致，与人类之间的一致性相当，且其书面理由强调相似的HEART维度。这种模式表明，在评估支持质量的标准方面存在日益增长的趋同。通过将人类和模型置于同等 footing，HEART将支持性对话重新界定为一个独立的能力轴，与一般推理或语言流利度分开。它为理解模型生成的支持如何与人类社会判断一致、分歧以及情感对话能力如何随模型大小变化提供了统一的实证基础。
英文摘要：
Supportive conversation depends on skills that go beyond language fluency, including reading emotions, adjusting tone, and navigating moments of resistance, frustration, or distress. Despite rapid progress in language models, we still lack a clear way to understand how their abilities in these interpersonal domains compare to those of humans. We introduce HEART, the first-ever framework that directly compares humans and LLMs on the same multi-turn emotional-support conversations. For each dialogue history, we pair human and model responses and evaluate them through blinded human raters and an ensemble of LLM-as-judge evaluators. All assessments follow a rubric grounded in interpersonal communication science across five dimensions: Human Alignment, Empathic Responsiveness, Attunement, Resonance, and Task-Following. HEART uncovers striking behavioral patterns. Several frontier models approach or surpass the average human responses in perceived empathy and consistency. At the same time, humans maintain advantages in adaptive reframing, tension-naming, and nuanced tone shifts, particularly in adversarial turns. Human and LLM-as-judge preferences align on about 80 percent of pairwise comparisons, matching inter-human agreement, and their written rationales emphasize similar HEART dimensions. This pattern suggests an emerging convergence in the criteria used to assess supportive quality. By placing humans and models on equal footing, HEART reframes supportive dialogue as a distinct capability axis, separable from general reasoning or linguistic fluency. It provides a unified empirical foundation for understanding where model-generated support aligns with human social judgment, where it diverges, and how affective conversational competence scales with model size.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
