arXiv 2605.282282026-05-27cs.CL

当求助者难以帮助：评估情感支持对话系统在最坏情况交互中的表现

When Seekers Are Hard to Help: Evaluating Emotional Support Dialogue Systems in Worst-Case Interactions

Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning（湖北人工智能与智能学习省级重点实验室）；National Language Resources Monitoring and Research Center for Network Media（网络媒体语言资源监测与研究中心）；School of Computer Science, Central China Normal University（华中师范大学计算机学院）；Faculty of Artificial Intelligence in Education, Central China Normal University（华中师范大学教育人工智能学院）；School of Chinese Language and Literature, Central China Normal University（华中师范大学中文语言文学学院）

Jiajie Yang, Yangchun Li, Guanyi Chen, Rui Fan, Xin Bai, Tingting He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究通过专家模拟和提出最坏情况评估框架，发现现有情感支持对话系统在面对低参与度、抗拒等困难求助者时性能显著下降，并验证了最坏情况模拟数据可提升模型鲁棒性。

摘要：

情感支持对话系统（ESDS）越来越多地使用大语言模型模拟的求助者进行评估和训练。然而，这类模拟求助者通常表现为合作、平均水平的用户，他们清晰披露、建设性回应并在几轮内接受支持。这可能导致过于乐观的评估，并掩盖ESDS是否能够处理困难的求助互动。在这项工作中，我们研究了在最坏情况交互下的ESDS评估，其中求助者由于低参与度、抗拒、有限的自我披露、情绪波动或僵化的负面解释而难以帮助。我们首先进行了一项专家模拟研究，邀请八位经验丰富的咨询专业人员模拟困难求助者，与现有的中文ESDS互动，提供量表评分，并参与半结构化访谈。基于这项研究，我们推导出最坏情况下的求助者行为，并识别出当前系统的关键局限性。然后，我们提出了一个最坏情况评估框架，包括一个基于LLM的最坏情况求助者模拟器和四个面向最坏情况的指标：深度情感理解、引导性探索、平衡的情感支持以及真实和接地气的支持。评估17个系统后，我们发现几乎所有模型在最坏情况交互下性能都大幅下降。大型通用LLM通常比专门的ESDS更稳健，但即使是最强的模型也难以维持参与度并改善求助者的情绪状态。最后，我们表明最坏情况模拟也可以生成有用的训练数据，提高较小模型的鲁棒性。

英文摘要：

Emotional Support Dialogue Systems (ESDSes) are increasingly evaluated and trained with LLM-simulated seekers. However, such simulated seekers often behave as cooperative, average-case users who disclose clearly, respond constructively, and accept support within a few turns. This can lead to overly optimistic evaluation and obscure whether ESDSes can handle difficult help-seeking interactions. In this work, we study ESDS evaluation under worst-case interactions, where seekers are hard to help due to low engagement, resistance, limited self-disclosure, emotional volatility, or rigid negative interpretations. We first conduct an expert simulation study with eight experienced counselling professionals, who simulate difficult seekers, interact with existing Chinese ESDSes, provide scale ratings, and participate in semi-structured interviews. Based on this study, we derive worst-case seeker behaviours and identify key limitations of current systems. We then propose a worst-case evaluation framework consisting of an LLM-based worst-case seeker simulator and four worst-case-oriented metrics: Deep Emotional Understanding, Guided Exploration, Balanced Emotional Support, and Authentic and Grounded Support. Evaluating 17 systems, we find that nearly all models suffer substantial performance drops under worst-case interactions. Large general-purpose LLMs are generally more robust than specialised ESDSes, but even the strongest models struggle to sustain engagement and improve seekers' emotional states. Finally, we show that worst-case simulation can also generate useful training data, improving the robustness of smaller models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。