arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.07698cs.CL
收藏

对情感支持模型的压力测试:从同质到多样求助者的转变

Stress-Testing Emotional Support Models: Moving from Homogeneous to Diverse Help Seekers

Graduate School of Data Science, Seoul National University(首尔国立大学数据科学研究生院)

Chaewon Heo, Cheyon Jin, Yohan Jo

AI总结:

本文提出可控的求助者模拟器,通过九个心理和语言特征训练MoE架构,提升行为多样性与可控性,评估7个支持模型发现性能退化问题,为情感支持聊天机器人提供更真实的压力测试。

摘要:

随着情感支持聊天机器人在研究和行业中的广泛应用,一种常见的评估策略是使用求助者模拟器与支持者聊天机器人交互。然而,当前的模拟器存在两个关键限制:(1)无法捕捉真实世界求助者的多样性,常将其表现为过度合作;(2)缺乏模拟特定求助者档案所需的可控性。为解决这些问题,我们提出了一种由九个心理和语言特征驱动的可控求助者模拟器。使用真实的Reddit对话,我们通过混合专家(MoE)架构训练模型,有效将多样化求助者行为区分到专用参数子空间,从而提升细粒度可控性。我们的模拟器在档案一致性和行为多样性方面优于现有方法。此外,使用我们的系统评估7个 prominent 支持者模型,揭示了之前被掩盖的性能退化问题。这些发现强调了我们的框架在为情感支持聊天机器人提供更真实和压力测试的评估方面的效用。

英文摘要:

As emotional support chatbots have recently gained significant traction across both research and industry, a common evaluation strategy has emerged: use help-seeker simulators to interact with supporter chatbots. However, current simulators suffer from two critical limitations: (1) they fail to capture the behavioral diversity of real-world seekers, often portraying them as overly cooperative, and (2) they lack the controllability required to simulate specific seeker profiles. To address these challenges, we present a controllable seeker simulator driven by nine psychological and linguistic features that underpin seeker behavior. Using authentic Reddit conversations, we train our model via a Mixture-of-Experts (MoE) architecture, which effectively differentiates diverse seeker behaviors into specialized parameter subspaces, thereby enhancing fine-grained controllability. Our simulator achieves superior profile adherence and behavioral diversity compared to existing approaches. Furthermore, evaluating 7 prominent supporter models with our system uncovers previously obscured performance degradations. These findings underscore the utility of our framework in providing a more faithful and stress-tested evaluation for emotional support chatbots.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑