对话AI中的崩溃：情感与伦理敏感情境中的互动失败
Breakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts
AI总结：
研究探讨对话代理在情感和伦理敏感行为中出现的崩溃现象，分析其对对话质量的影响，并提出维护伦理一致性和情感敏感性的设计改进。
摘要：
对话AI正越来越多地应用于情感强烈和伦理敏感的互动中。先前研究主要集中在情感基准或静态安全检查上，忽略了对话中对齐的动态发展。我们探讨了研究问题：当对话代理面对情感和伦理敏感行为时会出现哪些崩溃，以及这些崩溃如何影响对话质量？为了压力测试聊天机器人性能，我们开发了一种基于角色的用户模拟器，能够与心理角色进行多轮对话并模拟情感节奏。我们的分析显示，主流模型在情感轨迹加剧时表现出反复崩溃。我们识别出几种常见失败模式，包括情感错位、伦理指导失败以及跨维度权衡，其中同理心可能超越或削弱责任。我们将这些模式组织成一个分类法，并讨论设计影响，强调在动态互动中维持伦理一致性和情感敏感性的必要性。本研究为HCI社区提供了新的视角，以诊断和改进价值敏感和情感强烈情境中的对话AI。
英文摘要：
Conversational AI is increasingly deployed in emotionally charged and ethically sensitive interactions. Previous research has primarily concentrated on emotional benchmarks or static safety checks, overlooking how alignment unfolds in evolving conversation. We explore the research question: what breakdowns arise when conversational agents confront emotionally and ethically sensitive behaviors, and how do these affect dialogue quality? To stress-test chatbot performance, we develop a persona-conditioned user simulator capable of engaging in multi-turn dialogue with psychological personas and staged emotional pacing. Our analysis reveals that mainstream models exhibit recurrent breakdowns that intensify as emotional trajectories escalate. We identify several common failure patterns, including affective misalignments, ethical guidance failures, and cross-dimensional trade-offs where empathy supersedes or undermines responsibility. We organize these patterns into a taxonomy and discuss the design implications, highlighting the necessity to maintain ethical coherence and affective sensitivity throughout dynamic interactions. The study offers the HCI community a new perspective on the diagnosis and improvement of conversational AI in value-sensitive and emotionally charged contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
