信任源于设计：通过非建议式苏格拉底对话实现对话代理中的信任校准

中文摘要 AI 辅助

随着对话式人工智能系统越来越多地应用于敏感领域，核心挑战从可用性转向信任校准，即确保用户对系统的依赖既不过多也不过少。提供建议或解释的系统可能鼓励不当依赖，尤其是当用户将AI输出视为权威时。我们提出了CASELy，一种明确设计为通过非建议式苏格拉底对话限制自身权威的对话代理。该代理提出仅基于用户输入的反思性问题，并拒绝提供建议、推荐或解释。这种设计通过约束代理的自主性而非优化能力来实现信任校准。在一项针对高等教育学生的初步随机对照研究中，与苏格拉底对话互动的参与者报告的用户体验（UEQ-S总体=1.50）显著高于非对话对照组（0）。定性研究结果确定了支持校准信任的三种机制：通过可见的接地实现透明度、保留用户决策权威以及减少对评判的恐惧。我们认为，适当的依赖可以通过交互约束来实现，为敏感情境中值得信赖的对话式AI提供了一种设计模式。

英文摘要

As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining agent agency rather than optimizing capability. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judgment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts.

发表机构

• Braude College of Engineering（Braude工程学院）

机构由 AI 辅助整理，请以论文原文为准。