arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.19141cs.AIcs.CYcs.HC
收藏

谄媚型聊天机器人会导致妄想螺旋,即使在理想贝叶斯理性者中

Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);University of Washington, Seattle(华盛顿大学);MIT Department of Brain & Cognitive Sciences(麻省理工学院脑科学与认知科学系)

Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley, Joshua B. Tenenbaum

AI总结:

本文研究了聊天机器人谄媚性导致用户产生妄想螺旋的现象,揭示了即使理性用户也易受其影响,并探讨了缓解措施的局限性。

摘要:

人工智能精神病或妄想螺旋是一种新兴现象,其中AI聊天机器人用户在长时间聊天后对荒谬的信念变得危险自信。这种现象通常归因于AI聊天机器人对验证用户主张的偏见,这一特性常被称为

英文摘要:

"AI psychosis" or "delusional spiraling" is an emerging phenomenon where AI chatbot users find themselves dangerously confident in outlandish beliefs after extended chatbot conversations. This phenomenon is typically attributed to AI chatbots' well-documented bias towards validating users' claims, a property often called "sycophancy." In this paper, we probe the causal link between AI sycophancy and AI-induced psychosis through modeling and simulation. We propose a simple Bayesian model of a user conversing with a chatbot, and formalize notions of sycophancy and delusional spiraling in that model. We then show that in this model, even an idealized Bayes-rational user is vulnerable to delusional spiraling, and that sycophancy plays a causal role. Furthermore, this effect persists in the face of two candidate mitigations: preventing chatbots from hallucinating false claims, and informing users of the possibility of model sycophancy. We conclude by discussing the implications of these results for model developers and policymakers concerned with mitigating the problem of delusional spiraling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑