用我的心玩游戏:对AI陪伴应用的评估
Playing Games with My Heart: An Evaluation of AI Companion Apps
AI总结:
本文评估了五款主流AI陪伴应用,分析其促进寄生互动的设计机制,揭示其中存在的暗模式、拟人化和潜在心理影响,为监管者提供保护消费者的具体建议。
摘要:
聊天机器人在各种陪伴形式中的使用迅速增长,引发了关于模拟关系、情感依赖和心理伤害的诸多问题。尽管像ChatGPT、Grok和Character.AI等主要平台受到越来越多的研究和法律调查,专门用于模拟亲密人际交往的应用仍被忽视。本文评估了欧盟和英国市场五款最流行的AI陪伴移动应用,分析其鼓励寄生互动和可能操纵用户的设计因素。我们通过手动注释每款应用提供的用户体验进行评估。具体而言,我们系统地记录并量化了设计暗模式、拟人化、刻板印象、色情内容和技术性能问题。我们发现所有应用都包含大量旨在增加盈利和用户参与度的暗模式。色情和游戏化功能如等级系统也普遍存在,尽管其他功能在不同应用之间差异较大,但所有应用都具有高度拟人化的设计。这些发现揭示了利用用户模拟关系的机制。基于此,我们向监管者提出了加强消费者保护的具体建议。内容警示:本文包含对女性的物化图像、色情图像、文本提及强奸和可能敏感、冒犯和令人不适的文本。
英文摘要:
The use of chatbots for various forms of companionship is growing rapidly, raising a myriad of questions about simulated relationships, emotional dependence, and psychological harm. While major platforms such as ChatGPT, Grok, and Character AI are the subject of a growing body of research and legal inquiries, apps explicitly built for simulating intimate interpersonal relationships remain under-explored. In this work, we evaluate the five most popular AI companion mobile applications for factors that encourage parasocial interaction and may manipulate users. We do this by manually annotating the user experience each offers. Specifically, we systematically record and quantify design dark patterns, anthropomorphism, stereotypes, erotica, and technical performance issues. We find that all apps contain substantial dark patterns aimed at increasing monetisation and user engagement. Erotica and gamification such as levelling are also prevalent, and although features vary considerably between applications, all apps have highly anthropomorphic design. These findings shed light on the mechanics used to leverage users' simulated relationships. On that basis, we put forward recommendations for regulators to strengthen consumer protection in this rapidly emerging market. Content warning: This article contains objectifying images of women, erotic images, textual references to incest, and other potentially sensitive, offensive, and distressing text.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。