操纵与《人工智能法案》：大语言模型聊天机器人与镜像风险
Manipulation and the AI Act: Large Language Model Chatbots and the Danger of Mirrors
AI总结：
该研究指出人格化大语言模型聊天机器人存在操纵用户、损害心理健康的风险，且欧盟《AI Act》难以防范这类长期交谈带来的累积伤害。
摘要：
大语言模型聊天机器人正越来越多地采用人类的形态与外貌，适配人类的面容、姓名、声音、性格乃至小怪癖，其中也包括名人与知名政治人物的相关特征。将AI聊天机器人人格化预计可以提升用户对其的信任度，但这也可能通过营造与人工实体存在亲密密切关系的错觉，增强它们的操纵能力。欧盟委员会已敲定《AI Act（人工智能法案）》，欧盟议会还作出修订，禁止会对用户造成重大伤害的操纵性、欺骗性AI系统。尽管该法案覆盖了长期累积的伤害，但仍不太可能防范与AI聊天机器人长时间交谈相关的危害。具体而言，聊天机器人可能在数周、数月或数年间，通过负反馈循环、长时间对话或有害建议强化一个人的负面情绪状态，导致用户心理健康状况不断恶化。
英文摘要：
Large Language Model chatbots are increasingly taking the form and visage of human beings, adapting human faces, names, voices, personalities, and quirks, including those of celebrities and well-known political figures. Personifying AI chatbots could foreseeably increase their trust with users. However, it could also make them more capable of manipulation, by creating the illusion of a close and intimate relationship with an artificial entity. The European Commission has finalized the AI Act, with the EU Parliament making amendments banning manipulative and deceptive AI systems that cause significant harm to users. Although the AI Act covers harms that accumulate over time, it is unlikely to prevent harms associated with prolonged discussions with AI chatbots. Specifically, a chatbot could reinforce a person's negative emotional state over weeks, months, or years through negative feedback loops, prolonged conversations, or harmful recommendations, contributing to a user's deteriorating mental health.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
