arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.08007cs.HCcs.AI
收藏

人们在图灵测试中无法区分GPT-4和人类

People cannot distinguish GPT-4 from a human in a Turing test

UC San Diego(加州大学圣迭戈分校)

Cameron R. Jones, Benjamin K. Bergen

AI总结:

本研究通过随机对照的预注册图灵测试发现,GPT-4被判定为人类的比例达54%,首次实证证明人工系统可通过双人交互式图灵测试,且风格与社会情感因素对通过测试的影响大于传统智能概念。

摘要:

我们在一项随机、对照且预先注册的图灵测试中评估了3个系统(ELIZA、GPT-3.5和GPT-4)。人类参与者与人类或AI进行5分钟对话,并判断其认为对话对象是否为人类。GPT-4有54%的概率被判定为人类,表现优于ELIZA(22%),但落后于真实人类(67%)。该结果首次提供了有力的实证证明,即人工系统可以通过交互式双人图灵测试。这些结果对围绕机器智能的争论具有启示意义,更紧迫的是,这表明当前AI系统的欺骗行为可能不会被发现。对参与者策略和推理的分析表明,在通过图灵测试方面,风格和社会情感因素比传统的智能概念发挥着更大的作用。

英文摘要:

We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑