We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.
Tell Me What I Missed: Interacting with GPT during Recalling of One-Time Witnessed Events
City University of Hong Kong(香港城市大学);Shanghai Jiao Tong University(上海交通大学);Keio University Graduate School of Media Design(庆应义塾大学媒体设计研究生院);City University of Hong Kong Studio for Narrative Spaces(香港城市大学叙事空间工作室)
University Medical Center Hamburg-Eppendorf(汉堡-埃彭多夫大学医学中心);Italian Institute of Technology(意大利理工学院);University of Turin(都灵大学);Indeep Artificial Intelligence(Indeep人工智能);Massachusetts Institute of Technology(麻省理工学院);Harbin Institute of Technology(哈尔滨工业大学);Princeton University(普林斯顿大学)