在移位与反向图灵测试中,GPT-4被判定比人类更具人类特征
GPT-4 is judged more human than humans in displaced and inverted Turing tests
AI总结:
本研究通过反向和移位图灵测试发现,人类与大语言模型在非主动询问场景下难以区分人类与AI,且更易将GPT-4判定为人类,凸显了对话AI检测工具的研发紧迫性。
摘要:
日常AI检测需要在非正式的在线对话中区分人类与AI。在很多场景下,人们并不会直接与AI系统交互,而是阅读AI系统与其他人之间的对话记录。我们采用两种改进版图灵测试——反向图灵测试和移位图灵测试,衡量了人类与大语言模型的辨别能力。GPT-3.5、GPT-4以及移位人类裁判基于图灵测试的对话记录,判断智能体是人类还是AI。我们发现,AI裁判和移位人类裁判的准确率都低于交互式询问者,整体准确率低于随机水平。此外,三者都将表现最佳的GPT-4见证者判定为人类的频率,高于对人类见证者的判定频率。这表明,当人类和当前的大语言模型(LLM)没有主动询问对方时,都难以区分二者,凸显了开发更准确的对话场景AI检测工具的迫切需求。
英文摘要:
Everyday AI detection requires differentiating between people and AI in informal, online conversations. In many cases, people will not interact directly with AI systems but instead read conversations between AI systems and other people. We measured how well people and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. We found that both AI and displaced human judges were less accurate than interactive interrogators, with below chance accuracy overall. Moreover, all three judged the best-performing GPT-4 witness to be human more often than human witnesses. This suggests that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。