In the post-Turing era, evaluating large language models (LLMs) involves assessing generated text based on readers' reactions rather than merely its indistinguishability from human-produced content. This paper explores how LLM-generated text impacts readers' decisions, focusing on both amateur and expert audiences. Our findings indicate that GPT-4 can generate persuasive analyses affecting the decisions of both amateurs and professionals. Furthermore, we evaluate the generated text from the aspects of grammar, convincingness, logical coherence, and usefulness. The results highlight a high correlation between real-world evaluation through audience reactions and the current multi-dimensional evaluators commonly used for generative models. Overall, this paper shows the potential and risk of using generated text to sway human decisions and also points out a new direction for evaluating generated text, i.e., leveraging the reactions and decisions of readers. We release our dataset to assist future research.
GPT is becoming a Turing machine: Here are some ways to program it
Fred Hutchinson Cancer Research Center(弗雷德·哈钦森癌症研究中心);University of California San Diego(加利福尼亚大学圣迭戈分校);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Microsoft Research(微软研究院)
The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussions
University of Zurich(苏黎世大学);University of Konstanz(康斯坦茨大学);Complexity Science Hub(复杂性科学中心);University of Applied Sciences and Arts of Southern Switzerland(瑞士南部应用科学与艺术大学)
Findings of the Counter Turing Test: AI-Generated Text Detection
Kalyani Government Engineering College(卡利尼政府工程学院);IIIT Delhi(德里IIIT);BITS Pilani Hyderabad Campus(比斯汉学院海得拉巴校区);AI Institute, University of South Carolina(南卡罗来纳大学人工智能研究所);IIIT Guwahati(古瓦哈提IIIT);NIT Silchar(西里char理工学院);San José State University(圣何塞州立大学);UCLA(加州大学洛杉矶分校);Washington State University(华盛顿州立大学);Vishwakarma Institute of Information Technology(维斯瓦卡马信息科技学院);Meta AI;Amazon AI(亚马逊人工智能);BITS Pilani Goa(比斯汉学院果阿)
A Study of Generative Large Language Model for Medical Research and Healthcare
University of Florida(佛罗里达大学);University of Florida Health Cancer Center(佛罗里达大学健康癌症中心);NVIDIA(英伟达);UF Clinical and Translational Science Institute(佛罗里达大学临床与转化科学研究所)