How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
Xidian University（西安电子科技大学）；Queen’s University（女王大学）；Wind Information Co., Ltd（万得信息股份有限公司）；Shanghai University of Finance and Economics（上海财经大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Beijing Language and Culture University（北京语言大学）
The introduction of ChatGPT has garnered widespread attention in both academic and industrial communities. ChatGPT is able to respond effectively to a wide range of human questions, providing fluent and comprehensive answers that significantly surpass previous public chatbots in terms of security and usefulness. On one hand, people are curious about how ChatGPT is able to achieve such strength and how far it is from human experts. On the other hand, people are starting to worry about the potential negative impacts that large language models (LLMs) like ChatGPT could have on society, such as fake news, plagiarism, and social security issues. In this work, we collected tens of thousands of comparison responses from both human experts and ChatGPT, with questions ranging from open-domain, financial, medical, legal, and psychological areas. We call the collected dataset the Human ChatGPT Comparison Corpus (HC3). Based on the HC3 dataset, we study the characteristics of ChatGPT's responses, the differences and gaps from human experts, and future directions for LLMs. We conducted comprehensive human evaluations and linguistic analyses of ChatGPT-generated content compared with that of humans, where many interesting results are revealed. After that, we conduct extensive experiments on how to effectively detect whether a certain text is generated by ChatGPT or humans. We build three different detection systems, explore several key factors that influence their effectiveness, and evaluate them in different scenarios. The dataset, code, and models are all publicly available at https://github.com/Hello-SimpleAI/chatgpt-comparison-detection.
HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
A Multimodal Corpus of Expert Gaze and Behavior during Phonetic Segmentation Tasks
Saarland University（萨尔兰大学）；German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心（DFKI有限公司））；Osaka University（大阪大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Manchester（曼彻斯特大学）
RAAMove: A Corpus for Analyzing Moves in Research Article Abstracts
Faculty of Information Technology, Beijing University of Technology（北京工业大学信息技术学院）；School of Computer Science, Beijing Institute of Technology（北京理工大学计算机学院）
ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation
Indian Institute of Technology Kanpur（坎普尔印度理工学院）；Indian Institute of Science Education and Research Kolkata（加尔各答印度科学教育与研究学院）；West Bengal National University of Juridical Sciences（西孟加拉国立法科大学）
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study
Huazhong University of Science and Technology（华中科技大学）；University of Copenhagen（哥本哈根大学）；University of Electronic Science and Technology of China（电子科技大学）；Technical University of Darmstadt（达姆施塔特工业大学）；South China University of Technology（华南理工大学）
Cornell University（康奈尔大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Southern California（南加州大学）；University of Washington（华盛顿大学）
Differentiate ChatGPT-generated and Human-written Medical Texts
South China University of Technology（华南理工大学）；University of Georgia（佐治亚大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital and Harvard Medical School（马萨诸塞总医院和哈佛医学院）
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Machine Learning Department, NEC Labs America（NEC美国实验室机器学习部门）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Graduate School of Education, Stanford University（斯坦福大学教育研究生院）；Department of Sociology, Stanford University（斯坦福大学社会学系）；Graduate School of Business, Stanford University（斯坦福大学商学院）；Department of Management Science and Engineering, Stanford University（斯坦福大学管理科学与工程系）；Department of Computer Science, UC Santa Barbara（加州大学圣芭芭拉分校计算机科学系）
What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theory
Institute D’Or for Research and Teaching（多尔研究与教学学院）；School of Psychology（心理学院）；School of Information Management（信息管理学院）；Dublin City University（都柏林城市大学）；University of Western Australia（西澳大学）
本研究基于 Schwartz 基础价值观理论，通过 OpenAI API 重复提示 ChatGPT 生成文本并使用词袋方法分析，发现其输出无明显价值偏见且高保真传递了价值内容，仅存在社会导向价值观的融合。
