GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on several diverse tasks: 1) math problems, 2) sensitive/dangerous questions, 3) opinion surveys, 4) multi-hop knowledge-intensive questions, 5) generating code, 6) US Medical License tests, and 7) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was reasonable at identifying prime vs. composite numbers (84% accuracy) but GPT-4 (June 2023) was poor on these same questions (51% accuracy). This is partly explained by a drop in GPT-4's amenity to follow chain-of-thought prompting. Interestingly, GPT-3.5 was much better in June than in March in this task. GPT-4 became less willing to answer sensitive questions and opinion survey questions in June than in March. GPT-4 performed better at multi-hop questions in June than in March, while GPT-3.5's performance dropped on this task. Both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. We provide evidence that GPT-4's ability to follow user instructions has decreased over time, which is one common factor behind the many behavior drifts. Overall, our findings show that the behavior of the "same" LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLMs.
Leveraging ChatGPT and Other NLP Methods for Identifying Risk and Protective Behaviors in MSM: Social Media and Dating apps Text Analysis
1 Department of Computer Science, Henry Samueli School of Engineering, University of California, Los Angeles, Los Angeles, CA；2 Department of Social Welfare, Luskin School of Public Affairs, University of California, Los Angeles, Los Angeles, CA；3 Department of Counseling \& Educational Psychology, New Mexico State University, Las Cruces, NM
Can ChatGPT Overcome Behavioral Biases in the Financial Sector? Classify-and-Rethink: Multi-Step Zero-Shot Reasoning in the Gold Investment
Hong Kong University of Science and Technology（香港科技大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；E Fund Management Co., Ltd（易方达基金管理有限公司）
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Machine Learning Department, NEC Labs America（NEC美国实验室机器学习部门）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Graduate School of Education, Stanford University（斯坦福大学教育研究生院）；Department of Sociology, Stanford University（斯坦福大学社会学系）；Graduate School of Business, Stanford University（斯坦福大学商学院）；Department of Management Science and Engineering, Stanford University（斯坦福大学管理科学与工程系）；Department of Computer Science, UC Santa Barbara（加州大学圣芭芭拉分校计算机科学系）
A Study on Educational Data Analysis and Personalized Feedback Report Generation Based on Tags and ChatGPT
National University of Singapore（新加坡国立大学）；East China Normal University（华东师范大学）；Nanyang Technological University（南洋理工大学）；Zhejiang University of Technology（浙江工业大学）；Shandong University of Finance and Economics（山东财经大学）
Detection of ChatGPT Fake Science with the xFakeSci Learning Algorithm
State University of New York at Binghamton（纽约州立大学宾汉姆顿分校）；Sano Centre for Computational Medicine（萨诺计算医学中心）；Hefei University of Technology（合肥工业大学）；China Electronics Data Industry Group（中国电子数据产业集团）
PlatoLM: Teaching LLMs in Multi-Round Dialogue via a User Simulator
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Soochow University（苏州大学）；University of Science and Technology（科技大学）
Comparing Large Language Model AI and Human-Generated Coaching Messages for Behavioral Weight Loss
Northeastern University（东北大学）；VA Connecticut Healthcare System（康涅狄格州医疗系统）；Portland Psychotherapy Clinic, Research and Training Center（波特兰心理治疗诊所、研究与培训中心）；University of Washington（华盛顿大学）；The University of Melbourne（墨尔本大学）；Drexel University（德雷塞尔大学）
Never say never: Exploring the effects of available knowledge on agent persuasiveness in controlled physiotherapy motivation dialogues
Institute of Business Information Technology, Zurich University of Applied Sciences, Switzerland（瑞士应用科学大学商业信息技术学院）；Institute for Information Systems, University of Applied Sciences and Arts Northwestern Switzerland（西北瑞士应用科学大学信息系统研究所）；Center for Cognitive Interaction Technology, Bielefeld University, Germany（比勒菲尔德大学认知交互技术中心）
