We asked ChatGPT to participate in an undergraduate computer science exam on ''Algorithms and Data Structures''. The program was evaluated on the entire exam as posed to the students. We hand-copied its answers onto an exam sheet, which was subsequently graded in a blind setup alongside those of 200 participating students. We find that ChatGPT narrowly passed the exam, obtaining 20.5 out of 40 points. This impressive performance indicates that ChatGPT can indeed succeed in challenging tasks like university exams. At the same time, the questions in our exam are structurally similar to those of other exams, solved homework problems, and teaching materials that can be found online and might have been part of ChatGPT's training data. Therefore, it would be inadequate to conclude from this experiment that ChatGPT has any understanding of computer science. We also assess the improvements brought by GPT-4. We find that GPT-4 would have obtained about 17\% more exam points than GPT-3.5, reaching the performance of the average student. The transcripts of our conversations with ChatGPT are available at \url{https://github.com/tml-tuebingen/chatgpt-algorithm-exam}, and the entire graded exam is in the appendix of this paper.
ChatGPT and Gemini participated in the Korean College Scholastic Ability Test -- Earth Science I
Institute for Research Excellence in Learning Sciences（学习科学卓越研究所）；Seoul National University（首尔大学）；National Taiwan Normal University（台湾师范大学）；Universitas Negeri Malang（玛琅国立大学）
This study evaluated the ability of ChatGPT, a recently developed artificial intelligence (AI) agent, to perform high-level cognitive tasks and produce text that is indistinguishable from human-generated text. This capacity raises concerns about the potential use of ChatGPT as a tool for academic misconduct in online exams. The study found that ChatGPT is capable of exhibiting critical thinking skills and generating highly realistic text with minimal input, making it a potential threat to the integrity of online exams, particularly in tertiary education settings where such exams are becoming m…
Benchmarking ChatGPT-4 on ACR Radiation Oncology In-Training (TXIT) Exam and Red Journal Gray Zone Cases: Potentials and Challenges for AI-Assisted Medical Education and Decision Making in Radiation Oncology
University Hospital Erlangen（埃尔兰根大学医院）；Comprehensive Cancer Center Erlangen-EMN（埃尔兰根-EMN综合癌症中心）；Pattern Recognition Lab（模式识别实验室）
Performance of ChatGPT-3.5 and GPT-4 on the United States Medical Licensing Examination With and Without Distractions
Magee-Womens Research Institute（马吉妇女研究所）；University of Pittsburgh School of Medicine（匹兹堡大学医学院）；Chaim Sheba Medical Center（哈伊姆·谢巴医疗中心）；Sackler Faculty of Medicine, Tel Aviv University（特拉维夫大学萨克勒医学院）；Ariel University（阿里尔大学）
Jožef Stefan Institute（约泽夫·斯蒂芬研究所）；Center za jezikovne vire in tehnologije（语言资源与技术中心）；Univerze v Ljubljani（卢布尔雅那大学）
ChatGPT has shown strong capabilities in natural language generation tasks, which naturally leads researchers to explore where its abilities end. In this paper, we examine whether ChatGPT can be used for zero-shot text classification, more specifically, automatic genre identification. We compare ChatGPT with a multilingual XLM-RoBERTa language model that was fine-tuned on datasets, manually annotated with genres. The models are compared on test sets in two languages: English and Slovenian. Results show that ChatGPT outperforms the fine-tuned model when applied to the dataset which was not seen…
The potential of large language models for improving probability learning: A study on ChatGPT3.5 and first-year computer engineering students
Rey Juan Carlos University（胡安·卡洛斯国王大学）；European Commission, Joint Research Centre（欧盟委员会联合研究中心）；CETINIA: Research Centre for Intelligent Information Technologies（CETINIA智能信息技术研究中心）
Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish
Centre for Language and Cognition, University of Groningen（语言与认知中心，格罗宁根大学）；Departament de Llenguatges i Sistemes Informàtics, Universitat d’Alacant（语言与信息系统部门，阿尔阿罕特大学）；Departament de Traducció i Comunicació, Universitat Jaume I（翻译与传播部门， Jaime I 大学）
