How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Institute of Modern Languages and Linguistics, Fudan University（复旦大学现代语言与语言学研究所）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing, Fudan University（复旦大学上海智能视觉计算协同创新中心）
Xuanting Chen, Junjie Ye, Can Zu, Nuo Xu, Rui Zheng, Minlong Peng, Jie Zhou, Tao Gui, Qi Zhang, Xuanjing Huang
The GPT-3.5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities. However, their robustness and abilities to handle various complexities of the open world have yet to be explored, which is especially crucial in assessing the stability of models and is a key aspect of trustworthy AI. In this study, we perform a comprehensive experimental analysis of GPT-3.5, exploring its robustness using 21 datasets (about 116K test samples) with 66 text transformations from TextFlint that cover 9 popular Natural Language Understanding (NLU) tasks. Our findings indicate that while GPT-3.5 outperforms existing fine-tuned models on some tasks, it still encounters significant robustness degradation, such as its average performance dropping by up to 35.74\% and 43.59\% in natural language inference and sentiment analysis tasks, respectively. We also show that GPT-3.5 faces some specific robustness challenges, including robustness instability, prompt sensitivity, and number sensitivity. These insights are valuable for understanding its limitations and guiding future research in addressing these challenges to enhance GPT-3.5's overall performance and generalization abilities.
Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts
University of Tokyo（东京大学）；Tokyo Institute of Technology（东京工业大学）；MIT（麻省理工学院）；National University of Singapore（新加坡国立大学）；Northwestern University（西北大学）；Bielefeld University（比勒费尔德大学）；University College Dublin（都柏林大学学院）；Moveworks
Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
AI Agent Lab, Vokram Group（AI代理实验室，Vokram集团）；Purdue University（普渡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Minnesota（明尼苏达大学）；Emory University（埃默里大学）；Imperial College London（伦敦帝国学院）
Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency
University of Guelph（圭尔夫大学）；University of North Texas（北德克萨斯大学）；Iowa State University（爱荷华州立大学）；Zayed University（扎耶德大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；National University of Computer & Emerging Sciences（国立计算机与新兴科学大学）
Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西湖大学工程学院）；University College London（伦敦大学学院）；University of Cambridge（剑桥大学）；LanBridge Group（兰桥集团）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖高等研究院先进技术研究院）
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stanford University（斯坦福大学）；University of California, Berkeley（加州大学伯克利分校）；Center for AI Safety（AI安全中心）；Microsoft Corporation（微软公司）；The Chinese University of Hong Kong（香港中文大学）
Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard
Department of Information Science（信息科学系）；Bar Ilan University（巴伊兰大学）；Department of Information Systems（信息系统系）；University of Haifa（海法大学）；Department of Computing（计算机系）；Jonkoping University（约翰·科普林大学）
Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions
Vellore Institute of Technology（韦洛尔理工学院）；Brandon University（布兰登大学）；China Medical University（中国医药大学）；Lebanese American University（黎巴嫩美国大学）；Pandit Deendayal Energy University（潘迪特·迪恩达尔能源大学）；City University of Hong Kong（香港城市大学）；University of Agder（阿格德大学）
A comprehensive study of LLM-based argument classification: from Llama through DeepSeek to GPT-5.2
Faculty of Computer Science, Faculty of Humanities（计算机科学系，人文科学系）；Institute of Mathematics Faculty of Humanities（人文科学学院数学研究所）；University of Silesia（斯蒂利亚大学）；AGH University of Krakow（克拉科夫AGH大学）
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing
An Analysis of Language Frequency and Error Correction for Esperanto
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Alibaba Group（阿里巴巴集团）；Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；Shenzhen Intelligent Strong Technology Co., Ltd（深圳智能强科技有限公司）
A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1
Faculty of Computer Science, Electronics, and Telecommunications AGH University of Krakow（计算机科学、电子与电信学院AGH克拉科夫大学）；Faculty of Humanities AGH University of Krakow（人文学院AGH克拉科夫大学）；Institute of Mathematics University of Silesia（数学学院桑特学院）
Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models
IDEA Research（IDEA研究机构）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanjing University（南京大学）；South China Normal University（华南师范大学）；DataArcTech Ltd.（DataArcTech有限公司）
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks
University of California, Santa Barbara（加州大学圣塔芭芭拉分校）；University of California, San Diego（加州大学圣迭戈分校）；University of Southern California（南加州大学）；Bytedance, US（字节跳动（美国））
TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model
School of Public Health, Zhejiang University（浙江大学公共卫生学院）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Mind meets machine: Unravelling GPT-4's cognitive psychology
Nowrosjee Wadia College（诺罗斯吉·瓦迪亚学院）；Indian Institute of Tropical Meteorology（印度热带气象研究所）；Defence Institute Of Advanced Technology（国防先进技术研究所）；Queen Mary University of London（伦敦玛丽女王大学）
Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today
Tsinghua University（清华大学）；The First Affiliated Hospital of Nanchang University（南昌大学第一附属医院）；University of Chinese Academy of Sciences（中国科学院大学）；Shandong University（山东大学）；East China Normal University（华东师范大学）
该研究在两个真实临床数据集上比较 GPT-4 与传统 AI 工具及医生的痴呆诊断表现，发现 GPT-4 虽具潜力，但准确性、可解释性和忠实性目前仍未超越传统工具。
Mayo Clinic（梅奥诊所）；the First Affiliated Hospital of Nanchang University（南昌大学第一附属医院）；University of Georgia（佐治亚大学）；University of Electronic Science and Technology of China（电子科技大学）；Eye Institute of Xiamen University（厦门大学眼科研究所）
GP-GPT: Large Language Model for Gene-Phenotype Mapping
Department of Computer Science and Engineering, The University of Texas at Arlington（计算机科学与工程系，德克萨斯理工大学 Arlington 分校）；School of Computing, University of Georgia（计算学院，佐治亚大学）；Department of Computer Science, Indiana University Indianapolis（计算机科学系，印第安纳大学 Indianapolis 分校）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（放射科，麻省总医院和哈佛医学院）；Department of Statistics, University of Georgia（统计系，佐治亚大学）；Department of Epidemiology & Biostatistics, University of Georgia（流行病学与生物统计学系，佐治亚大学）；Department of Environmental Health Science, University of Georgia（环境流行病学系，佐治亚大学）
