GPT-5 vs Other LLMs in Long Short-Context Performance
Bernoulli Institute RUG(伯努利研究所);ISRC, Khazar University(ISRC、哈扎尔大学);Department of Computer Engineering University of Guilan(计算机工程系、盖兰大学);QCE Department TU Delft(QCE部门、代尔夫特理工大学)
Nima Esmi, Maryam Nezhad-Moghaddam, Fatemeh Borhani, Asadollah Shahbahrami, Amin Daemdoost, Georgi Gaydadjiev
备注:
10 pages, 7 figures. Accepted for publication in the 3rd International Conference on Foundation and Large Language Models (FLLM2025). IEEE. The final version will be available in IEEE Xplore
With the significant expansion of the context window in Large Language Models (LLMs), these models are theoretically capable of processing millions of tokens in a single pass. However, research indicates a significant gap between this theoretical capacity and the practical ability of models to robustly utilize information within long contexts, especially in tasks that require a comprehensive understanding of numerous details. This paper evaluates the performance of four state-of-the-art models (Grok-4, GPT-4, Gemini 2.5, and GPT-5) on long short-context tasks. For this purpose, three datasets were used: two supplementary datasets for retrieving culinary recipes and math problems, and a primary dataset of 20K social media posts for depression detection. The results show that as the input volume on the social media dataset exceeds 5K posts (70K tokens), the performance of all models degrades significantly, with accuracy dropping to around 50-53% for 20K posts. Notably, in the GPT-5 model, despite the sharp decline in accuracy, its precision remained high at approximately 95%, a feature that could be highly effective for sensitive applications like depression detection. This research also indicates that the "lost in the middle" problem has been largely resolved in newer models. This study emphasizes the gap between the theoretical capacity and the actual performance of models on complex, high-volume data tasks and highlights the importance of metrics beyond simple accuracy for practical applications.
Does GPT-4 surpass human performance in linguistic pragmatics?
Institute for Artificial Intelligence Research and Development of Serbia(塞尔维亚人工智能研究与发展研究所);University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab(贝尔格莱德大学哲学与社会理论研究所数字社会实验室);Complexity Science Hub(复杂性科学中心);University of Novi Sad, Faculty of Philosophy(诺维萨德大学哲学学院);University of Kragujevac, Faculty of Engineering(克拉古耶瓦茨大学工程学院)
Prompt-Efficient Fine-Tuning for GPT-like Deep Models to Reduce Hallucination and to Improve Reproducibility in Scientific Text Generation Using Stochastic Optimisation Techniques
Higher School of Economics (HSE)(高等经济大学(HSE));AI for Computational biology (AIC)(计算生物学人工智能研究所(AIC))
Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today
Tsinghua University(清华大学);The First Affiliated Hospital of Nanchang University(南昌大学第一附属医院);University of Chinese Academy of Sciences(中国科学院大学);Shandong University(山东大学);East China Normal University(华东师范大学)
该研究在两个真实临床数据集上比较 GPT-4 与传统 AI 工具及医生的痴呆诊断表现,发现 GPT-4 虽具潜力,但准确性、可解释性和忠实性目前仍未超越传统工具。
Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
Department of Computer Science, University of British Columbia(不列颠哥伦比亚大学计算机科学系);Vector Institute for AI(人工智能矢量研究所);Department of Linguistics, University of British Columbia(不列颠哥伦比亚大学语言学系);Mila – Quebec AI Institute(魁北克人工智能研究所);School of Computer Science, McGill University(麦吉尔大学计算机科学学院);Department of Sociology, University of British Columbia(不列颠哥伦比亚大学社会学系)
Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard
Department of Information Science(信息科学系);Bar Ilan University(巴伊兰大学);Department of Information Systems(信息系统系);University of Haifa(海法大学);Department of Computing(计算机系);Jonkoping University(约翰·科普林大学)
Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery
Stanford School of Medicine(斯坦福医学院);Clinical Excellence Research Center(临床卓越研究中心);Stanford Health Care(斯坦福医疗保健系统);Warren Alpert Medical School of Brown University(布朗大学沃伦·阿尔珀特医学院);Atropos Health(阿特罗波斯健康公司);University of Ottawa(渥太华大学);University of California San Francisco(加利福尼亚大学旧金山分校);Microsoft Corporation(微软公司);Stanford Human-Centered AI(斯坦福以人为本人工智能中心)