Does GPT-4 surpass human performance in linguistic pragmatics?
Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab（贝尔格莱德大学哲学与社会理论研究所数字社会实验室）；Complexity Science Hub（复杂性科学中心）；University of Novi Sad, Faculty of Philosophy（诺维萨德大学哲学学院）；University of Kragujevac, Faculty of Engineering（克拉古耶瓦茨大学工程学院）
As Large Language Models (LLMs) become increasingly integrated into everyday life as general purpose multimodal AI systems, their capabilities to simulate human understanding are under examination. This study investigates LLMs ability to interpret linguistic pragmatics, which involves context and implied meanings. Using Grice communication principles, we evaluated both LLMs (GPT-2, GPT-3, GPT-3.5, GPT-4, and Bard) and human subjects (N = 147) on dialogue-based tasks. Human participants included 71 primarily Serbian students and 76 native English speakers from the United States. Findings revealed that LLMs, particularly GPT-4, outperformed humans. GPT4 achieved the highest score of 4.80, surpassing the best human score of 4.55. Other LLMs performed well: GPT 3.5 scored 4.10, Bard 3.75, and GPT-3 3.25. GPT-2 had the lowest score of 1.05. The average LLM score was 3.39, exceeding the human cohorts averages of 2.80 (Serbian students) and 2.34 (U.S. participants). In the ranking of all 155 subjects (including LLMs and humans), GPT-4 secured the top position, while the best human ranked second. These results highlight significant progress in LLMs ability to simulate understanding of linguistic pragmatics. Future studies should confirm these findings with more dialogue-based tasks and diverse participants. This research has important implications for advancing general-purpose AI models in various communication-centered tasks, including potential application in humanoid robots in the future.
Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs
Department of Cognitive Science, Carleton University（卡尔顿大学认知科学系）；School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；Mila – Quebec AI Institute（魁北克人工智能研究所）
When Humans Leave Context Unsaid: Implicit Context as a Risk for Large Language Models
University of Mississippi（密西西比大学）；Michigan State University（密歇根州立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）
LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encoder
DCST, BNRist（DCST、BNRist；KIRC、人工智能研究院、清华大学，中国）；KIRC, Institute for Artificial Intelligence, Tsinghua University, China（中文语言文学系、北京大学，中国）；Department of Chinese Language and Literature, Peking University, China
Do language models accommodate their users? A study of linguistic convergence
Khoury College of Computer Sciences, Northeastern University（东北大学克劳尔计算机科学学院）；Faculty of Computer Science, University of Vienna（维也纳大学计算机科学系）；Faculty of Philological and Cultural Studies, University of Vienna（维也纳大学语言与文化研究系）
The Pragmatic Persona: Discovering LLM Persona through Bridging Inference
Department of Artificial Intelligence, Chung-Ang University, Seoul, 06974, Republic of Korea（Chung-Ang大学人工智能系）；Department of Computer Science, University of British Columbia, Vancouver, BC V6T 1Z4, Canada（不列颠哥伦比亚大学计算机科学系）；Van Lang University, Ho Chi Minh City, Vietnam（文-lang大学）
Do LLMs Use Cultural Knowledge Without Being Told? A Multilingual Evaluation of Implicit Pragmatic Adaptation
Network Analysis and Social Influence Modelling (NASIM) Lab（网络分析与社会影响力建模实验室）；School of Physics Maths and Computing（物理数学与计算学院）；The University of Western Australia（西澳大学）；School of Computing（计算学院）；Macquarie University（麦考瑞大学）；Computational Social Science & Systems Analysis（计算社会科学与系统分析）；University of Münster（明斯特大学）
A fine-grained comparison of pragmatic language understanding in humans and language models
Massachusetts Institute of Technology（麻省理工学院）；Sarah Lawrence College（莎拉·劳伦斯学院）；Carleton University（卡尔顿大学）；McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑科学研究所）
Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard
Department of Information Science（信息科学系）；Bar Ilan University（巴伊兰大学）；Department of Information Systems（信息系统系）；University of Haifa（海法大学）；Department of Computing（计算机系）；Jonkoping University（约翰·科普林大学）
Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
Center for Information and Language Processing (CIS)（信息与语言处理中心）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；ScaDS.AI and TU Dresden（ScaDS.AI 和 梅克伦堡-前波美拉尼亚技术大学）；Department of Statistics, LMU Munich（统计学系，慕尼黑大学）；University of Maryland, College Park（马里兰大学 College Park 分校）
FPT: Feature Prompt Tuning for Few-shot Readability Assessment
Peking University（北京大学）；National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；School of Computer Science, Peking University（北京大学计算机学院）
