Uncovering inequalities in new knowledge learning by large language models across different languages
Peking University Shenzhen Graduate School（北京大学深圳研究生院）；University of Science and Technology of China（中国科学技术大学）；Wuhan University（武汉大学）；Hong Kong University of Science and Technology（香港科技大学）；Princeton University（普林斯顿大学）；Microsoft Research（微软研究院）；Microsoft Research Asia（微软亚洲研究院）；Guanghua School of Management, Peking University（北京大学光华管理学院）
As large language models (LLMs) gradually become integral tools for problem solving in daily life worldwide, understanding linguistic inequality is becoming increasingly important. Existing research has primarily focused on static analyses that assess the disparities in the existing knowledge and capabilities of LLMs across languages. However, LLMs are continuously evolving, acquiring new knowledge to generate up-to-date, domain-specific responses. Investigating linguistic inequalities within this dynamic process is, therefore, also essential. In this paper, we explore inequalities in new knowledge learning by LLMs across different languages and four key dimensions: effectiveness, transferability, prioritization, and robustness. Through extensive experiments under two settings (in-context learning and fine-tuning) using both proprietary and open-source models, we demonstrate that low-resource languages consistently face disadvantages across all four dimensions. By shedding light on these disparities, we aim to raise awareness of linguistic inequalities in LLMs' new knowledge learning, fostering the development of more inclusive and equitable future LLMs.
Uncovering Overfitting in Large Language Model Editing
Shandong University（山东大学）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
LLM Factoscope: Uncovering LLMs' Factual Discernment through Inner States Analysis
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Identifying the Achilles' Heel: An Iterative Method for Dynamically Uncovering Factual Errors in Large Language Models
Renmin University of China（中国人民大学）；The Chinese University of Hong Kong（香港中文大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学深圳校区）；Johns Hopkins University（约翰霍普金斯大学）；Nanyang Technological University（南洋理工大学）；Xiaohongshu Inc.（小红书公司）；Tencent Inc.（腾讯公司）
Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；State Key Laboratory of Cyberspace Security Defense（网络空间安全防御国家重点实验室）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
Uncovering Limitations of Large Language Models in Information Seeking from Tables
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS)（中国科学院智能信息处理重点实验室）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Harbin Engineering University（哈尔滨工程大学）；Peng Cheng Laboratory（鹏城实验室）
MSEarth: A Multimodal Scientific Dataset and Benchmark for Phenomena Uncovering in Earth Science
The Hong Kong Polytechnic University（香港理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）
Uncovering Entity Identity Confusion in Multimodal Knowledge Editing
New Laboratory of Pattern Recognition (NLPR)（模式识别新实验室）；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)（多模态人工智能系统国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；Huazhong University of Science and Technology（华中科技大学）；Shandong University（山东大学）
Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models
Tsinghua University（清华大学）；Zhongguancun Laboratory（中关村实验室）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shandong Key Laboratory of Artificial Intelligence Security（山东省人工智能安全重点实验室）
