Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, this critical property remains largely under-explored in multilingual contexts. In this work, we conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages, revealing that non-English languages suffer from systematically worse calibration. To diagnose this, we investigate the model's internal representations and find that the final layer, biased by English-centric training, provides a poor signal for multilingual confidence. In contrast, our layer-wise analysis uncovers a key insight that late-intermediate layers consistently offer a more reliable and better-calibrated signal. Building on this, we introduce a suite of training-free methods, including Language-Aware Confidence Ensemble (LACE), which adaptively selects an optimal ensemble of layers for each specific language. Our study highlights the hidden costs of English-centric alignment and offer a new path toward building more globally equitable and trustworthy LLMs by looking beyond the final layer.
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind
X-LANCE Lab, Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence, SJTU AI Institute Shanghai Jiao Tong University, Shanghai, China(X-LANCE实验室,计算机科学与工程系,MoE人工智能关键实验室,上海交通大学人工智能研究院,上海,中国);Suzhou Laboratory, Suzhou, China(苏州实验室,苏州,中国)
Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning
Mila - Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);The University of Tokyo(东京大学);Western University(西方大学);Vector Institute(向量研究所);Polytechnique Montréal(蒙特利尔理工学院);CIFAR AI Chair(CIFAR人工智能主席);AIST(日本产业技术综合研究所)
On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning
Department of Computer Science, TU Dortmund University(图恩-多特蒙德大学计算机科学系);Bonn-Aachen International Center for Information Technology, University of Bonn(波恩-亚琛国际信息科技中心,波恩大学);Computer Science School, University of Sheffield(谢菲尔德大学计算机科学学院);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所)
Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations
Indian Institute of Technology Bombay(印度理工学院孟买分校);Indian Institute of Technology Mandi(印度理工学院曼迪分校);Indian Institute of Technology Hyderabad(印度理工学院海得拉巴分校)
Reliable Multilingual Orthopedic Decision Support from Clinical Narratives: Language-Aware Adaptation and Verification-Guided Deferral
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Software, Nanjing University of Information Science and Technology (NUIST)(南京信息工程大学软件学院);Department of Orthopedic, Bahawal Victoria Hospital(巴哈瓦尔维多利亚医院骨科部门)
N2C2: Nearest Neighbor Enhanced Confidence Calibration for Cross-Lingual In-Context Learning
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);School of Computer Science and Informatics, Cardiff University(卡迪夫大学计算机科学与信息学院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)
MultiLingPoT: Enhancing Mathematical Reasoning with Multilingual Program Fine-tuning
Fudan University(复旦大学);MYbank, Ant Group(蚂蚁集团MYbank);School of Computer Science, Fudan University(复旦大学计算机学院);School of Data Science, Fudan University(复旦大学数据科学学院)
Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource Languages
School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand(数学与计算科学学院,梅西大学,奥克兰,新西兰);Department of Computer Science, The Open University, Milton Keynes, United Kingdom(计算机科学系,开放大学,米利顿凯斯,英国)
Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学);Shanghai University(上海大学)
RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
Nilekani Centre at AI4Bharat(AI4Bharat 奈尔肯中心);Singapore University of Technology and Design(新加坡科技设计大学);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);National Institute of Information and Communications Technology(信息与通信技术国家研究所);Indian Institute of Technology Bombay(印度理工学院孟买分校);Microsoft(微软);IT University of Copenhagen(哥本哈根IT大学)
Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
Dwarkadas Jivanlal Sanghvi College of Engineering(达沃拉斯·吉文拉尔工程学院);Indian Institute of Technology Jodhpur(印度理工学院乔浦尔分校);King’s College London(伦敦国王学院);Indian Institute of Technology Patna(印度理工学院帕特纳分校)
CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
The Hong Kong University of Science and Technology(香港科学与技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Monash University(莫纳什大学);The Chinese University of Hong Kong(香港中文大学)
MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models
University of Bristol(布里斯托大学);King Abdullah University of Science and Technology(卡利斯勒国王大学科学与技术学院);University of Georgia(佐治亚大学);Southern University of Science and Technology(南方科技大学)
Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages
Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning(慕尼黑机器学习中心);Technical University of Munich(慕尼黑工业大学)
Training Multilingual Machine Translation by Alternately Freezing Language-Specific Encoders-Decoders
TALP Research Center(TALP研究中心);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学);IXA NLP Group(IXA自然语言处理组);University of the Basque Country (UPV/EHU)(巴斯克大学(UPV/EHU))