arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2510.03136cs.CL
收藏

超越最终层:大型语言模型中用于更好多语言校准的中间表征

Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models

Language Technology Lab, University of Cambridge(剑桥大学语言技术实验室)

Ej Zhou, Caiqi Zhang, Tiancheng Hu, Chengzu Li, Nigel Collier, Ivan Vulić, Anna Korhonen

AI总结:

本研究针对多语言校准问题,发现LLMs的后期中间层比最终层更适合提供可靠的多语言置信信号,提出无训练方法LACE以提升多语言校准,助力构建更公平可信的LLMs。

摘要:

置信校准是指模型预测的置信度与实际准确率的对齐,对于大型语言模型(LLMs)的可靠部署至关重要。然而,这一关键特性在多语言语境中仍未得到充分探索。本研究首次针对六个模型家族、100多种语言开展大规模系统的多语言校准研究,发现非英语语言的校准系统性更差。为诊断该问题,我们研究了模型的内部表征,发现受以英语为中心的训练影响的最终层,无法为多语言置信度提供良好信号。相比之下,我们的分层分析揭示了一个关键洞见:后期中间层始终能提供更可靠、校准更好的信号。基于此,我们提出了一套无训练方法,包括语言感知置信度集成(LACE),它会为每种特定语言自适应选择最优的层集成方案。本研究凸显了以英语为中心的对齐的隐性代价,为构建更具全球公平性和可信度的LLMs提供了超越最终层的新路径。

英文摘要:

Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, this critical property remains largely under-explored in multilingual contexts. In this work, we conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages, revealing that non-English languages suffer from systematically worse calibration. To diagnose this, we investigate the model's internal representations and find that the final layer, biased by English-centric training, provides a poor signal for multilingual confidence. In contrast, our layer-wise analysis uncovers a key insight that late-intermediate layers consistently offer a more reliable and better-calibrated signal. Building on this, we introduce a suite of training-free methods, including Language-Aware Confidence Ensemble (LACE), which adaptively selects an optimal ensemble of layers for each specific language. Our study highlights the hidden costs of English-centric alignment and offer a new path toward building more globally equitable and trustworthy LLMs by looking beyond the final layer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑