发表机构
School of Computation, Information and Technology, TU Munich; Munich Center for Machine Learning (MCML); Department of Computer Science, University of Copenhagen（慕尼黑工业大学计算、信息与技术学院; 慕尼黑机器学习中心; 哥本哈根大学计算机科学系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出通过在平行多语言表示上训练特定层自编码器并应用推理时修正的跨语言隐空间干预方法，在不降低事实准确率的前提下提升了LLMs跨语言事实一致性。
AI 中文摘要
大型语言模型（LLMs）在不同语言中对同一事实问题的回答往往存在差异。本文研究跨语言隐空间干预是否能减少这种不一致性。我们在平行多语言表示上训练特定层的自编码器（AE），并对事实问答（QA）提示应用推理时修正。研究发现，隐空间干预可改善语言间的几何对齐，且这种改善转化为与英语在开放式和多项选择QA格式下的跨语言一致性的稳定提升，同时不会降低事实准确率。在开放式QA中，英语与非英语语言间的斯皮尔曼等级相关系数大幅提升，英语-阿拉伯语对提升0.16，英语-俄语对提升0.20；在多项选择QA中，与英语的答案一致性在KLAR和mParaRel数据集上均稳定提升。消融实验表明，自编码器（AE）重建可在不损失准确率的情况下实现稳定提升，而主成分分析（PCA）投影贡献较小，均值偏移在开放式QA中可带来显著更大的一致性提升，但会损失部分准确率。
英文摘要
Large Language Models (LLMs) often answer the same factual question differently across languages. We study whether cross-lingual latent-space intervention can reduce this inconsistency. We train layer-specific autoencoders on parallel multilingual representations and apply inference-time corrections to factual QA prompts. We find that latent intervention improves geometric alignment between languages, and that this improvement translates into consistent gains in cross-lingual consistency with English across both open-ended and multiple-choice QA formats, without degrading factual accuracy. In open-ended QA, Spearman's rank correlation between English and non-English languages improves substantially, with gains of 0.16 for English-Arabic and 0.20 for English-Russian pairs. In multiple-choice QA, answer agreement with English improves consistently across both KLAR and mParaRel. Ablations show that AE reconstruction yields consistent gains at no accuracy cost, while PCA projection contributes marginally, and mean-shift produces substantially larger consistency gains in open-ended QA at the cost of some accuracy.
CommentsAccepted at EMNLP 2026
