arXiv 2503.012752025-03-05cs.CL

通过深度监督微调增强以英语为中心的大语言模型的非英语能力

Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning

Harbin Institution of Technology（哈尔滨工业大学）；Pengcheng Laboratory（鹏城实验室）；Huawei Technologies Co., Ltd（华为技术有限公司）

Wenshuai Huo, Xiaocheng Feng, Yichong Huang, Chengpeng Fu, Baohang Li, Yangfan Ye, Zhirui Zhang, Dandan Tu, Duyu Tang, Yunfei Lu, Hui Wang, Bing Qin

备注：: Accepted at AAAI 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出深度监督微调方法（DFT），通过在LLaMA-2和Gemma-2的底层和中间层分别引入logits与feature监督信号，约束非英语向英语的转换与推理，显著提升了以英语为中心的大语言模型的非英语能力。

摘要：

大语言模型（LLMs）在多语言理解和生成方面取得了显著进展。然而，由于训练数据的不平衡，其在非英语语言上的能力有限。近期研究揭示了LLMs的英语枢纽多语言机制，即LLMs在底层隐式地将非英语查询转换为英语，并在中间层采用英语进行思考。然而，由于LLMs中间层缺乏跨语言对齐的显式监督，这些阶段的内部表示可能变得不准确。在本工作中，我们引入了一种深度监督微调方法（DFT），在模型内部层引入额外监督以指导其工作流程。具体而言，我们在LLMs的不同层引入两个训练目标：一个在底层约束目标语言向英语的转换，另一个在中间层约束英语推理。为有效实现指导目的，我们设计了两种监督信号：logits和feature，分别代表较严格的约束和相对宽松的指导。我们的方法引导模型在处理非英语输入时不仅考虑最终生成结果，还确保内部表示的准确性。我们在典型的以英语为中心的大模型LLaMA-2和Gemma-2上进行了大量实验，多个多语言数据集上的结果表明，我们的方法显著优于传统微调方法。

英文摘要：

Large language models (LLMs) have demonstrated significant progress in multilingual language understanding and generation. However, due to the imbalance in training data, their capabilities in non-English languages are limited. Recent studies revealed the English-pivot multilingual mechanism of LLMs, where LLMs implicitly convert non-English queries into English ones at the bottom layers and adopt English for thinking at the middle layers. However, due to the absence of explicit supervision for cross-lingual alignment in the intermediate layers of LLMs, the internal representations during these stages may become inaccurate. In this work, we introduce a deep supervision fine-tuning method (DFT) that incorporates additional supervision in the internal layers of the model to guide its workflow. Specifically, we introduce two training objectives on different layers of LLMs: one at the bottom layers to constrain the conversion of the target language into English, and another at the middle layers to constrain reasoning in English. To effectively achieve the guiding purpose, we designed two types of supervision signals: logits and feature, which represent a stricter constraint and a relatively more relaxed guidance. Our method guides the model to not only consider the final generated result when processing non-English inputs but also ensure the accuracy of internal representations. We conducted extensive experiments on typical English-centric large models, LLaMA-2 and Gemma-2, and the results on multiple multilingual datasets show that our method significantly outperforms traditional fine-tuning methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。