arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.05613cs.AI
收藏

通过局部梯度冲突解决的多语言微调

Multilingual Fine-Tuning via Localized Gradient Conflict Resolution

Singapore University of Technology and Design(新加坡科技设计大学);Salesforce AI Research(Salesforce人工智能研究);Nanyang Technological University(南洋理工大学)

Long P. Hoang, Yiran Zhao, Wei Lu, Wenxuan Zhang

AI总结:

提出Bucket-Level MOO框架,将多语言微调重构为多目标优化问题,通过局部梯度冲突解决提升多语言性能。

摘要:

大型语言模型(LLMs)的快速发展已将跨语言多功能性确立为现代系统的定义特征。然而,微调这些模型经常引发跨语言的负面干扰。为了解决这个问题,我们将多语言微调重构为多目标优化(MOO)问题。具体来说,我们引入了Bucket-Level MOO,一个可扩展的分布式框架,它在参数桶上局部应用基于梯度的MOO算法。这使得冲突感知更新成为可能,而无需重建完整梯度向量的高昂通信开销。理论上,我们证明了这种局部解决自然地强制执行精炼帕累托平稳性,这是帕累托最优性的一个严格更紧的必要条件。实验上,Bucket-Level MOO通过驱动LLMs构建特定的语言维度来减轻干扰,提高了表示的可分离性。在四个基础LLM上的广泛实验表明,我们的方法在标准微调范式上显著提高了所见和未见的多语言性能。

英文摘要:

Cross-lingual versatility has become a defining capability of Large Language Models (LLMs). However, fine-tuning these models often induces negative interference, where gains in one language come at another's expense. To address this, we cast multilingual fine-tuning as a multi-objective optimization (MOO) problem. Specifically, we introduce Bucket-Level MOO, a scalable distributed framework that applies gradient-based MOO algorithms locally, within the parameter buckets of distributed training. This enables conflict-aware updates without ever materializing a full gradient vector for any objective, and improves multilingual performance over standard fine-tuning by up to $4.3$ points. Theoretically, we define bucketwise Pareto stationarity, show it is necessary for Pareto optimality under any fixed partition, and prove descent and convergence for representative Bucket-Level variants. Our work brings conventional MOO within reach for multilingual LLMs at a far lower cost. More broadly, Bucket-Level MOO is agnostic to both the objectives and the update rule, readily extending to other multi-task learning at scale.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑