打破物理与语言边界：面向低资源语言的多语言联邦提示微调
Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages
AI总结：
本文提出多语言联邦提示微调范式，在遵守数据共享限制的同时利用参数高效微调，实验表明其相比传统本地跨语言迁移方法准确率提升6.9%，并显著改善低资源语言的数据效率和跨语言互增强能力。
摘要：
预训练大语言模型（LLMs）已成为现代自然语言处理的基石，其能力广泛覆盖各类应用和语言。然而，多语言LLMs的微调，尤其是针对低资源语言的微调，面临来自数据共享限制（物理边界）和固有语言差异（语言边界）所带来的重大挑战。这些障碍使得不同语言的用户，尤其是低资源地区的用户，难以充分受益于LLMs的优势。为应对这些挑战，我们提出了面向多语言场景的联邦提示微调范式（Federated Prompt Tuning Paradigm），该范式在遵守数据共享限制的同时利用参数高效微调。我们设计了一组全面的实验，并使用一种新颖的语言距离概念对其进行分析，以突出我们范式的优势：即使在计算资源受限的情况下，我们的方法不仅提高了数据效率，还促进了语言之间的相互增强，尤其使低资源语言受益。与传统的本地跨语言迁移微调方法相比，我们的方法在提高数据效率的同时实现了6.9%的更高准确率，并展现出更强的稳定性和泛化能力。这些发现凸显了我们的方法在促进社会公平和倡导语言多样性方面的潜力，确保没有语言被落下。
英文摘要：
Pre-trained large language models (LLMs) have become a cornerstone of modern natural language processing, with their capabilities extending across a wide range of applications and languages. However, the fine-tuning of multilingual LLMs, especially for low-resource languages, faces significant challenges arising from data-sharing restrictions (the physical border) and inherent linguistic differences (the linguistic border). These barriers hinder users of various languages, particularly those in low-resource regions, from fully benefiting from the advantages of LLMs. To address these challenges, we propose the Federated Prompt Tuning Paradigm for multilingual scenarios, which utilizes parameter-efficient fine-tuning while adhering to data sharing restrictions. We design a comprehensive set of experiments and analyze them using a novel notion of language distance to highlight the strengths of our paradigm: Even under computational constraints, our method not only improves data efficiency but also facilitates mutual enhancements across languages, particularly benefiting low-resource ones. Compared to traditional local cross-lingual transfer tuning methods, our approach achieves 6.9\% higher accuracy with improved data efficiency, and demonstrates greater stability and generalization. These findings underscore the potential of our approach to promote social equality and champion linguistic diversity, ensuring that no language is left behind.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
