COMPASS：持续多语言PEFT与自适应语义采样
COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling
AI总结：
COMPASS通过自适应语义采样策略，利用参数高效微调方法，在多语言任务中提升模型性能，减少跨语言干扰，实现持续学习框架以适应动态环境。
摘要：
大型语言模型（LLMs）在不同语言间常表现出性能差异，传统多语言微调常因负向跨语言干扰而降级。为解决此问题，我们引入COMPASS（COntinual Multilingual PEFT with Adaptive Semantic Sampling），一种数据导向的框架，通过在精心选择的辅助多语言数据子集上训练轻量级语言特定适配器，利用多语言嵌入和聚类识别语义间隙，优先采样欠代表的语义簇以最大化正向跨语言迁移。我们将其扩展为持续学习框架COMPASS-ECDA，监控生产中的数据分布变化并动态更新适配器以防止模型老化，平衡新数据适应与现有知识保留。在三种模型架构（Phi-4-Mini, Llama-3.1-8B, Qwen2.5-7B）和多个挑战性多语言基准（Global-MMLU, MMLU-ProX）及未见长上下文任务（OneRuler）上，COMPASS一致优于基于语言相似度的基线方法，提供了一种有效、高效且可持续的多语言模型开发和维护解决方案。
英文摘要：
Large language models (LLMs) often exhibit performance disparities across languages, with naive multilingual fine-tuning frequently degrading performance due to negative cross-lingual interference. To address this, we introduce COMPASS (COntinual Multilingual PEFT with Adaptive Semantic Sampling), a novel data-centric framework for adapting LLMs to target languages. COMPASS leverages parameter-efficient fine-tuning (PEFT) by training lightweight, language-specific adapters on a judiciously selected subset of auxiliary multilingual data. The core of our method is a distribution-aware sampling strategy that uses multilingual embeddings and clustering to identify semantic gaps between existing training data and a target usage distribution. By prioritizing auxiliary data from under-represented semantic clusters, COMPASS maximizes positive cross-lingual transfer while minimizing interference. We extend this into a continual learning framework, COMPASS-ECDA, which monitors for data distribution shifts in production and dynamically updates adapters to prevent model staleness, balancing adaptation to new data with the preservation of existing knowledge. Across three different model architectures (Phi-4-Mini, Llama-3.1-8B, and Qwen2.5-7B) and multiple challenging multilingual benchmarks (Global-MMLU, MMLU-ProX), including unseen long-context tasks (OneRuler), we demonstrate that COMPASS consistently outperforms baseline methods guided by linguistic similarity, providing an effective, efficient, and sustainable solution for developing and maintaining high-performing multilingual models in dynamic environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
