FLeX：基于傅里叶的低秩扩展用于多语言迁移
FLeX: Fourier-based Low-rank EXpansion for multilingual transfer
AI总结：
本文研究了通过低秩适应和优化器改进提升Python到Java等语言的跨语言迁移效果，发现低秩适应和傅里叶正则化能显著提升迁移性能。
摘要：
跨语言代码生成在企业环境中至关重要，但为每种语言单独微调大型语言模型（LLMs）计算成本过高。本文探讨了参数高效微调方法和优化器改进是否能提升从Python到Java等语言的跨语言迁移。我们使用低秩适应（LoRA）微调Code Llama 7B模型，优化少量参数，并比较Adam和Sophia优化器，同时探索一种新的基于傅里叶的正则化技术。我们的贡献包括：（1）证明在高质量的小数据集（MBPP）上使用LoRA微调可超越更广泛微调的Code Llama-Python-7B模型（40.1% vs. 38.4%）；（2）显示Sophia比Adam收敛更快，但最终pass@1分数差异微小；（3）证明在微调过程中使用傅里叶正则化显著提升跨语言迁移，实现Java任务42.1%的pass@1，优于34.2%的基线。这些发现表明，结合LoRA、优化训练方法和频域正则化可高效地将单语言LLM适应到多种编程语言。
英文摘要：
Cross-lingual code generation is critical in enterprise environments where multiple programming languages coexist. However, fine-tuning large language models (LLMs) individually for each language is computationally prohibitive. This paper investigates whether parameter-efficient fine-tuning methods and optimizer enhancements can improve cross-lingual transfer from Python to languages like Java. We fine-tune the Code Llama 7B model using low-rank adaptation (LoRA) to optimize a small subset of parameters and compare Adam and Sophia optimizers, while exploring a novel Fourier-based regularization technique. Our contributions include: (1)demonstrating that LoRA fine-tuning on a small, high-quality dataset (MBPP) can exceed the pass@1 performance of the more broadly fine-tuned Code Llama-Python-7B model (40.1% vs. 38.4%); (2) showing that while Sophia achieves faster convergence than Adam, final pass@1 scores show marginal differences; and (3) presenting evidence that Fourier-based regularization during fine-tuning significantly improves cross-lingual transfer, achieving 42.1% pass@1 on Java tasks compared to the 34.2% baseline. These findings suggest that combining LoRA, optimized training methods, and frequency-domain regularization can efficiently adapt single-language LLMs to perform well across multiple programming languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
