像玩乐高一样合并LoRA:通过按秩聚类将LoRA的模块化推向极致
Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
AI总结:
针对现有LoRA组合方法需额外训练、合并技术未充分利用模块化导致性能下降的问题,提出LoRA-LEGO框架,通过按秩聚类将不同LoRA的最小语义单元分组为k个簇,以簇质心组装合并LoRA并优化规模,实验表明其合并性能优于现有方法。
摘要:
低秩适配(LoRA)因其模块化设计和在Huggingface等平台的广泛可用性,已成为将大型语言模型(LLM)微调至不同领域的流行技术。这种模块化引发了人们对组合多个LoRA以增强LLM能力的兴趣。然而,现有的LoRA组合方法主要关注需要额外训练的特定任务适配,当前的模型合并技术往往无法充分利用LoRA的模块化特性,导致参数干扰和性能下降。在本文中,我们研究了以更精细的粒度拆解和重组多个LoRA的可行性,类似于组装乐高积木。我们引入了最小语义单元(MSU)的概念,其中LoRA中每个秩对应的参数作为独立单元。这些MSU表现出置换不变性和连接-求和等价性,能够灵活组合以创建新的LoRA。基于这些见解,我们提出了LoRA-LEGO框架。该框架通过将不同LoRA的MSU分组为k个簇来进行按秩参数聚类。每个簇的质心作为代表性MSU,能够组装出秩调整为k的合并LoRA。此外,我们应用双重重加权策略来优化合并LoRA的规模。在各种基准上的实验表明,我们的方法在LoRA合并方面优于现有方法。
英文摘要:
Low-Rank Adaptation (LoRA) has emerged as a popular technique for fine-tuning large language models (LLMs) to various domains due to its modular design and widespread availability on platforms like Huggingface. This modularity has sparked interest in combining multiple LoRAs to enhance LLM capabilities. However, existing methods for LoRA composition primarily focus on task-specific adaptations that require additional training, and current model merging techniques often fail to fully leverage LoRA's modular nature, leading to parameter interference and performance degradation. In this paper, we investigate the feasibility of disassembling and reassembling multiple LoRAs at a finer granularity, analogous to assembling LEGO blocks. We introduce the concept of Minimal Semantic Units (MSUs), where the parameters corresponding to each rank in LoRA function as independent units. These MSUs demonstrate permutation invariance and concatenation-summation equivalence properties, enabling flexible combinations to create new LoRAs. Building on these insights, we propose the LoRA-LEGO framework. This framework conducts rank-wise parameter clustering by grouping MSUs from different LoRAs into $k$ clusters. The centroid of each cluster serves as a representative MSU, enabling the assembly of a merged LoRA with an adjusted rank of $k$. Additionally, we apply a dual reweighting strategy to optimize the scale of the merged LoRA. Experiments across various benchmarks demonstrate that our method outperforms existing approaches in LoRA merging.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。