arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.01415cs.AIcs.LG
收藏

条件化 LoRA 参数生成

Conditional LoRA Parameter Generation

National University of Singapore(新加坡国立大学);Shanghai Jiao Tong University(上海交通大学)

Xiaolong Jin, Kai Wang, Dongwen Tang, Wangbo Zhao, Yukun Zhou, Junshu Tang, Yang You

AI总结:

本文提出 COND P-DIFF,通过自编码器与条件潜在扩散模型实现可控的高性能 LoRA 参数生成,并在视觉和语言任务中验证其有效性与泛化能力。

摘要:

生成模型已在图像、视频和文本领域取得了显著成功。受此启发,研究人员探索利用生成模型来生成神经网络参数。然而,这些努力一直受到参数规模以及生成高性能参数实用性的限制。在本文中,我们提出 COND P-DIFF,一种新颖的方法,展示了可控高性能参数生成的可行性,特别是在微调过程中针对 LoRA(低秩适配)权重。具体来说,我们采用自编码器来提取参数的高效潜在表示。然后,我们训练一个条件潜在扩散模型,根据特定任务条件从随机噪声中合成高性能模型参数。在计算机视觉和自然语言处理领域的实验结果一致表明,COND P-DIFF 能够基于给定任务生成高性能参数。此外,我们观察到 COND P-DIFF 生成的参数分布与通过常规优化方法获得的分布存在差异,表明其具有一定的泛化能力。我们的工作为进一步探索条件驱动的参数生成铺平了道路,为神经网络的任务特定适配提供了一个有前景的方向。

英文摘要:

Generative models have achieved remarkable success in image, video, and text domains. Inspired by this, researchers have explored utilizing generative models to generate neural network parameters. However, these efforts have been limited by the parameter size and the practicality of generating high-performance parameters. In this paper, we propose COND P-DIFF, a novel approach that demonstrates the feasibility of controllable high-performance parameter generation, particularly for LoRA (Low-Rank Adaptation) weights, during the fine-tuning process. Specifically, we employ an autoencoder to extract efficient latent representations for parameters. We then train a conditional latent diffusion model to synthesize high-performing model parameters from random noise based on specific task conditions. Experimental results in both computer vision and natural language processing domains consistently demonstrate that COND P-DIFF can generate high-performance parameters conditioned on the given task. Moreover, we observe that the parameter distribution generated by COND P-DIFF exhibits differences compared to the distribution obtained through normal optimization methods, indicating a certain level of generalization capability. Our work paves the way for further exploration of condition-driven parameter generation, offering a promising direction for task-specific adaptation of neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑