Recently, there has been a demand to deploy Large Language Models (LLMs) on personal devices such as laptops and smartphones. These LLMs have different model variants when handling different tasks. However, personal devices have limited resources and require reduced storage overhead. To address this, there are two key methods available: the first is model compression, which compresses LLMs into smaller sizes; the second is LoRA, which can transfer an LLM to other tasks with very few parameters, avoiding the storage of multiple model variants in multi-task scenarios by only preserving LoRAs. However, our experiments show that directly combining these two methods yields sub-optimal performance. Considering that the open-source community has already contributed many LoRAs to LLMs, we propose to adapt these existing LoRAs from the LLMs to their compressed version and introduce a Compression-Aware LoRA (CA-LoRA) framework. We incorporate knowledge inheritance and recovery strategies to recover the lost knowledge caused by model compression. Experiment results demonstrate that CA-LoRA outperforms the vanilla LoRA methods applied to a compressed LLM and achieves comparable performance to the non-compressed LLM with existing LoRA modules. The source code of CA-LoRA is available at https://github.com/thunlp/CA-LoRA.
P-React: Synthesizing Topic-Adaptive Reactions of Personality Traits via Mixture of Specialized LoRA Experts
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Shanghai Institute of AI for Education, East China Normal University(东华大学人工智能教育研究所);Xiaohongshu Inc.(小红书公司)
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Beihang University(北京航空航天大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Science and Technology Beijing(北京科技大学)
A Language-Guided Bayesian Optimization for Efficient LoRA Hyperparameter Search
Grad. School of AI, POSTECH, Pohang, Korea(POSTECH人工智能研究生院,韩国坡安);School of EE, KAIST, Daejeon, Korea(韩国科学技术院电子工程学院,韩国大田);School of Computing, KAIST, Daejeon, Korea(韩国科学技术院计算学院,韩国大田)