本文提出Chain of LoRA（COLA）迭代优化框架，通过残差学习合并并重置LoRA模块，在不增加计算和内存开销的情况下弥合LoRA与全参数微调的差距，在多种模型和七个基准任务上持续优于LoRA。
摘要：
微调是将预训练大型语言模型适配到特定任务的主要方法。随着模型规模和任务多样性的扩大，参数高效微调方法变得至关重要。其中最广泛使用的一类方法是低秩适应（LoRA）及其变体。LoRA将权重更新编码为两个低秩矩阵的乘积。尽管具有优势，但在某些任务的泛化误差方面，LoRA仍不及全参数微调。我们引入了Chain of LoRA（COLA），这是一个受Frank-Wolfe算法启发的迭代优化框架，旨在弥合LoRA与全参数微调之间的差距，且不会产生额外的计算成本或内存开销。COLA采用残差学习过程，将学习到的LoRA模块合并到预训练语言模型参数中，并为新生的LoRA模块重新初始化优化。我们提供了理论收敛保证以及实验结果来验证算法的有效性。在多种模型（OPT和llama-2）及七个基准任务上，我们证明COLA能够在没有额外计算或内存成本的情况下持续优于LoRA。
英文摘要：
Fine-tuning is the primary methodology for tailoring pre-trained large language models to specific tasks. As the model's scale and the diversity of tasks expand, parameter-efficient fine-tuning methods are of paramount importance. One of the most widely used family of methods is low-rank adaptation (LoRA) and its variants. LoRA encodes weight update as the product of two low-rank matrices. Despite its advantages, LoRA falls short of full-parameter fine-tuning in terms of generalization error for certain tasks.
We introduce Chain of LoRA (COLA), an iterative optimization framework inspired by the Frank-Wolfe algorithm, to bridge the gap between LoRA and full parameter fine-tuning, without incurring additional computational costs or memory overheads. COLA employs a residual learning procedure where it merges learned LoRA modules into the pre-trained language model parameters and re-initilize optimization for new born LoRA modules. We provide theoretical convergence guarantees as well as empirical results to validate the effectiveness of our algorithm. Across various models (OPT and llama-2) and seven benchmarking tasks, we demonstrate that COLA can consistently outperform LoRA without additional computational or memory costs.
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
Tulane University（杜兰大学）；New York University（纽约大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Northeastern University（东北大学）；The Chinese University of Hong Kong（香港中文大学）
LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
School of Politics and Public Administration, South China Normal University（华南师范大学政治与公共管理学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shenyang institute of computing technology, Chinese academy of sciences（中国科学院沈阳计算技术研究所）
Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
New York University（纽约大学）；University of Minnesota（明尼苏达大学）；Tulane University（杜兰大学）；The Chinese University of Hong Kong（香港中文大学）；Stevens Institute of Technology（史蒂文斯理工学院）
LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Nanjing University（南京大学）
SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRA
Shanghai Qi Zhi Institute（上海启智研究院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Xiongan AI Institute（雄安人工智能研究院）
Task-Specific Directions: Definition, Exploration, and Utilization in Parameter Efficient Fine-Tuning
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（人工智能基础理论实验室，人工智能研究院，上海交通大学）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（电子信息与电气工程学院，上海交通大学）；School of Engineering and Applied Sciences, Harvard University（工程与应用科学学院，哈佛大学）
RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks
Singapore University of Technology and Design（新加坡科技设计大学）；LinkedIn（领英公司）；Technical University of Munich（慕尼黑工业大学）；Texas A&M University（德克萨斯农工大学）；Rutgers University（罗格斯大学）
MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China（知识驱动人机智能工程研究中心，教育部，中国）；International Center of Future Science, Jilin University（未来科学国际中心，吉林大学）
How Relevance Emerges: Interpreting LoRA Fine-Tuning in Reranking LLMs
Center for Intelligent Information Retrieval, Manning College of Information and Computer Sciences, University of Massachusetts Amherst（智能信息检索中心，信息与计算机科学学院，马萨诸塞大学阿姆赫斯特分校）
MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Baidu Inc.（百度公司）
mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs
Sichuan University（四川大学）；Ant Group（蚂蚁集团）；Zhejiang computation（浙江计算）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；CNIC, Chinese Academy of Science（中国科学院计算机网络信息中心）
