Though Large Language Models (LLMs) have demonstrated the powerful capabilities of few-shot learning through prompting methods, supervised training is still necessary for complex reasoning tasks. Because of their extensive parameters and memory consumption, both Parameter-Efficient Fine-Tuning (PEFT) methods and Memory-Efficient Fine-Tuning methods have been proposed for LLMs. Nevertheless, the issue of large annotated data consumption, the aim of Data-Efficient Fine-Tuning, remains unexplored. One obvious way is to combine the PEFT method with active learning. However, the experimental results show that such a combination is not trivial and yields inferior results. Through probe experiments, such observation might be explained by two main reasons: uncertainty gap and poor model calibration. Therefore, in this paper, we propose a novel approach to effectively integrate uncertainty-based active learning and LoRA. Specifically, for the uncertainty gap, we introduce a dynamic uncertainty measurement that combines the uncertainty of the base model and the uncertainty of the full model during the iteration of active learning. For poor model calibration, we incorporate the regularization method during LoRA training to keep the model from being over-confident, and the Monte-Carlo dropout mechanism is employed to enhance the uncertainty estimation. Experimental results show that the proposed approach outperforms existing baseline models on three complex reasoning tasks.
SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRA
Shanghai Qi Zhi Institute(上海启智研究院);Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Xiongan AI Institute(雄安人工智能研究院)
MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(知识驱动人机智能工程研究中心,教育部,中国);International Center of Future Science, Jilin University(未来科学国际中心,吉林大学)
mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs
Sichuan University(四川大学);Ant Group(蚂蚁集团);Zhejiang computation(浙江计算);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);CNIC, Chinese Academy of Science(中国科学院计算机网络信息中心)
LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
Department of Mathematics, McMaster University, Hamilton, Canada(麦斯特大学数学系);Vector Institute, Toronto, Canada(向量研究所);University of Oxford, Oxford, United Kingdom(牛津大学)
Echo-LoRA: Parameter-Efficient Fine-Tuning via Cross-Layer Representation Injection
School of Computer Science and Software Engineering, Southwest Petroleum University(西南石油大学计算机科学与软件工程学院);School of Electronic Information and Artificial Intelligence, Leshan Normal University(乐山师范学院电子信息与人工智能学院)
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Georgia Institute of Technology(佐治亚理工学院);Massachusetts Institute of Technology(麻省理工学院)
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
Tulane University(杜兰大学);New York University(纽约大学);Stevens Institute of Technology(史蒂文斯理工学院);Northeastern University(东北大学);The Chinese University of Hong Kong(香港中文大学)
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Duke University(杜克大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Massachusetts Institute of Technology(麻省理工学院)
LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
School of Politics and Public Administration, South China Normal University(华南师范大学政治与公共管理学院);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Shenyang institute of computing technology, Chinese academy of sciences(中国科学院沈阳计算技术研究所)
Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
New York University(纽约大学);University of Minnesota(明尼苏达大学);Tulane University(杜兰大学);The Chinese University of Hong Kong(香港中文大学);Stevens Institute of Technology(史蒂文斯理工学院)