参数高效指令微调：一项实证研究
Parameter Efficient Instruction Tuning: An Empirical Study
AI总结：
针对预训练语言模型全参数指令微调成本过高问题，本研究系统探究代表性参数高效微调（PEFT）方法，揭示其超参数、模型规模等影响，发现理想设置下LoRA和适配器接近全微调效果。
摘要：
指令微调已成为微调预训练语言模型以更好遵循人类指令并在各类任务上泛化的重要步骤。如今，预训练语言模型日益庞大，全参数微调成本极高。因此，参数高效微调（PEFT）因相较于全微调在计算、内存和存储成本上显著更低，成为指令微调的经济高效实践。尽管PEFT方法应用广泛，但庞大的超参数空间、PEFT方法数量以及指令微调能力的不同侧重点，使得厘清各方面影响较为困难。本研究系统调查了几种代表性PEFT方法，探究超参数选择（包括训练超参数和PEFT特定超参数）的影响、不同模型规模和指令任务数量对性能的作用，以及任务内分布记忆和开放式指令遵循能力。实证研究表明，仅LoRA和适配器在理想训练设置下能接近全微调效果；理想设置包括合适的学习率、允许的最大LoRA秩或适配器尺寸以及多样化训练任务。另一方面，若未满足理想训练条件，LoRA和适配器会出现训练不稳定问题。此外，LoRA需更多任务以实现有效未见任务泛化，学习速度较慢，且任务级记忆能力较弱。最后，在开放式指令微调设置中，LoRA和适配器在复杂推理、编码和长文本生成上逊于全微调，但LoRA能力强于适配器。
英文摘要：
Instruction tuning has become an important step for finetuning pretrained language models to better follow human instructions and generalize on various tasks. Nowadays, pretrained language models become increasingly larger, and full parameter finetuning is overwhelmingly costly. Therefore, Parameter Efficient Finetuning (PEFT) has arisen as a cost-effective practice for instruction tuning because of significantly smaller computational, memory, and storage cost compared to full finetuning. Despite their widespread adaptations, the vast hyperparameter spaces, the number of PEFT methods, the different focus of instruction tuning capabilities make disentangling the impact of each aspect difficult. This study systematically investigates several representative PEFT methods, surveying the effect of hyperparameter choices including training hyperparameters and PEFT-specific hyperparameters, how different models sizes and the number of instruction tasks affect the performance, in-task-distribution memorization and open instruction following capability. Our empirical study shows that only LoRA and adapter can get close to full finetuning with ideal training settings. The ideal training setting includes an appropriate learning rate, largest LoRA rank or adapter size allowed and diverse training tasks. On the other hand, LoRA and adapter suffer from training instability if such an ideal training condition is not met. Additionally, LoRA requires a greater number of tasks for effective unseen task generalization, exhibit slower learning speed. Moreover, LoRA has weaker task-level memorization. Lastly, LoRA and adapter fall short in complex reasoning, coding and long-form generation compared to finetuning in open instruction tuning settings but it shows stronger capabilities compared to adapter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
