面向大型语言模型的联邦数据高效指令微调
Federated Data-Efficient Instruction Tuning for Large Language Models
AI总结：
提出联邦数据高效指令微调方法FedHDS，通过减少客户端内外数据冗余选取代表性子集，在仅用不到1.5%数据下使未见任务Rouge-L平均提升10.72%，显著提升训练效率。
摘要：
指令微调是提升预训练大型语言模型（LLMs）对人类指令响应能力的关键步骤。联邦学习（FL）有助于利用来自客户端的海量私有指令数据，通过提升数据多样性而在LLM微调中日益流行。现有的联邦微调方法简单消耗所有本地数据，导致过高的计算开销以及对本地数据的过拟合，而集中式的数据高效解决方案由于隐私顾虑不适用于FL。本工作提出FedHDS，一种联邦数据高效指令微调方法，该方法使用边缘端数据的代表性子集对LLM进行微调。它在不共享原始数据的情况下，减少了客户端内部和客户端之间的数据冗余。在多种LLM、数据集和数据划分上的实验表明，FedHDS在未见任务上的Rouge-L较SOTA全量数据联邦指令微调方法平均提升10.72%，同时使用不到1.5%的数据样本，将训练效率提升多达数十倍。
英文摘要：
Instruction tuning is a crucial step in improving the responsiveness of pretrained large language models (LLMs) to human instructions. Federated learning (FL) helps to exploit the use of vast private instruction data from clients, becoming popular for LLM tuning by improving data diversity. Existing federated tuning simply consumes all local data, causing excessive computational overhead and overfitting to local data, while centralized data-efficient solutions are not suitable for FL due to privacy concerns. This work presents FedHDS, a federated data-efficient instruction tuning approach, which tunes LLMs with a representative subset of edge-side data. It reduces the data redundancy at both intra- and inter-client levels without sharing raw data. Experiments with various LLMs, datasets and partitions show that FedHDS improves Rouge-L on unseen tasks by an average of 10.72% over the SOTA full-data federated instruction tuning methods, while using less than 1.5% of the data samples, improving training efficiency by up to tens of times.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
