利用非结构化文本数据开展大语言模型的联邦指令微调
Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models
AI总结：
针对联邦指令微调要求客户端具备结构化指令数据的不切实际问题，提出FedIT-U2S框架，通过基于检索的少样本生成将非结构化文本转为指令对再微调，拓宽应用场景且在多领域显著提升模型效果。
摘要：
联邦指令微调可让多个客户端协同微调一个共享的大语言模型（LLM），使其能够遵循人类指令，且无需直接共享原始数据。然而，现有研究提出的要求并不切实际：它们假设所有客户端都已具备指令微调数据（即结构化的指令-响应对），但客户端的数据通常是非结构化文本，因此需要大量人工标注才能满足要求。针对这一问题，我们提出了一种新颖且灵活的框架FedIT-U2S，它能自动将非结构化语料转化为结构化数据，用于联邦指令微调。FedIT-U2S包含两个关键步骤：（1）少样本指令微调数据生成：将每一条非结构化数据与若干示例组合，作为提示输入LLM以生成指令-响应对。为进一步提升灵活性，我们提出了一种基于检索的示例选择技术，可根据客户端数据条目与示例池的相关性自动选择示例，无需预先确定示例。（2）基于生成数据执行标准的联邦指令微调流程。总体而言，只要客户端拥有有价值的文本语料，FedIT-U2S就可应用于多种场景，拓宽了联邦指令微调的应用范围。我们在医学、知识、数学三个领域开展了一系列实验，结果表明所提出的FedIT-U2S相较于基础LLM能够持续带来显著提升。
英文摘要：
Federated instruction tuning enables multiple clients to collaboratively fine-tune a shared large language model (LLM) that can follow humans' instructions without directly sharing raw data. However, existing literature impractically requires that all the clients readily hold instruction-tuning data (i.e., structured instruction-response pairs), which necessitates massive human annotations since clients' data is usually unstructured text instead. Addressing this, we propose a novel and flexible framework FedIT-U2S, which can automatically transform unstructured corpus into structured data for federated instruction tuning. FedIT-U2S consists two key steps: (1) few-shot instruction-tuning data generation, where each unstructured data piece together with several examples is combined to prompt an LLM in generating an instruction-response pair. To further enhance the flexibility, a retrieval-based example selection technique is proposed, where the examples are automatically selected based on the relatedness between the client's data piece and example pool, bypassing the need of determining examples in advance. (2) A typical federated instruction tuning process based on the generated data. Overall, FedIT-U2S can be applied to diverse scenarios as long as the client holds valuable text corpus, broadening the application scope of federated instruction tuning. We conduct a series of experiments on three domains (medicine, knowledge, and math), showing that our proposed FedIT-U2S can consistently and significantly brings improvement over the base LLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
