AIDE：面向任务特定微调中数据稀缺问题的属性引导多跳数据扩展
AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning
AI总结：
提出属性引导多跳数据扩展框架AIDE，通过提取主题与属性进行多跳合成并引入残差连接，仅需10个种子即可高效微调LLM，性能超越人类数据及现有方法。
摘要：
针对特定任务微调大型语言模型（LLMs）需要多样化、高质量的训练数据。然而，获取足够的训练数据仍然是一个重大挑战。现有的数据合成方法要么依赖大量的种子数据集，要么难以平衡任务相关性和数据多样性。为了应对这些挑战，我们提出了属性引导多跳数据扩展（AIDE），这是一种新颖的数据合成框架，它使用多跳过程来扩展极少数种子数据点，同时确保数据多样性和任务相关性。AIDE从种子中提取主要主题和关键知识属性，以指导合成步骤。该过程重复K跳，并将生成的数据作为种子。为了防止随着跳数深度增加而产生不相关的数据，AIDE引入了残差连接机制。我们的实验结果表明，AIDE能够仅从10个种子数据点微调Mistral-7B、Llama-3.1-8B和Llama-3.2-3B，其性能超越了在人类精选数据上微调的模型。此外，在任务特定微调中，AIDE的性能比Evolutionary-Instruct等最先进的数据合成方法高出30%以上。代码可在https://github.com/Code4Graph/AIDE获取。
英文摘要：
Fine-tuning large language models (LLMs) for specific tasks requires diverse, high-quality training data. However, obtaining sufficient relevant data remains a significant challenge. Existing data synthesis methods either depend on extensive seed datasets or struggle to balance task relevance and data diversity. To address these challenges, we propose Attribute-guided multI-hop Data Expansion (AIDE), a novel data synthesis framework that uses a multi-hop process to expand very few seed data points while ensuring data diversity and task relevance. AIDE extracts the main topic and key knowledge attributes from the seeds to guide the synthesis steps. The process repeats for K hops, using the generated data as seeds. To prevent irrelevant data generation as the hop depth increases, AIDE incorporates a residual connection mechanism. Our empirical results show that AIDE enables fine-tuning of Mistral-7B, Llama-3.1-8B and Llama-3.2-3B from 10 seeds, surpassing the models fine-tuned on human curated data. Furthermore, AIDE outperforms state-of-the-art data synthesis methods, such as Evol-Instruct, by over 30% in task-specific fine-tuning. Code is available at https://github.com/Code4Graph/AIDE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
