Ada-Instruct：面向复杂推理的自适应指令生成器
Ada-Instruct: Adapting Instruction Generators for Complex Reasoning
AI总结：
针对Self-Instruct无法生成长复杂指令的缺陷，本文提出通过少量示例微调开源LLM的自适应指令生成器Ada-Instruct，能够生成与复杂推理任务分布一致的长指令。
摘要：
指令扩充是释放大语言模型（LLMs）在下游任务中全部潜力的关键步骤。现有的Self-Instruct方法主要依靠上下文学习，从少量初始指令中模拟生成新指令。然而，我们的研究发现该方法存在一个关键缺陷：即使使用GPT4o，Self-Instruct也无法生成长度≥100的复杂指令，而这类指令在代码补全等复杂任务中是必需的。为解决这一问题，我们的核心见解是：仅用十个示例对开源LLMs进行微调，即可生成在复杂推理任务中保持分布一致性的复杂指令。我们提出了Ada-Instruct，一种通过微调开发的自适应指令生成器。我们在不同应用中实证验证了Ada-Instruct的有效性。结果表明，Ada-Instruct具备生成长、复杂且分布一致的指令的能力。
英文摘要：
Instructions augmentation is a crucial step for unleashing the full potential of large language models (LLMs) in downstream tasks. Existing Self-Instruct methods primarily simulate new instructions from a few initial instructions with in-context learning. However, our study identifies a critical flaw in this approach: even with GPT4o, Self-Instruct cannot generate complex instructions of length $\ge 100$, which is necessary in complex tasks such as code completion. To address this issue, our key insight is that fine-tuning open source LLMs with only ten examples can produce complex instructions that maintain distributional consistency for complex reasoning tasks. We introduce Ada-Instruct, an adaptive instruction generator developed through fine-tuning. We empirically validated Ada-Instruct's efficacy across different applications. The results highlight Ada-Instruct's capacity to generate long, intricate, and distributionally consistent instructions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
