发表机构
University of Notre Dame; Vanderbilt University; University of Pennsylvania（圣母大学; 范德堡大学; 宾夕法尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究在合成数据用于指令微调时避免模型崩溃的问题，提出KITE两阶段框架，结合失败引导数据生成与边界感知不确定性整理，实验表明该框架比合成数据基线能更稳定地提升模型性能。
AI 中文摘要
模型崩溃是从合成数据学习中的核心挑战：随着后期大语言模型在越来越多的模型生成数据上训练，性能可能因覆盖范围变窄和偏差累积而下降。现有工作主要研究如何限制这种下降。然而在迭代模型演进中，更有意义的目标是确保每个后续模型比其前身有所改进，这需要以对数据整理可行的粒度诊断崩溃。我们在合成数据自我改进用于指令微调中研究此问题。我们表明这种情况下的崩溃不只是性能均匀下降，还可能表现为能力两极分化，即合成训练强化已有强大技能而进一步削弱薄弱技能。基于此观察，我们提出KITE（通过探索进行知识边界指令微调），这是一个两阶段框架，将失败引导的数据生成与边界感知的不确定性整理相结合。跨多个数据集和多个开源大语言模型的实验表明，KITE比强大的合成数据基线产生更稳定的改进。
英文摘要
Model collapse is a central challenge in learning from synthetic data: as later-generation large language models (LLMs) are trained on an increasing proportion of model-generated data, performance can degrade due to narrowed coverage and accumulated bias. Existing work mainly studies how to bound this degradation. In iterative model evolution, however, the more meaningful objective is to ensure that each successive model improves over its predecessor, which requires diagnosing collapse at a granularity that is actionable for data curation. We study this problem in synthetic data self-improving for instruction tuning. We show that collapse in this setting is not simply uniform performance degradation, but can appear as a polarization of competence, where synthetic training reinforces already strong skills while further degrading weak ones. Motivated by this observation, we propose KITE (Knowledge-boundary Instruction Tuning via Exploration), a two-stage framework that combines failure-guided data generation with boundary-aware uncertainty curation. Experiments across several datasets and multiple open-source LLMs show that KITE yields more stable improvement than strong synthetic-data baselines.
