从实例选择到固定池数据配方搜索用于监督微调
From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
AI总结：
本文提出固定池数据配方搜索方法，通过结合过滤、混合和去重操作，高效生成高质量监督微调数据集，实验显示其优于全数据训练和随机搜索等方法。
摘要：
监督微调（SFT）数据选择通常被公式化为实例排序：对每个示例评分并保留前k个子集。然而，有效的SFT训练子集通常通过有序的curate配方生成，其中过滤、混合和去重操作共同塑造最终的数据分布。我们将其问题公式化为固定池数据配方搜索：给定一个原始指令池和一个接地操作库，目标是发现一个可执行的配方，以在有限的全SFT评估预算下构建高质量的选定子集，而无需生成、重写或增强训练样本。我们引入了AutoSelection，一个两层求解器，通过缓存任务、数据和模型侧信号解耦固定池的材料化，利用预热探针、实现的子集状态、局部配方编辑、高斯过程辅助排序和停滞触发重新播种。在90,000个指令池的实验中，AutoSelection在三个基础模型上实现了最强的分布内推理平均值，优于全数据训练、随机配方搜索、随机top-k和单操作选择器。额外的分布外图推理结果、搜索稳定性分析、结构消融和1.5B到7B的转移检查进一步表明，配方结构在个体选择操作符之外也起作用。代码可在https://github.com/w253/AutoSelection获取。
英文摘要：
Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top-$k$ subset. However, effective SFT training subsets are often produced through ordered curation recipes, where filtering, mixing, and deduplication operators jointly shape the final data distribution. We formulate this problem as fixed-pool data recipe search: given a raw instruction pool and a library of grounded operators, the goal is to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations, without generating, rewriting, or augmenting training samples. We introduce AutoSelection, a two-layer solver that decouples fixed-pool materialization based on cached task-, data-, and model-side signals from expensive full evaluation, using warmup probes, realized subset states, local recipe edits, Gaussian-process-assisted ranking, and stagnation-triggered reseeding. Experiments on a 90K instruction pool show that AutoSelection achieves the strongest in-distribution reasoning average across three base models, outperforming full-data training, random recipe search, random top-$k$, and single-operator selectors. Additional Out-of-distribution graph-reasoning results, search-stability analyses, structural ablations, and 1.5B-to-7B transfer checks further show that recipe structure matters beyond individual selection operators. Code is available at https://github.com/w253/AutoSelection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
