InsBank: 面向持续对齐的指令子集演化
InsBank: Evolving Instruction Subset for Ongoing Alignment
AI总结：
针对大语言模型持续对齐中指令子集演化研究不足的问题，提出持续更新的指令库InsBank及渐进式演化框架PIBE，通过渐进选择、表征多样性得分等实现高效演化，实验证明其效果与适应性优于基线。
摘要：
大语言模型（LLM）通常需经过指令微调以提升对齐效果。近期研究强调，指令数据的质量与多样性比数量更为关键，凸显了筛选多样化、高质量子集以降低训练成本的必要性。然而，如何随着新指令数据的涌现持续演化这些已筛选的子集，目前仍缺乏充分研究。为实现大语言模型的持续对齐，我们提出了指令库（Instruction Bank，简称InsBank）——一个整合最新高价值指令数据的持续更新仓库。我们进一步提出渐进式指令库演化框架（Progressive Instruction Bank Evolution，简称PIBE），这是一种旨在随时间高效且有效地演化InsBank的新型框架。PIBE采用渐进式数据选择策略以维持长期效率，利用基于表征的多样性得分捕捉数据点间的关联，并保留历史信息以进行全面的多样性评估。该框架还支持在数据选择和排序过程中灵活结合多样性得分与质量得分。大量实验表明，PIBE在InsBank演化任务上的表现显著优于基线方法，且能够提取符合预算要求的特定子集，展现出其有效性与适应性。
英文摘要：
Large language models (LLMs) typically undergo instruction tuning to enhance alignment. Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. However, how to evolve these selected subsets alongside the development of new instruction data remains insufficiently explored. To achieve LLMs' ongoing alignment, we introduce Instruction Bank (\textbf{InsBank}), a continuously updated repository that integrates the latest valuable instruction data. We further propose Progressive Instruction Bank Evolution (\textbf{PIBE}), a novel framework designed to evolve InsBank effectively and efficiently over time. PIBE employs a gradual data selection strategy to maintain long-term efficiency, leveraging a representation-based diversity score to capture relationships between data points and retain historical information for comprehensive diversity evaluation. This also allows for flexible combination of diversity and quality scores during data selection and ranking. Extensive experiments demonstrate that PIBE significantly outperforms baselines in InsBank evolution and is able to extract budget-specific subsets, demonstrating its effectiveness and adaptability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
