通过任务能力共进化发现新型大语言模型专家
Discovering Novel LLM Experts via Task-Capability Coevolution
AI总结：
本文提出AC/DC框架，通过模型合并和合成数据生成实现大语言模型与任务的共进化，发现超越大模型能力的新型LLM，无需显式优化基准，在多智能体最佳N选择中提升性能。
摘要：
前沿模型开发者旨在持续训练模型以获得新兴、多样的能力。为扩展能力，当前预训练和后训练范式需要每次手动启动训练运行，使用静态数据集或奖励函数。针对这一限制，我们的工作提出开放性（通过模型与任务的共进化）可在单次运行中发现具备越来越新技能的模型。我们引入了一个新的模型开发框架，将共进化扩展到大语言模型（LLM）发现，即开放性的Assessment Coevolving with Diverse Capabilities（AC/DC）。AC/DC通过模型合并和自然语言任务的合成数据生成进化LLM和任务。AC/DC发现不断增长的LLM档案，这些LLM在不占用大量GPU内存的情况下超越了更大LLM的能力。特别是，我们的LLM种群在下游基准测试中比其他精选模型或基线具有更广的专长覆盖范围，而无需任何显式基准优化。此外，AC/DC随时间推移提高覆盖范围，持续在任务和模型上创新，并在多智能体最佳N选择中提升性能。我们的发现突显了共进化作为从基础LLM中发现更广泛能力集的潜力。总体而言，AC/DC使我们更接近一种新的LLM开发范式，其中通过利用现有模型作为跳板来加速模型能力多样性的持续改进。
英文摘要：
Frontier model developers aim to train models continually to possess emergent, diverse capabilities. To extend capabilities, the current pre-training and post-training paradigm requires manually starting training runs with static datasets or reward functions every time. Addressing this limitation, our work pursues the insight that open-endedness (via the coevolution of models and tasks) can discover models with increasingly novel skills in a single run. We introduce a new model development framework that extends coevolution to large language model (LLM) discovery, open-ended \textit{Assessment Coevolving with Diverse Capabilities} (AC/DC). AC/DC evolves both LLMs via model merging and natural language tasks via synthetic data generation. AC/DC discovers growing archives of LLMs that surpass the capabilities of larger LLMs while taking up less GPU memory. In particular, our LLM populations achieve a broader Coverage of expertise than other curated models or baselines on downstream benchmarks, without \textit{any} explicit benchmark optimization. Furthermore, AC/DC improves Coverage over time, continually innovates on tasks and models, and improves performance in multi-agent best-of-N selection. Our findings highlight the potential of coevolution as a means of discovering broader sets of capabilities from base LLMs. Overall, AC/DC brings us one step closer to a profoundly new paradigm of LLM development, where continual improvements to the diversity of model capabilities can be accelerated by leveraging existing models as stepping stones to increasingly powerful models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
