发表机构
The University of Tokyo（东京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SLIM通过混料设计构建二次代理模型优化LLM合并系数，以最少评估实现准确预测和竞争性合并性能。
AI 中文摘要
优化大型语言模型的合并系数可能需要大量昂贵的基准评估。我们提出单纯形格插值合并（SLIM），该方法利用经典混料设计在系数单纯形上构建聚合性能的二次代理模型。对单个专家和等权配对进行评估，以最少测量次数确定该域上一般二次函数所需的代理模型。SLIM随后在无需进一步目标指标评估的情况下优化该代理模型。在两个模型架构上的实验表明，SLIM能够准确预测未见过的多专家混合，并在有限评估预算下实现具有竞争力的合并性能。匹配预算的比较显示，结构化评估点相比随机设计（包括使用正则化拟合的设计）能提高预测保真度。
英文摘要
Optimizing merging coefficients for large language models can require many costly benchmark evaluations. We propose \textbf{Simplex-Lattice Interpolation Merging (SLIM)}, which constructs a quadratic surrogate of aggregate performance on the coefficient simplex using a classical mixture design. Evaluations of individual experts and equal-weight pairs determine the surrogate with the minimum number of measurements needed to identify a general quadratic on this domain. SLIM then optimizes the surrogate without further target-metric evaluations. Experiments on two model architectures demonstrate accurate prediction of unseen multi-expert mixtures and competitive merge performance under limited evaluation budgets. Matched-budget comparisons show that structured evaluation points improve prediction fidelity over random designs, including those using regularized fitting.
