arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.13699cs.CL
收藏

面向增强大语言模型推理的无约束模型合并

Unconstrained Model Merging for Enhanced LLM Reasoning

The Hong Kong Polytechnic University（香港理工大学）；Zhejiang University（浙江大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Reallm Labs（Reallm实验室）；South China University of Technology（华南理工大学）；Simon Fraser University（西蒙菲莎大学）

Yiming Zhang, Baoyi He, Shengyu Zhang, Yuhao Fu, Qi Zhou, Zhijie Sang, Zijin Hong, Kejing Yang, Wenjun Wang, Jianbo Yuan, Guanghan Ning, Linyi Li, Chunlin Ji, Fei Wu, Hongxia Yang

备注：
Under review, correct typos
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出无约束模型合并框架，支持同构与异构LLM合并以增强推理能力，实验表明合并涌现的组合推理超越简单加性效应，为去中心化LLM发展奠定基础。

摘要：

近年来，构建领域专用大语言模型（LLMs）的进展取得了显著成功，尤其是在需要推理能力的任务中，例如对复杂关系进行逻辑推断以及多步骤问题求解。然而，由于需要专有数据和庞大的计算资源，创建一个强大的全能型大语言模型仍然具有挑战性。作为一种资源友好的替代方案，我们探索了将多个专家模型合并为单一LLM的潜力。现有的模型合并研究主要关注通用型LLM，而非领域专家模型，或者仅关注相同架构和规模的LLM。在本工作中，我们提出了一种无约束模型合并框架，该框架能够同时容纳同构和异构模型架构，并聚焦于推理任务。针对同构模型合并，我们设计了一种细粒度的逐层权重合并策略；而异构模型合并则建立在从指令-响应微调数据中推导出的概率分布知识之上。在7个基准和9个推理优化LLM上，我们揭示了关键发现：合并过程中涌现出的组合推理能力超越了简单的加性效应。我们提出，无约束模型合并可以作为去中心化LLM的基础，标志着从现有中心化LLM框架的显著进步。这一演进能够促进更广泛的参与，并激发人工智能领域的进一步发展，从而有效应对中心化模型所带来的限制。

英文摘要：

Recent advancements in building domain-specific large language models (LLMs) have shown remarkable success, especially in tasks requiring reasoning abilities like logical inference over complex relationships and multi-step problem solving. However, creating a powerful all-in-one LLM remains challenging due to the need for proprietary data and vast computational resources. As a resource-friendly alternative, we explore the potential of merging multiple expert models into a single LLM. Existing studies on model merging mainly focus on generalist LLMs instead of domain experts, or the LLMs under the same architecture and size. In this work, we propose an unconstrained model merging framework that accommodates both homogeneous and heterogeneous model architectures with a focus on reasoning tasks. A fine-grained layer-wise weight merging strategy is designed for homogeneous models merging, while heterogeneous model merging is built upon the probabilistic distribution knowledge derived from instruction-response fine-tuning data. Across 7 benchmarks and 9 reasoning-optimized LLMs, we reveal key findings that combinatorial reasoning emerges from merging which surpasses simple additive effects. We propose that unconstrained model merging could serve as a foundation for decentralized LLMs, marking a notable progression from the existing centralized LLM framework. This evolution could enhance wider participation and stimulate additional advancement in the field of artificial intelligence, effectively addressing the constraints posed by centralized models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑