arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.07990cs.CL
收藏

FuseChat:聊天模型的知识融合

FuseChat: Knowledge Fusion of Chat Models

School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院)

Fanqi Wan, Longguang Zhong, Ziyi Yang, Ruijun Chen, Xiaojun Quan

AI总结:

本研究提出FuseChat聊天模型知识融合框架,通过成对结构对齐微调与参数空间合并两阶段,将多架构多规模聊天LLM融合为统一模型,7B版本在指令遵循基准上表现优异,接近GPT-3.5-Turbo-1106水平。

摘要:

虽然从头训练大语言模型(LLM)确实能打造出具备独特能力与优势的模型,但这一过程成本高昂,还可能造成能力冗余。知识融合旨在通过轻量持续训练,将架构、能力各异的现有LLM整合为更强大的LLM,从而降低LLM开发的高昂成本。本研究提出了一套面向聊天类LLM知识融合的新框架,分为两个主要阶段,最终得到FuseChat。首先,我们对结构、规模各不相同的源聊天LLM开展成对知识融合,通过轻量微调生成多个结构、规模完全一致的目标LLM。在此过程中,我们引入了一种基于统计的token对齐方法,作为融合不同结构LLM的核心基础。其次,我们在参数空间内合并这些目标LLM,并提出了一种基于微调前后参数更新幅度确定合并系数的新方法。我们使用6款架构、规模各异的主流聊天LLM实现并验证了FuseChat,包括OpenChat-3.5-7B、Starling-LM-7B-alpha、NH2-SOLAR-10.7B、InternLM2-Chat-20B、Mixtral-8x7B-Instruct和Qwen-1.5-Chat-72B。在AlpacaEval 2.0和MT-Bench两个指令遵循基准上的实验结果表明,FuseChat-7B的表现优于不同规模的基线模型,甚至可与更大的Mixtral-8x7B-Instruct相媲美,在MT-Bench上的表现接近GPT-3.5-Turbo-1106。我们的代码、模型权重和数据已在https://github.com/fanqiwan/FuseAI 公开。

英文摘要:

While training large language models (LLMs) from scratch can indeed lead to models with distinct capabilities and strengths, it incurs substantial costs and may lead to redundancy in competencies. Knowledge fusion aims to integrate existing LLMs of diverse architectures and capabilities into a more potent LLM through lightweight continual training, thereby reducing the need for costly LLM development. In this work, we propose a new framework for the knowledge fusion of chat LLMs through two main stages, resulting in FuseChat. Firstly, we conduct pairwise knowledge fusion on source chat LLMs of varying structures and scales to create multiple target LLMs with identical structure and size via lightweight fine-tuning. During this process, a statistics-based token alignment approach is introduced as the cornerstone for fusing LLMs with different structures. Secondly, we merge these target LLMs within the parameter space, where we propose a novel method for determining the merging coefficients based on the magnitude of parameter updates before and after fine-tuning. We implement and validate FuseChat using six prominent chat LLMs with diverse architectures and scales, including OpenChat-3.5-7B, Starling-LM-7B-alpha, NH2-SOLAR-10.7B, InternLM2-Chat-20B, Mixtral-8x7B-Instruct, and Qwen-1.5-Chat-72B. Experimental results on two instruction-following benchmarks, AlpacaEval 2.0 and MT-Bench, demonstrate the superiority of FuseChat-7B over baselines of various sizes. Our model is even comparable to the larger Mixtral-8x7B-Instruct and approaches GPT-3.5-Turbo-1106 on MT-Bench. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/FuseAI}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑