arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04998cs.CLcs.AI
收藏

ProFuser:大语言模型的渐进式融合

ProFuser: Progressive Fusion of Large Language Models

Tianyuan Shi, Fanqi Wan, Canbin Huang, Xiaojun Quan, Chenliang Li, Ming Yan, Ji Zhang, Minhua Huang, Wu Kai

AI总结:

针对现有大语言模型融合方法仅靠训练阶段交叉熵评估模型优势的局限,本文提出ProFuser渐进式融合框架,结合训练与推理双模式评估优势,融合三款7B模型后在知识、推理、安全性能上超过基线。

摘要:

融合不同大语言模型的能力与优势是构建更强大、更通用模型的可行路径,但训练过程中如何恰当选择优势模型是一项核心挑战。现有融合方法主要采用教师强制设置下基于真值的交叉熵训练模式衡量模型优势,对模型优势的评估维度十分有限。本文提出一种新方法,通过同时纳入训练模式与推理模式优化融合过程:该方法不仅在训练阶段通过交叉熵评估模型优势,还结合推理输出进行评估,实现更全面的优势衡量。为有效结合两种模式,本文提出ProFuser,实现从推理模式到训练模式的渐进式过渡。为验证ProFuser的有效性,研究融合了Vicuna-7B-v1.5、Llama-2-7B-Chat、MPT-7B-8K-Chat三个模型,实验证明其在知识、推理、安全维度的性能均优于基线方法。

英文摘要:

While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model's advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser's effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑