关于模型 MoErging 的综述：在专门化专家之间进行回收与路由以实现协作学习
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
AI总结：
本文综述了模型 MoErging 方法，提出了一种新的分类法来编目关键设计选择，并探讨了相关软件工具与应用，为该新兴领域提供了统一概述与研究基础。
摘要：
高性能预训练模型的可用性导致了针对特定领域或任务专门化的微调专家模型的激增。Model MoErging 方法旨在回收这些专家模型，以创建一个具有改进性能或泛化能力的聚合系统。MoErging 方法的一个关键组件是创建一个路由器，用于决定针对特定输入或应用使用哪个或哪些专家模型。MoErging 的前景、有效性和庞大的设计空间在过去几年中推动了许多新方法的开发。这种快速的开发节奏使得比较不同的 MoErging 方法变得具有挑战性，它们很少被相互比较，且往往在不同的实验设置中进行验证。为了弥补这些差距，我们提出了一份关于 MoErging 方法的全面综述，其中包含一个新颖的分类法，用于编目关键设计选择并阐明每种方法的适用应用。除了调研 MoErging 研究外，我们还盘点了利用 MoErging 的软件工具和应用程序。我们还额外讨论了相关的研究领域，例如模型合并、多任务学习和混合专家模型。总体而言，我们的综述提供了对现有 MoErging 方法的统一概述，并为这一新兴领域的未来工作奠定了坚实的基础。
英文摘要：
The availability of performant pre-trained models has led to a proliferation of fine-tuned expert models that are specialized to a particular domain or task. Model MoErging methods aim to recycle expert models to create an aggregate system with improved performance or generalization. A key component of MoErging methods is the creation of a router that decides which expert model(s) to use for a particular input or application. The promise, effectiveness, and large design space of MoErging has spurred the development of many new methods over the past few years. This rapid pace of development has made it challenging to compare different MoErging methods, which are rarely compared to one another and are often validated in different experimental setups. To remedy such gaps, we present a comprehensive survey of MoErging methods that includes a novel taxonomy for cataloging key design choices and clarifying suitable applications for each method. Apart from surveying MoErging research, we inventory software tools and applications that make use of MoErging. We additionally discuss related fields of study such as model merging, multitask learning, and mixture-of-experts models. Taken as a whole, our survey provides a unified overview of existing MoErging methods and creates a solid foundation for future work in this burgeoning field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
