arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.07490cs.CL
收藏

MoDEM:领域专家模型混合体

MoDEM: Mixture of Domain Expert Models

The University of Melbourne(墨尔本大学)

Toby Simonds, Kemal Kurniawan, Jey Han Lau

AI总结:

本研究提出MoDEM方法,通过基于BERT的路由器将提示路由至领域专用专家模型,相比同规模通用大语言模型性能更优、成本更低,为LLM发展提供了新范式。

摘要:

我们提出了一种提升大语言模型(LLM)性能与效率的新方法,将领域提示路由与领域专用模型相结合。我们构建了一个系统,使用基于BERT的路由器将输入提示路由至最合适的领域专家模型,这些专家模型是针对健康、数学、科学等领域专门微调的。研究表明,该方法的性能显著优于同等规模的通用模型,在各类基准测试中实现了更优的性能成本比。本研究的结论预示着大语言模型开发与部署可能出现范式转变:未来AI的发展方向或许并非一味打造更大的通用模型,而是构建由小型高专用性模型与复杂路由系统组成的生态,这一方法可提升资源利用效率、降低计算成本并实现更优的整体性能。

英文摘要:

We propose a novel approach to enhancing the performance and efficiency of large language models (LLMs) by combining domain prompt routing with domain-specialized models. We introduce a system that utilizes a BERT-based router to direct incoming prompts to the most appropriate domain expert model. These expert models are specifically tuned for domains such as health, mathematics and science. Our research demonstrates that this approach can significantly outperform general-purpose models of comparable size, leading to a superior performance-to-cost ratio across various benchmarks. The implications of this study suggest a potential paradigm shift in LLM development and deployment. Rather than focusing solely on creating increasingly large, general-purpose models, the future of AI may lie in developing ecosystems of smaller, highly specialized models coupled with sophisticated routing systems. This approach could lead to more efficient resource utilization, reduced computational costs, and superior overall performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑