arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.12031cs.LGcs.AI
收藏

RouterBench:面向多LLM路由系统的基准测试

RouterBench: A Benchmark for Multi-LLM Routing System

Martian(火星公司);UC Berkeley(加州大学伯克利分校);UC San Diego(加州大学圣迭戈分校)

Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, Shriyash Kaustubh Upadhyay

AI总结:

本文提出RouterBench基准及超405k推理结果数据集,并建立LLM路由理论框架,系统评估多LLM路由系统,推动经济可行的LLM部署。

摘要:

随着大语言模型(LLM)的应用范围不断扩大,对高效服务解决方案的需求变得愈发关键。尽管LLM具有广泛的适用性,但没有任何单一模型能够在所有任务和应用中都达到最优效果,尤其是在需要同时平衡性能与成本的情况下。这一局限推动了LLM路由系统的发展,这类系统通过整合不同模型的优势来克服单个LLM的不足。然而,目前缺乏用于评估LLM路由器性能的标准化基准,阻碍了该领域的进步。为弥补这一空白,我们提出了RouterBench,这是一个新颖的评估框架,旨在系统性地评估LLM路由系统的有效性,同时提供了一个包含超过405k条来自代表性LLM的推理结果的综合数据集,以支持路由策略的开发。我们进一步提出了一个用于LLM路由的理论框架,并通过RouterBench对各种路由方法进行了对比分析,揭示了它们在我们评估框架内的潜力与局限。这项工作不仅规范并推动了LLM路由系统的发展,还为其评估设立了标准,为更易获取且经济可行的LLM部署铺平了道路。代码和数据可在https://github.com/withmartian/routerbench获取。

英文摘要:

As the range of applications for Large Language Models (LLMs) continues to grow, the demand for effective serving solutions becomes increasingly critical. Despite the versatility of LLMs, no single model can optimally address all tasks and applications, particularly when balancing performance with cost. This limitation has led to the development of LLM routing systems, which combine the strengths of various models to overcome the constraints of individual LLMs. Yet, the absence of a standardized benchmark for evaluating the performance of LLM routers hinders progress in this area. To bridge this gap, we present RouterBench, a novel evaluation framework designed to systematically assess the efficacy of LLM routing systems, along with a comprehensive dataset comprising over 405k inference outcomes from representative LLMs to support the development of routing strategies. We further propose a theoretical framework for LLM routing, and deliver a comparative analysis of various routing approaches through RouterBench, highlighting their potentials and limitations within our evaluation framework. This work not only formalizes and advances the development of LLM routing systems but also sets a standard for their assessment, paving the way for more accessible and economically viable LLM deployments. The code and data are available at https://github.com/withmartian/routerbench.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑