arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.15518cs.LG
收藏

Eagle:面向多大语言模型推理的高效免训练路由器

Eagle: Efficient Training-Free Router for Multi-LLM Inference

University of Michigan(密歇根大学)

Zesen Zhao, Shuowei Jin, Z. Morley Mao

AI总结:

针对现有LLM路由器在高流量在线环境中可扩展性与实时适配性不足的问题,提出结合全局和局部ELO排名模块的免训练路由方法Eagle,在多个数据集上AUC最高提升23.52%,初始化及增量更新效率大幅提升。

摘要:

大语言模型(LLM)的数量激增且能力与成本各异,这使得人工智能系统中需要高效的模型选择机制。LLM路由器通过根据任务需求和预算约束为给定查询动态选择最合适的模型来满足这一需求。然而,现有路由器在可扩展性和实时适配性方面面临挑战,尤其是在高流量在线环境中。我们提出了Eagle,这是一种新颖的LLM路由方法,结合了全局和局部ELO排名模块来克服这些限制。通过评估大语言模型的通用能力和专业能力,Eagle提供了一种可扩展的免训练解决方案,既提升了模型选择质量,又降低了计算开销。我们在多个数据集上的实验表明,Eagle的性能始终优于基线方法,曲线下面积(AUC)得分最高提升了23.52%。此外,Eagle展现出了卓越的效率:初始化所需时间仅为基线方法的1/20,在在线场景中的增量更新速度快100到200倍,非常适合动态、高流量的在线服务环境。

英文摘要:

The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need by dynamically choosing the most suitable model for a given query based on task requirements and budget constraints. However, existing routers face challenges in scalability and real-time adaptation, particularly in high-volume online environments. We present Eagle, a novel LLM routing approach that combines global and local ELO ranking modules to overcome these limitations. By evaluating both general and specialized LLM abilities, Eagle provides a scalable, training-free solution that enhances model selection quality while reducing computational overhead. Our experiments across multiple datasets show Eagle consistently outperforms baseline methods, with improvements of up to 23.52 percent in Area Under Curve (AUC) scores. Moreover, Eagle demonstrates remarkable efficiency, requiring only 1/20 of baseline methods' time for initialization and 100 to 200 times faster incremental updates in online scenarios, making it well-suited for dynamic, high-volume online serving environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑