arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10729cs.DC
收藏

BOute: 通过多目标贝叶斯优化实现高效的LLM服务

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization

University of Cambridge(剑桥大学);Shanghai Jiao Tong University(上海交通大学)

Youhe Jiang, Fangcheng Fu, Eiko Yoneki

AI总结:

BOute通过多目标贝叶斯优化实现高效LLM服务,优化路由和部署策略以提升成本效率。

摘要:

大语言模型(LLM)的快速部署增长使得高效服务系统变得至关重要。最近的努力通过两个主要视角来提高系统成本效率:(i)从算法视角出发,利用异构模型能力将简单查询路由到低成本模型,复杂查询路由到高成本模型(即异构查询路由);(ii)从系统视角出发,利用异构GPU资源作为高成本同构GPU的替代方案(即异构模型部署)。然而,为了实现高效的LLM服务,算法-系统协同设计需要复杂的管理:(i)在延迟和质量要求下确定最优查询路由策略;(ii)在异构GPU上配置模型部署,采用适当的资源分配和并行策略;(iii)协同优化路由和部署决策以最大化整体系统性能。为了解决这些挑战,我们提出了BOute,一个具有质量意识的调度系统,利用异构模型和GPU能力实现高效的LLM服务。BOute采用多目标贝叶斯优化(MOBO)框架,共同优化路由策略和模型部署,从而在保证响应质量的同时最大化服务系统的成本效率。评估结果表明,BOute在相同成本预算和质量要求下,比最先进的LLM服务系统平均提高了157%和59%,或在保持相同性能目标的情况下,将服务成本降低了15%-61%(平均38%),验证了其在实现高效LLM服务方面的有效性。

英文摘要:

The rapid growth of large language model (LLM) deployments has made cost-efficient serving systems essential. Recent efforts to enhance system cost-efficiency adopt two main perspectives: (i) An algorithmic perspective that exploits heterogeneous model capabilities to route simpler queries to lower-cost models and complex queries to higher-cost models (i.e., heterogeneous query routing); and (ii) a systems perspective that utilizes heterogeneous GPU resources as cost-effective alternatives to homogeneous high-end GPUs (i.e., heterogeneous model deployment). However, algorithm-system co-design for cost-efficient LLM serving necessitates sophisticated management: (i) Determining optimal query routing strategies under latency and quality requirements, (ii) configuring model deployment across heterogeneous GPUs with appropriate resource allocation and parallelism strategies, and (iii) co-optimizing routing and deployment decisions to maximize overall system performance. To address these challenges, we present BOute, a quality-aware scheduling system that jointly exploits heterogeneous model and GPU capabilities for cost-efficient LLM serving. BOute employs a multi-objective Bayesian optimization (MOBO) framework to co-optimize the routing strategy and model deployment, thereby maximizing the cost-efficiency of the serving system while guaranteeing response quality. Evaluation results demonstrate that BOute outperforms state-of-the-art LLM serving systems by up to 157% and 59% on average under identical cost budgets and quality requirements, or reducing serving costs by 15%-61% (38% on average) while maintaining the same performance targets, validating its effectiveness in achieving cost-efficient LLM serving.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑