测量焦耳，学习路由：面向节能LLM服务的路由学习
Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving
浏览论文内容
中文总结 AI 辅助
本文提出基于语言模型的路由器，利用离线测量数据训练，实现按查询上下文选择模型，在七个基准任务上改善准确性-能源权衡，并发现相变现象。
中文摘要 AI 辅助
随着模型规模的扩大和推理轨迹的延长，大型语言模型（LLM）和智能体AI系统正在产生快速增长推理能源需求。虽然在实际中，许多查询并不需要最大可用模型的能力，但常规性地将此类查询导向高能力模型可能会引入不必要的、可观的计算和能源消耗。在本文中，我们研究了跨异构LLM池的自适应路由是否能在不显著损害任务性能的情况下减轻这种能源负担。我们设计了一个基于语言模型的路由器，它读取每个查询并从固定的候选池中选择一个答案模型。候选模型首先通过离线锦标赛进行性能分析，记录每个查询的正确性、延迟、功率和GPU能源。利用这些测量结果，路由器通过监督微调，随后结合定制范式的组相对策略优化（GRPO）进行训练。结果表明，学习路由可以根据查询上下文有选择地分配昂贵的模型容量，并改善多LLM服务中的准确性-能源权衡。在七个基准任务中，我们还观察到路由器之间出现急剧的准确性-能源相变，为在保持LLM性能的同时提高能源效率提供了实用见解。
英文摘要
Large language models (LLMs) and agentic AI systems are creating rapidly growing inference energy demands as model sizes grow and reasoning trajectories extend. While in practice, many queries do not require the capabilities of the largest available model, and routinely directing such queries to a high-capability model can introduce unnecessary, considerable computation and energy consumption. In this paper, we investigate whether adaptive routing across a heterogeneous pool of LLMs can reduce this energy burden without substantially compromising task performance. We design a language-model-based router that reads in each query and selects an answer model from a fixed candidate pool. The candidate models are first profiled through an offline tournament that records their correctness, latency, power, and GPU energy for each query. Using these measurements, the router is trained through supervised fine-tuning followed by group relative policy optimization (GRPO) with the tailored paradigms. Results demonstrate that learned routing can selectively allocate expensive model capacity based on query context and improve the accuracy-energy tradeoff in multi-LLM serving. Across seven benchmark tasks, we also observe a sharp accuracy-energy phase transition among routers, providing practical insights into improving energy efficiency while maintaining LLM performance.
发表机构
- University of Alberta（阿尔伯塔大学）
- University of California San Diego（加州大学圣地亚哥分校）
- Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
- New York University（纽约大学）
机构由 AI 辅助整理，请以论文原文为准。
