arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18665cs.LGcs.AIcs.CL
收藏

RouteLLM:利用偏好数据学习路由大语言模型

RouteLLM: Learning to Route LLMs with Preference Data

UC Berkeley(加州大学伯克利分校);Anyscale;Canva

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, Ion Stoica

AI总结:

本文提出RouteLLM,利用人类偏好数据训练高效路由模型,在推理时动态选择强弱LLM,以在成本与质量间取得平衡,显著降低成本且不损响应质量,并展现出强迁移学习能力。

摘要:

大语言模型(LLMs)在广泛的任务中展现出令人印象深刻的能力,然而选择使用哪个模型往往涉及性能与成本之间的权衡。更强大的模型虽然有效,但成本更高,而能力较弱的模型则更具成本效益。为了解决这一困境,我们提出了几种高效的路由模型,在推理过程中动态地在较强和较弱的大语言模型之间进行选择,旨在优化成本与响应质量之间的平衡。我们开发了一个利用人类偏好数据和数据增强技术来提升性能的路由器训练框架。我们在广泛认可的基准上的评估表明,我们的方法显著降低了成本——在某些情况下降低超过2倍——且不损害响应质量。有趣的是,我们的路由模型还展现出显著的迁移学习能力,即使在测试时更换强模型和弱模型,也能保持其性能。这凸显了这些路由器在部署大语言模型时提供成本效益高且高性能解决方案的潜力。

英文摘要:

Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More powerful models, though effective, come with higher expenses, while less capable models are more cost-effective. To address this dilemma, we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality. We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance. Our evaluation on widely-recognized benchmarks shows that our approach significantly reduces costs-by over 2 times in certain cases-without compromising the quality of responses. Interestingly, our router models also demonstrate significant transfer learning capabilities, maintaining their performance even when the strong and weak models are changed at test time. This highlights the potential of these routers to provide a cost-effective yet high-performance solution for deploying LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑