arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09902cs.GTcs.AIcs.LG
收藏

路由、级联与用户选择用于大语言模型

Routing, Cascades, and User Choice for LLMs

University of Ottawa(渥太华大学);NVIDIA(NVIDIA公司)

Rafid Mahmood

AI总结:

本文研究了大语言模型路由策略与用户行为之间的博弈,揭示了提供者与用户在效用和成本排序上的不一致,提出了基于Stackelberg博弈的路由优化方法,并得出了单用户单提供者互动的阈值规则。

摘要:

为了缓解性能与成本之间的权衡,大语言模型(LLM)提供者根据任务难度和延迟将用户任务路由到不同的模型。我们研究了LLM路由相对于用户行为的影响。我们提出了一种博弈,其中LLM提供者有两个模型(标准和推理)与一个用户之间进行博弈,用户可以重新提示或放弃任务如果路由的模型无法解决它们。用户的目标是最大化其效用减去使用模型的延迟,而提供者则最小化服务用户成本。我们通过完全刻画用户最佳响应并简化提供者问题来解决这个Stackelberg博弈。我们观察到在几乎所有情况下,最优路由策略涉及一个静态策略,没有级联,该策略取决于模型对用户的预期效用。此外,我们揭示了当用户和提供者对模型的效用和成本排序不同时,提供者最优与用户偏好的路由之间的不一致差距。最后,我们展示了极端不一致的条件,其中提供者被激励通过降低模型延迟来最小化成本,从而抑制用户效用。结果为单提供者、单用户互动提供了简单的阈值规则,并澄清了路由、级联和 throttling 帮助或损害的条件。

英文摘要:

To mitigate the trade-offs between performance and costs, LLM providers route user tasks to different models based on task difficulty and latency. We study the effect of LLM routing with respect to user behavior. We propose a game between an LLM provider with two models (standard and reasoning) and a user who can re-prompt or abandon tasks if the routed model cannot solve them. The user's goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user. We solve this Stackelberg game by fully characterizing the user best response and simplifying the provider problem. We observe that in nearly all cases, the optimal routing policy involves a static policy with no cascading that depends on the expected utility of the models to the user. Furthermore, we reveal a misalignment gap between the provider-optimal and user-preferred routes when the user's and provider's rankings of the models with respect to utility and cost differ. Finally, we demonstrate conditions for extreme misalignment where providers are incentivized to throttle the latency of the models to minimize their costs, consequently depressing user utility. The results yield simple threshold rules for single-provider, single-user interactions and clarify when routing, cascading, and throttling help or harm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑