动态LLM路由器常常被误导
Dynamic LLM Routers are Often Misguided
浏览论文内容
中文总结 AI 辅助
本文分析六种商业动态LLM路由器，发现均不如随机选择两个精选模型，归因于四种误导模式，并提出不奖励这些模式的评估方法与双模型路由器，但收益有限。
中文摘要 AI 辅助
动态LLM路由器承诺通过将每个查询发送给能够正确回答该查询的最便宜模型来降低推理成本。我们在一个涵盖八个任务类别的多样化基准上，对14种设置下的六种商业路由器进行了分析，发现它们中没有一个优于在匹配成本下随机选择两个精心挑选的模型之一的路由器。有些路由器的性能落后超过10个百分点。我们将这一差距归因于路由器中普遍存在的四种模式：难度盲区、长度反转、语义匹配和名单次优性。我们表明，前三种模式正是标准目标所奖励的：在实现成本上的成本-准确性帕累托效率倾向于将中等难度的查询升级到最难查询之上，将较短的查询升级到较长查询之上，并根据查询的来源而非难度进行路由。我们还论证了支持大型名单的两个假设——模型粒度和模型专业化——在经验上并不成立。我们提出了一种替代评估方法，该方法不奖励这些模式，并作为概念验证，我们设计了一个简单的双模型路由器，它避免了所有四种模式。然而，与随机路由相比，其收益有限，因为精心挑选的名单几乎没有留下可路由的空间。
英文摘要
Dynamic LLM routers promise to cut inference costs by sending each query to the cheapest model that can answer it correctly. We analyze six commercial routers across 14 settings on a diverse benchmark spanning eight task categories, finding that none of them outperforms a router that randomly selects between two well-chosen models at matched cost. Some underperform by more than 10 percentage points. We trace this gap to four patterns prevalent across routers: difficulty blindness, length reversal, semantic matching, and roster suboptimality. We show that the first three are what the standard objective rewards: cost-accuracy Pareto efficiency on realized costs favors escalating moderately hard queries over the hardest ones, shorter queries over longer ones, and routing by a query's source over its difficulty. We also argue that the two assumptions that would justify large rosters, model granularity and model specialization, do not hold empirically. We propose an alternative evaluation methodology that does not reward these patterns, and as a proof of concept, we design a simple two-model router that avoids all four. Nevertheless, its gain over random routing is limited, because a well-chosen roster leaves little to route.
发表机构
- Fastino Labs
机构由 AI 辅助整理，请以论文原文为准。
