发表机构
Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文设计了一个基于反向第二价格拍卖的采购平台，通过提供商竞争动态定价，学习质量并路由查询，实验显示定价利润率可从10%到71%，显著优于固定价格市场。
AI 中文摘要
基准测试和路由平台日益充当连接大型语言模型提供商与终端用户的中间人。然而，这些平台上的提供商通常采用固定的每令牌价格，这阻碍了用户为其任务获得最具竞争力的价格。在这项工作中，我们设计了一个采购平台，其中每项任务的令牌价格由提供商竞争驱动，从而使用户能够以有保证的质量水平获得有竞争力的价格。为此，该平台通过反向第二价格拍卖顺序路由查询，该拍卖激励模型提供商如实出价其最佳估计，即服务用户查询的平均成本。在路由查询时，平台学习每个提供商提供的质量，并逐步将查询路由到在满足所需质量阈值的提供商中成本最具竞争力的提供商。为了验证我们的设计，我们使用来自Llama和Qwen家族的多个大型语言模型在流行的数学推理和问答基准上进行了实验。结果表明，我们平台上成本最具竞争力的提供商的定价利润率根据任务和质量阈值显著变化——从10%到71%。这表明当前固定价格市场存在严重的低效率，并证明我们的平台可能使用户在竞争市场条件允许时获得最大节省。
英文摘要
Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on these platforms typically use a fixed price per token, preventing users from achieving the most competitive price for their tasks. In this work, we design a procurement platform where token prices for each task are driven by provider competition, enabling users to secure competitive pricing for guaranteed quality levels. To this end, the platform sequentially routes queries via a reverse second-price auction that incentivizes model providers to truthfully bid their best estimate of the average cost to serve a user's query. As it routes queries, the platform learns the quality offered by each provider and progressively routes queries to the most cost-competitive provider among those meeting a desired quality threshold. To validate our design, we conduct experiments with multiple LLMs from the Llama and Qwen families on popular mathematical reasoning and question-answering benchmarks. The results show that the pricing margin of the most cost-competitive provider on our platform varies significantly---from $10\%$ to $71\%$---depending on the task and quality threshold. This suggests a substantial inefficiency in the current fixed-price market, and it demonstrates that our platform may enable users to capture maximum savings whenever competitive market conditions permit.
