价格反转现象:当更便宜的推理模型成本更高时
The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More
AI总结:
本文首次系统研究推理模型标价与实际成本的偏差,发现32%的模型对比较中存在价格反转现象,并基于Shapley值建立成本归因框架,揭示思考令牌消耗和交互轮次的高度异质性是主要原因。
摘要:
开发者和消费者越来越根据列出的API价格选择推理模型(RMs)。然而,这些价格在多大程度上准确反映了实际推理成本?我们首次系统研究这一问题,评估了8个前沿RM在12个不同任务上的表现,涵盖竞赛数学、科学问答、代码生成和多领域智能体。我们发现了定价反转现象:在32%的模型对比较中,标价较低的模型实际上产生了更高的总成本,反转幅度高达28倍。例如,Gemini 3 Flash的标价比GPT-5.4便宜80%,但其在所有任务上的实际成本却高出38%。我们基于Shapley值构建了一个正式的成本归因框架,并利用它追溯了思考令牌消耗和交互轮次数量巨大异质性的主要贡献因素:对于同一查询,一个模型可能比另一个模型多使用900%的思考令牌,或多出10倍的环境交互轮次。我们进一步表明,每次查询的成本预测本质上是困难的:同一查询的重复运行产生的思考令牌变化高达9.7倍,为任何预测器建立了不可约的噪声底限。因此,我们提出成本分布预测作为一个开放挑战。我们的发现表明,列出的API定价是实际成本的不可靠代理,呼吁进行成本感知的模型选择和透明的每次请求成本监控。
英文摘要:
Developers and consumers increasingly choose reasoning models (RMs) based on their listed API prices. However, how accurately do these prices reflect actual inference costs? We conduct the first systematic study of this question, evaluating 8 frontier RMs across 12 diverse tasks covering competition math, science QA, code generation, and multi-domain agents. We uncover the pricing reversal phenomenon: in 32% of model-pair comparisons, the model with a lower listed price actually incurs a higher total cost, with reversal magnitude reaching up to 28x. For example, Gemini 3 Flash's listed price is 80% cheaper than GPT-5.4's, yet its actual cost across all tasks is 38% higher. We build a formal cost attribution framework based on Shapley value, and leverage it to trace the dominating contributors to vast heterogeneity in thinking token consumption and number of interaction turns: on the same query, one model may use 900% more thinking tokens than another, or 10x more turns of environment interactions. We further show that per-query cost prediction is fundamentally difficult: repeated runs of the same query yield thinking token variation up to 9.7x, establishing an irreducible noise floor for any predictor. Thus, we propose cost distribution prediction as an open challenge. Our findings demonstrate that listed API pricing is an unreliable proxy for actual cost, calling for cost-aware model selection and transparent per-request cost monitoring.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。