推理并非免费：面向LLM-as-a-Judge的鲁棒自适应成本高效路由
Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge
AI总结：
本文通过对比推理与非推理法官，发现推理在结构验证任务中提升准确性，但成本高。提出RACER方法，通过分布鲁棒优化动态选择推理或非推理法官，实现成本效率的最优平衡。
摘要：
能够推理的大型语言模型（LLMs）最近被用作自动化法官，但其在LLM-as-a-Judge场景中的利弊仍不明确。通过对比推理与非推理法官，我们发现显式推理在需要结构验证的任务（如数学和编程）中显著提升判断准确性，但在简单评估中收益有限且计算成本高。这些发现表明推理应选择性使用而非普遍应用。我们提出鲁棒自适应成本高效路由（RACER），通过将路由建模为受限分布鲁棒优化问题，在固定预算下动态选择推理或非推理法官。RACER通过KL散度不确定性集显式考虑分布偏移，采用高效的对偶算法，并具有理论保证，包括最优策略的唯一性和线性收敛性。广泛实验表明，RACER在分布偏移下实现了更优的准确度-成本权衡。
英文摘要：
Reasoning-capable large language models (LLMs) have recently been adopted as automated judges, but their benefits and costs in LLM-as-a-Judge settings remain unclear. Through controlled comparisons between reasoning and non-reasoning judges, we show that explicit reasoning substantially improves judgment accuracy on tasks requiring structured verification (e.g., math and coding), while offering limited or even negative gains on simpler evaluations and incurring significantly higher computational cost. These findings motivate that reasoning should be used selectively rather than universally, with awareness of possible distribution shift. We propose a Robust Adaptive Cost-Efficient Routing (RACER), which dynamically selects between reasoning and non-reasoning judges under a fixed budget by formulating routing as a constrained distributionally robust optimization problem. RACER explicitly accounts for distribution shift via a KL-divergence uncertainty set, admits an efficient primal--dual algorithm, and enjoys theoretical guarantees including uniqueness of the optimal policy and linear convergence. Extensive experiments show that RACER achieves superior accuracy--cost trade-offs under distribution shift.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
