统计 scouting 找到辩论安全但不实用的案例:开放权重 LLM 推理协议的匹配天花板研究
Statistical Scouting Finds Debate-Safe but Not Debate-Useful Cases: A Matched-Ceiling Study of Open-Weight LLM Reasoning Protocols
AI总结:
研究探讨在生成 token 数限制下,不同推理协议的路由头寸及恢复潜力,发现辩论安全性和实用性不一致,需更复杂的信号来优化。
摘要:
当语言模型应直接回答、采样投票或参与多代理辩论时,近期研究表明投票常解释了辩论带来的大部分收益,而选择性辩论系统仅在不确定的例子上激活讨论。本文问:在生成 token 数限制(每例960个)下,每个例的路由头寸有多少,以及从廉价的前期信号中能恢复多少?我们评估了贪婪解码、三样本投票和双代理批评-修订辩论,在 MuSiQue 和 GSM8K 上使用 Llama 3.1 8B Instruct 和 Ministral 3 8B Instruct。在 MuSiQue 上,一个选择正确协议的 oracle 获得比最佳固定协议高 14.0 和 13.7 个百分点。最佳固定协议是模型和数据集依赖的:每个(模型,数据集)单元有不同的胜者。这种头寸很难从廉价的前期信号中恢复。投票-熵阈值是唯一能方向性击败最佳固定协议的控制器(+1.3 和 +1.7 个百分点),尽管个体配对 Bootstrap CIs 包含零。联合分析(元分析 +1.6 个百分点,p=0.125;贝叶斯 P(both>0)=0.59)在方向上一致但不显著。学习控制器(LR、GBT)不优于阈值。关键发现是结构性的:投票熵预测辩论安全的位置,而非需要辩论的位置。高熵显著减少辩论反噬,但 66% 的辩论有帮助的例子(31/47)发生在投票一致但错误时。在 Llama 上,单提示自我批评探针在 127/127 一致案例中翻转答案,与辩论有帮助标签无互信息;我们无法排除提示合规性 artifacts,但两种解释都使探针无法作为路由器。恢复剩余头寸需要避免格式合规性混淆的行为探针,在 8B 规模下。
英文摘要:
When should a language model answer directly, sample and vote, or engage in multi-agent debate? Recent work shows voting often explains much of the gain attributed to debate, while selective-debate systems activate deliberation only on uncertain examples. We ask: under a matched ceiling on generated tokens (960 per example), how much per-example routing headroom exists, and how much is recoverable from cheap pre-deliberation signals? We evaluate greedy decoding, three-sample voting, and a two-agent critique-revise debate on MuSiQue and GSM8K using Llama 3.1 8B Instruct and Ministral 3 8B Instruct. On MuSiQue, an oracle selecting the correct protocol per example gains +14.0 and +13.7 pp over the best fixed one. The best fixed protocol is model- and dataset-dependent: each (model, dataset) cell has a different winner. This headroom is hard to recover from cheap ex-ante signals. A vote-entropy threshold is the only controller that directionally beats the best fixed protocol on both models (+1.3 and +1.7 pp), though individual paired-bootstrap CIs include zero. A joint analysis (meta-analysis +1.6 pp, p=0.125; Bayesian P(both>0)=0.59) is directionally consistent but not significant. Learned controllers (LR, GBT) do not outperform the threshold. The key finding is structural: vote entropy predicts where debate is safe, not where debate is needed. High entropy sharply reduces debate backfire, but 66% of debate-helpful examples (31/47) occur when voting is unanimous but wrong. A single-prompt self-critique probe on Llama flips the answer in 127/127 unanimous cases, yielding zero mutual information with the debate-helpful label; we cannot rule out a prompt-compliance artifact, but either interpretation disqualifies the probe as a router. Recovering the remaining headroom requires behavioral probes that avoid format-compliance confounds at the 8B scale.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。