StepCOPS：语言模型策略选择的闭式检验下尾证书

中文摘要 AI 辅助

后训练流程必须从众多检查点、提示词和解码规则中选择一个语言模型策略。平均评估分数可能掩盖罕见故障，而同时逐候选置信界可能过于保守。我们提出StepCOPS，该方法利用独立提案分割为每个候选指定一个下尾下限，在全新认证分割上执行精确二项检验，并采用Holm逐步下降程序来认证一组下限。在至少$1-\delta$的概率下，每个被认证的下限（包括用于策略选择的最大下限）均低于其候选总体的下$\alpha$分位数。该保证假设评估单元独立同分布，同时允许候选间存在任意的单元内依赖。在24个预声明配置和11个基准上，StepCOPS在500次配对试验中实现了96.4%的选定策略覆盖率，将认证下限比提案-邦费罗尼方法和精确COPS提高了1.5个百分点，比大型参考评审团预言机低0.6个百分点，并在2.4%的试验中弃权（不执行）。影子评审、基准原生、工件和留一评审审计刻画了代理边界：该保证适用于固定评审团分数，而非直接适用于人类安全。

英文摘要

Post-training pipelines must select one language-model policy from many checkpoints, prompts, and decoding rules. Mean evaluator scores can conceal rare failures, whereas simultaneous candidate-wise confidence bounds can be unnecessarily conservative. We introduce StepCOPS, which uses an independent proposal split to nominate one lower-tail floor per candidate, exact binomial tests on a fresh certification split, and Holm's step-down procedure to certify a set of floors. With probability at least $1-δ$, every certified floor, including the largest floor used for policy selection, is below its candidate's population lower $α$-quantile. This guarantee assumes i.i.d. evaluation units while allowing arbitrary within-unit dependence across candidates. Across 24 predeclared configurations and 11 benchmarks, StepCOPS obtains 96.4% selected-policy coverage over 500 paired trials, raises the certified floor by 1.5 points over both proposal-Bonferroni and exact COPS, remains 0.6 points below the large-reference jury oracle, and abstains in 2.4% of trials. Shadow-judge, benchmark-native, artifact, and leave-one-judge-out audits characterize the proxy boundary: the guarantee applies to the fixed jury score, not directly to human safety.

发表机构

• Iowa State University（爱荷华州立大学）

机构由 AI 辅助整理，请以论文原文为准。