AI 中文总结
本研究提出保留最佳N个方法，实现无偏评估选择策略的预期奖励，并分析其代价，证明在特定条件下无偏性及风险速率。
AI 中文摘要
重用选择最佳N个（Best-of-$N$）获胜者的分数可能会高估其预期奖励。我们研究从一个固定的矩阵中评估，该矩阵包含每个候选者的$K$个独立分数，而策略使用$J$个新分数进行选择。仅基于此矩阵的单一估计器，对于每个独立的、稳定的候选特定分数法则集合，在且仅当$J<K$且对于每个池大小$M\ge N\ge2$时，对预期评判奖励是精确无偏的。在$J=K-1$时，随着$K$增长，选择器加深。对于具有共同方差和固定$M\ge N\ge2$的独立高斯分数，此情况下的无偏极小化风险阶为$\sigma^2/\sqrt K$，由保留（Holdout）方法达到；允许偏差可将速率改善至$\sigma^2/K$。对于两个候选者，我们在已知方差下推导出最小方差无偏估计器，以及尖锐的渐近无偏极小化常数$1/(\pi\sqrt2)$，保留方法在未知方差下也能达到该常数。子集和并列上的循环平均可以在$O(MK\log M)$次操作中计算。在固定选择器深度下，有界分数的循环评估具有$O(K^{-1})$风险，且均匀于池大小。不可能性结果涉及固定矩阵：一个额外的新获胜者分数允许对所有$K$策略进行无偏评估。
英文摘要
Reusing the scores that select a Best-of-$N$ winner can overstate its expected reward. We study evaluation from a fixed matrix of $K$ independent scores per candidate for a policy that selects using $J$ fresh scores. A single estimator based only on this matrix is exactly unbiased for expected judge reward under every independent, stable collection of candidate-specific score laws if and only if $J<K$, for every pool size $M\ge N\ge2$. At $J=K-1$, the selector deepens as $K$ grows. For independent Gaussian scores with common variance and fixed $M\ge N\ge2$, the unbiased minimax risk in this regime is of order $σ^2/\sqrt K$, attained by Holdout; allowing bias improves the rate to $σ^2/K$. For two candidates, we derive the minimum-variance unbiased estimator at known variance and the sharp asymptotic unbiased minimax constant $1/(π\sqrt2)$, which Holdout attains without knowing the variance. The cyclic average over subsets and ties can be computed in $O(MK\log M)$ operations. At fixed selector depth, cyclic evaluation of bounded scores has $O(K^{-1})$ risk uniformly in pool size. The impossibility result concerns the fixed matrix: one additional fresh winner score permits unbiased evaluation of the all-$K$ policy.
Comments25 pages, 2 figures, 3 tables
