福利、可改进性与方差：最优基准测试项聚合的主-代理方法
Welfare, Improvability, and Variance: A Principal-Agent Approach to Optimal Benchmark Item Aggregation
AI总结：
提出将基准测试建模为多任务主-代理博弈，通过福利、可改进性和方差三个维度评估项目，并应用于OLMES数据集识别帕累托劣势项目。
摘要：
AI基准测试存在记录完善的局限性，先前研究探讨了污染、饱和以及构造不明确等问题。聚合受到的关注要少得多：基准测试通常通过统一平均项目级分数来总结，隐含地将每个测试项目视为同等重要。我们将基准测试建模为多任务主-代理博弈，并表明基准测试的福利损失由三个项目级原始要素共同决定：与规范性福利优先级的一致性、边际可改进性和性能方差。我们将该理论转化为一个审计框架，沿这三个轴对项目进行排序，并使用WORKBank（福利）、EvoLM 4B套件（可改进性）和PolyPythias 410M面板（方差）将其应用于OLMES项目。该框架揭示了在OLMES中，在亲工人福利操作化下帕累托劣势的项目。所有代码可在 https://github.com/stair-lab/principal-agent-benchmarks 获取。
英文摘要：
AI benchmarks have well-documented limitations, with prior work examining contamination, saturation, and construct underspecification. Aggregation has received far less attention: benchmarks are typically summarized by uniformly averaging item-level scores, implicitly treating every test item as equally valuable. We model benchmarking as a multitask principal-agent game and show that the welfare loss from a benchmark is determined jointly by three item-level primitives: alignment with normative welfare priorities, marginal improvability, and performance variance. We translate the theory into an audit framework that ranks items along each of these three axes, and apply it to OLMES items using WORKBank for welfare, the EvoLM 4B suite for improvability, and the PolyPythias 410M panel for variance. The framework surfaces items that are Pareto-inferior within OLMES subject to a pro-worker welfare operationalization. All code is available at https://github.com/stair-lab/principal-agent-benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
