发表机构
Centific（Centific）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过交换参考标准，测量了多语言基准分数对参考选择的敏感性，发现分数可移动高达7.55个F1点，并揭示了未记录的聚合默认设置导致参考成为输出副本，主张报告参考敏感性带。
AI 中文摘要
基准分数将系统输出与参考进行比较，方法论上的关注几乎完全集中在第一个术语上。我们衡量第二个术语。一个六语言个人可识别信息基准的保留注释记录包含两个独立的注释者标注，聚合后的版本作为金标发布，以及一个由独立专家对样本进行重新注释的审阅金标。利用这些数据，我们固定被评分的输出并交换参考。分数对一个输出移动了4.95个F1点，对另一个移动了2.00个F1点（95%置信区间分别为[3.23, 6.27]和[0.26, 3.37]），在最差的语言中移动了7.55个点。两个输出之间的比较也发生了移动：参考与系统之间的配对交互为+2.95点（置信区间[+1.97, +3.87]），通过了多重检验校正，并直接改变了一种语言的差距。原因是未记录的聚合默认设置，当裁决未触发时通常保留一个注释者，使得发布的参考成为被评分输出的部分副本。我们报告了由此产生的参考敏感性带，展示了如何从任何保留的注释记录中计算它，并主张该量应紧邻分数呈现。
英文摘要
A benchmark score compares a system output against a reference, and methodological attention falls almost entirely on the first term. We measure the second. The retained annotation record of a six-language benchmark for personally identifiable information contains two independent annotator labellings, the aggregate shipped as gold, and a reviewer gold from independent expert re-annotation of a sample. Using it, we hold the scored output fixed and exchange the reference. The score moves by 4.95 F1 points for one output and 2.00 for the other (95% CIs [3.23, 6.27] and [0.26, 3.37]), and by 7.55 in the worst language. The comparison between two outputs moves as well: the paired interaction between reference and system is +2.95 points (CI [+1.97, +3.87]), survives correction for multiple testing, and changes one language's margin outright. The cause is an undocumented aggregation default that usually kept one annotator when adjudication did not fire, making the shipped reference a partial copy of an output being scored. We report the resulting reference-sensitivity band, show how to compute one from any retained annotation record, and argue that the quantity belongs beside the score.
Comments13 pages (8 main), 1 figure, 8 tables. Accepted as a poster at TAE (Trust-AI-Eval): Can We Trust AI Evaluation?, a NeurIPS 2026 workshop (non-archival). The workshop name comes from the acceptance email in reviews.md
