arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.07093cs.CLcs.AIcs.LG

翻译税并非标量:对中国多语言基准测试中英语源提示继承的反事实审计

The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks

OpenAI;NeurIPS 2025 workshop(NeurIPS 2025 工作坊)

Zezheng Lin, Fengming Liu, Handi Li

AI总结:

本文通过反事实审计揭示翻译税并非单一标量,指出翻译基准测试中存在估计器和项目依赖的有效性风险,并提供相关证据和检查清单。

摘要:

翻译税常被视为标量:翻译基准测试被假设通过保留英语源提示来提高分数。我们在英语到中文的设定中审计这一主张。三种代理估计器得出不同结论:回译差距较小且解析器脆弱;提示-分数校准无法预测项目级增益;六模型原生对照显示模型族而非统一基准效应。我们添加了一个相同项目LLM-自然化压力测试,固定答案、选项和内容,仅重写中文表层形式。在修正提示构造错误后,此对比不再支持模型族交互,但保留了残余剂量-反应:高残余项目受益而低残余项目无益。结果并非单一翻译税,而是一组估计器和项目依赖的有效性风险。我们发布每单元证据、自然化协议、人工QC和翻译多语言基准论文的报告清单。

英文摘要:

The Translation Tax is often treated as a scalar: translated benchmarks are assumed to inflate scores by preserving English-source cues. We audit this claim in an English-to-Chinese setting. Three proxy estimators disagree: back-translation gaps are small and parser-fragile; cue-score calibration does not predict item-level gains; and a six-model native-control comparison shows model-family rather than uniform benchmark effects. We add a same-item LLM-naturalization stress test that holds answer, options, and content fixed while rewriting Chinese surface form. After correcting a prompt-construction bug, this contrast no longer supports a model-family interaction, but it preserves a residue dose-response: high-residue items benefit while low-residue items do not. The result is not a single Translation Tax, but a set of estimator- and item-dependent validity risks. We release per-cell evidence, the naturalization protocol, human QC, and a reporting checklist for translated multilingual benchmark papers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑