发表机构
Case Western Reserve University（凯斯西储大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出含四组件的跨语言公平性评估框架，应用于多方案、生成器、语言及模型 regime，揭示单语言评估无法发现的失效模式，发现水印跨语言公平性差距源于语言属性的结构性问题。
AI 中文摘要
大型语言模型输出的水印方案几乎仅在英文文本上评估，使用各方案的检测阈值和少量质量测量指标。多语言部署暴露出在英文上无关紧要但会决定跨语言结论的评估设计选择。我们提出含四个组件的评估框架：按部署语境经验校准的检测阈值、区分校准失败与检测失败的阈值无关配套测量、三种不重叠的质量测量范式（分布型、配对语义型、参考困惑度型），以及按类型学家族划分的跨语言差异广义熵分解。将该框架应用于六种水印方案、三个开放权重生成器、十一种分属四种文字和八个类型学家族的语言，以及基础和指令微调两种 regime，它揭示了单语言单范式评估无法发现的失效模式。在检测与质量方面，观察到的差异在类型学划分上主要为家族间差异，表明水印的跨语言公平性差距是语言属性的结构性问题，而非特定语言的特质。
英文摘要
Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme's detection threshold and a narrow set of quality measurements. Multilingual deployment exposes evaluation-design choices that are inconsequential on English but determine conclusions cross-lingually. We propose an evaluation framework with four components: detection thresholds calibrated empirically per deployment context, a threshold-independent companion measurement that distinguishes calibration failures from detection failures, three disjoint quality measurement paradigms (distributional, paired-semantic, and reference-perplexity), and a generalized-entropy decomposition of cross-language disparity over a typological family partition. Applied to six watermarking schemes, three open-weight generators, eleven languages spanning four scripts and eight typological families, and both base and instruction-tuned regimes, the framework reveals failure modes that single-language single-paradigm evaluation cannot surface. Across detection and quality, observed disparity is predominantly between-family on the typological partition, indicating that cross-lingual fairness gaps in watermarking are structural to language properties rather than idiosyncratic to particular languages.
Comments24 pages
