IFMTBench：多语言翻译指令遵循的综合基准
IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
AI总结：
提出IFMTBench基准，涵盖7种语言、4506个单约束和2838个多约束项，通过确定性检查器和基于LLM的评分器评估翻译指令遵循能力，揭示指令遵循随模型规模增长快于翻译质量，且术语表和结构化格式约束难度最高。
摘要：
现代翻译工作流程要求的不只是语义等价。用户通常要求模型保留JSON或HTML模式、遵循精心策划的术语表、利用提供的上下文消除歧义，并匹配指定的语域，往往同时满足多个条件。传统的BLEU和xCOMET等指标能捕捉语义保真度，但对约束遵循的指示甚少，而一般的指令遵循基准则忽略了翻译的跨语言性质。我们引入了\bench，一个涵盖七种语言的多语言翻译指令遵循基准，包含4506个单约束和2838个多约束项，跨越六个约束维度和五种组合模式，指令以所有七种语言发出。约束分为由确定性检查器验证的门控子集和由基于评分规则的LLM法官评分的连续子集，通过乘法规则组合以抵抗奖励黑客攻击。评估15个模型揭示了先前协议遗漏的系统性差距：指令遵循随模型规模增长比翻译质量更显著，术语表和结构化格式约束主导了难度梯度，而一般指令遵循排名与翻译行为的相关性很弱。我们的基准可在https://github.com/Tencent-Hunyuan/Hy-MT2/tree/main/IFMTBench获取。
英文摘要：
Modern translation workflows demand more than semantic equivalence. Users routinely require models to preserve JSON or HTML schemas, honor curated glossaries, disambiguate with provided context, and match prescribed registers, often several at once. Conventional metrics such as BLEU and xCOMET capture semantic fidelity but provide little signal on constraint adherence, while general instruction following benchmarks ignore the cross-lingual nature of translation. We introduce \bench, a benchmark for multilingual translation instruction following covering seven languages, with 4,506 single-constraint and 2,838 multi-constraint items spanning six constraint dimensions and five compositional patterns with instructions issued in all seven languages. Constraints are split into a gating subset verified by deterministic checkers and a continuous subset scored by a rubric-based LLM judge, combined under a multiplicative rule that resists reward hacking. Evaluating 15 models reveals systematic gaps that prior protocols miss: Instruction following scales with size more sharply than translation quality, glossary and structured-format constraints dominate the difficulty gradient, and general instruction following rankings correlate only weakly with translation behavior. Our benchmark are available at https://github.com/Tencent-Hunyuan/Hy-MT2/tree/main/IFMTBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
