TACTICS：面向机器翻译的基于分类法的智能语料采样
TACTICS: Taxonomy-Aware Intelligent Corpus Sampling for Machine Translation
浏览论文内容
中文总结 AI 辅助
针对随机采样无法保证覆盖的问题，提出TACTICS方法，通过从风格指南归纳分类法并优化覆盖与分布保真度，在更少样本下准确评估机器翻译系统。
中文摘要 AI 辅助
大规模机器翻译（MT）系统通常在从语料库中随机抽取的样本上进行评估，而该语料库的分布构成是其组装方式的人为产物。这样的样本继承了集合中恰好包含的现象，而非系统必须处理的完整空间，涵盖规则性约定（术语、标点、货币格式）和上下文相关现象（语气、敬语、文档级连贯性），因此无法为评估鲁棒性提供覆盖保证。我们提出TACTICS（基于分类法的覆盖优化智能语料采样），将覆盖重新定义为显式目标。TACTICS从语言风格指南中归纳出层次化分类法，据此对片段进行分类，并选择一个固定预算的子集，联合优化稀有类别的覆盖、文档级连贯性以及对完整语料库的分布保真度。应用于四个翻译方向的MT评估，TACTICS在稀有类别覆盖上优于基于词汇和基于嵌入的选择。通过针对区分系统的现象，TACTICS使固定评估预算发挥更大作用，在存在真实质量差距的情况下，从远少于随机采样的片段中恢复真实的系统排名，并且在不存在差异时从不发出差异信号。
英文摘要
Large-scale machine-translation (MT) systems are typically evaluated on random samples from a corpus whose distributional composition is an artifact of how it was assembled. Such a sample inherits the phenomena the collection happens to contain rather than the full space a system must handle, spanning rule-governed conventions (terminology, punctuation, currency formatting) and context-dependent phenomena (tone, honorifics, document-level coherence), and thus provides no coverage guarantee for assessing robustness. We propose TACTICS (Taxonomy-Aware Coverage-opTimized Intelligent Corpus Sampling), which recasts coverage as an explicit objective. TACTICS induces a hierarchical taxonomy from a locale style guide, classifies segments against it, and selects a fixed-budget subset jointly optimizing coverage of rare categories, document-level coherence, and distributional fidelity to the full corpus. Applied to MT evaluation across four translation directions, TACTICS improves coverage of rare categories over lexical and embedding-based selection. By targeting the phenomena that separate systems, TACTICS makes a fixed evaluation budget go further, recovering the true system ranking from far fewer segments than random sampling wherever a real quality gap exists and never signaling a difference where none exists.
发表机构
- Amazon（亚马逊）
机构由 AI 辅助整理，请以论文原文为准。
