利用自训练消除可训练MT模型中的偏差
Removing Biases from Trainable MT Metrics by Using Self-Training
摘要:
大多数可训练的机器翻译(MT)度量训练它们的权重是基于人类对最先进的MT系统输出的判断。这使得可训练的指标在许多方面存在偏差。其中之一是喜欢较长的翻译。当用于调优时,这些有偏见的指标正在评估不同类型的翻译-具有非常不同质量的n个最佳翻译列表。使用这些指标调优的系统往往会产生过长的翻译,这是指标所偏好的,但不是人类所偏好的。这通常是通过手动调整指标的权重来解决的,以使召回率和精度相等。我们的解决方案更通用:(1)它不仅解决了召回偏差,而且还解决了数据中可能存在的所有其他偏差,以及(2)它不需要任何关于所使用的特征类型的知识,这在不可能手动调整度量的权重的情况下是有用的。这是通过使用最初在标准人类判断上训练的度量在未标记的n最佳列表上进行自我训练来实现的。看待这一点的一种方式是将其视为从最先进的MT翻译领域到不同的n最佳列表翻译的领域自适应。
英文摘要:
Most trainable machine translation (MT) metrics train their weights on human judgments of state-of-the-art MT systems outputs. This makes trainable metrics biases in many ways. One of them is preferring longer translations. These biased metrics when used for tuning are evaluating different types of translations -- n-best lists of translations with very diverse quality. Systems tuned with these metrics tend to produce overly long translations that are preferred by the metric but not by humans. This is usually solved by manually tweaking metric's weights to equally value recall and precision. Our solution is more general: (1) it does not address only the recall bias but also all other biases that might be present in the data and (2) it does not require any knowledge of the types of features used which is useful in cases when manual tuning of metric's weights is not possible. This is accomplished by self-training on unlabeled n-best lists by using metric that was initially trained on standard human judgments. One way of looking at this is as domain adaptation from the domain of state-of-the-art MT translations to diverse n-best list translations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。