arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08600cs.CL
收藏

超越标量评分：用于机器翻译错误感知质量估计的强化学习

Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translation

Institute for People-Centred AI（以人为本的人工智能研究所）；University of Surrey（塞弗尔大学）；NICE Research Group（NICE研究组）；University of Oslo（奥斯陆大学）；Centre for Translation Studies（翻译研究中心）

Archchana Sindhujan, Girish A. Koushik, Shenbin Qian, Diptesh Kanojia, Constantin Orăsan

备注：
Currently this article is under review for Natural Language Processing Journal
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出ALOPE-RL框架，通过强化学习实现基于错误感知的机器翻译质量估计，在低资源语言上取得优异效果。

摘要：

质量估计（QE）旨在在不依赖参考翻译的情况下评估机器翻译（MT）输出的质量，这对于实际应用中的大规模MT评估至关重要。大型语言模型（LLMs）在推动机器翻译质量估计领域的发展中展现了显著的前景。然而，大多数QE方法仅依赖于标量质量评分，没有提供关于翻译错误的显式信息，这些信息应驱动这些判断。此外，在低资源语言中，由于标注QE数据有限，现有方法难以实现可靠的性能。为了解决这些挑战，我们引入了第一个基于段落的QE数据集，用于英语到马拉雅拉语的翻译质量估计，这是一个在QE领域中资源极度匮乏的语言对，包含由人工标注的直接评估（DA）评分和翻译质量评论（TQR），这些是简短的、上下文相关的、自由形式的标注者评论，用于描述翻译错误。我们进一步引入了ALOPE-RL，一种基于策略的强化学习框架，该框架通过基于DA评分和TQR的策略奖励训练高效的适配器。将具有错误感知的奖励与ALOPE-RL结合，使LLMs能够超越数值评分进行翻译质量的推理。尽管是在一个小规模的QE数据集上训练的，ALOPE-RL在使用紧凑的LLM（<=4B参数）并通过LoRA和4位量化进行微调后，在英语到马拉雅拉语的QE任务中实现了最先进的性能，优于更大的基于LLM的基线方法和领先的编码器基QE模型。我们的结果表明，在数据和计算预算有限的情况下，具有错误感知的基于策略的学习可以提供强大的QE性能。我们发布了我们的数据集、代码和训练好的模型以支持未来的研究。

英文摘要：

Quality Estimation (QE) aims to assess the quality of machine translation (MT) outputs without relying on reference translations, making it essential for real-world, large-scale MT evaluation. Large Language Models (LLMs) have shown significant promise in advancing the field of quality estimation of machine translation. However, most of the QE approaches solely rely on scalar quality scores, offering no explicit information about the translation errors that should drive these judgments. Moreover, for low-resource languages where annotated QE data is limited, existing approaches struggle to achieve reliable performance. To address these challenges, we introduce the first segment-level QE dataset for English to Malayalam, a severely resource-scarce language pair in the QE domain, comprising human-annotated Direct Assessment (DA) scores and Translation Quality Remarks (TQR), which are short, contextual, free-form annotator comments that describe translation errors. We further introduce ALOPE-RL, a policy-based reinforcement learning framework that trains efficient adapters based on policy rewards derived from DA score and TQR. Integrating error-aware rewards with ALOPE-RL, enables LLMs to reason about translation quality beyond numeric scores. Despite being trained on a small-scale QE dataset, ALOPE-RL achieves state-of-the-art performance on English to Malayalam QE using compact LLMs (<=4B parameters}) fine-tuned with LoRA and 4-bit quantization, outperforming both larger LLM-based baselines and leading encoder-based QE models. Our results demonstrate that error-aware, policy-based learning can deliver strong QE performance under limited data and compute budgets. We release our dataset, code, and trained models to support future research.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑