arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.02577cs.CL
收藏

DeepL与Supertext的翻译性能对比

A comparison of translation performance between DeepL and Supertext

Supertext

Alex Flückiger, Chantal Amrhein, Tim Graf, Frédéric Odermatt, Martin Pömsl, Philippe Schläpfer, Florian Schottmann, Samuel Läubli

AI总结:

本研究对比DeepL与Supertext两款商用MT系统在未分段文本上的翻译性能,发现片段级评估无明显偏好,但文档级评估显示四个语言方向中三个更偏好Supertext,呼吁采用上下文敏感的MT评估方法并开源了相关数据与脚本。

摘要:

随着高性能机器翻译(MT)系统越来越多地基于大语言模型(LLM),可靠的质量基准测试需要能够捕捉其利用扩展上下文能力的方法。本研究对比了两款商用机器翻译系统——DeepL和Supertext,评估它们在未分段文本上的表现。我们在四个语言方向上评估翻译质量,由专业译员在完整文档级上下文下对译文片段进行评分。尽管片段级评估显示多数情况下两种系统没有明显偏好,但文档级分析表明在四个语言方向中有三个更偏好Supertext,说明其在长文本中的一致性更优。我们提倡采用更具上下文敏感性的评估方法,以确保机器翻译质量评估能反映实际可用性。我们在https://github.com/supertext/evaluation_deepl_supertext发布了所有评估数据和脚本,供进一步分析与复现使用。

英文摘要:

As strong machine translation (MT) systems are increasingly based on large language models (LLMs), reliable quality benchmarking requires methods that capture their ability to leverage extended context. This study compares two commercial MT systems -- DeepL and Supertext -- by assessing their performance on unsegmented texts. We evaluate translation quality across four language directions with professional translators assessing segments with full document-level context. While segment-level assessments indicate no strong preference between the systems in most cases, document-level analysis reveals a preference for Supertext in three out of four language directions, suggesting superior consistency across longer texts. We advocate for more context-sensitive evaluation methodologies to ensure that MT quality assessments reflect real-world usability. We release all evaluation data and scripts for further analysis and reproduction at https://github.com/supertext/evaluation_deepl_supertext.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑