自动评估与人工评估之间的趋同与分歧：来自比较ChatGPT生成翻译与神经机器翻译的启示
Convergences and Divergences between Automatic Assessment and Human Evaluation: Insights from Comparing ChatGPT-Generated Translation and Neural Machine Translation
AI总结：
本研究比较了ChatGPT与NMT系统的翻译质量，发现自动评估与人工评估在形式保真度上一致，但在语义和语用保真度上存在分歧，突显了人工判断的不可或缺性。
摘要：
大型语言模型已展现出与神经机器翻译（NMT）系统相当甚至更优的翻译性能。然而，现有的比较研究主要依赖自动化指标，引发了对这些指标的可行性及其与人工判断一致性的质疑。本研究调查了在评估ChatGPT和三个NMT系统的机器翻译质量时，自动化指标与人工评估之间的趋同与分歧。为进行自动评估，采用了四种自动化指标，而人工评估则结合了DQF-MQM错误类型学和六项评分准则。值得注意的是，自动评估与人工评估在衡量形式保真度（如错误率）时趋于一致，但在评估语义和语用保真度时出现分歧，自动化指标未能捕捉到提示工程为ChatGPT翻译带来的改进。这些结果突显了在现阶段评估高级翻译工具性能时人工判断的不可或缺作用。
英文摘要：
Large language models have demonstrated parallel and even superior translation performance compared to neural machine translation (NMT) systems. However, existing comparative studies between them mainly rely on automated metrics, raising questions into the feasibility of these metrics and their alignment with human judgment. The present study investigates the convergences and divergences between automated metrics and human evaluation in assessing the quality of machine translation from ChatGPT and three NMT systems. To perform automatic assessment, four automated metrics are employed, while human evaluation incorporates the DQF-MQM error typology and six rubrics. Notably, automatic assessment and human evaluation converge in measuring formal fidelity (e.g., error rates), but diverge when evaluating semantic and pragmatic fidelity, with automated metrics failing to capture the improvement of ChatGPT's translation brought by prompt engineering. These results underscore the indispensable role of human judgment in evaluating the performance of advanced translation tools at the current stage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
