人类判断作为指南针导航形式转换的自动导航
Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer
摘要：
虽然文本风格迁移近年来发展迅速，但目前还没有一个既定的评估标准，这是使用几个自动度量来执行的，缺乏总是诉诸于人类判断的可能性。我们关注的是正式迁移的任务，以及通常评估的三个方面：风格强度，内容保留和流畅性。为了阐明这些方面是如何通过常见的和新的指标进行评估的，我们运行了一个基于人类的评估，并执行了丰富的相关性分析。然后，我们能够提供一些建议，使用这些指标的正式转让，也着眼于他们的普遍性（或不）相关的任务。
英文摘要：
Although text style transfer has witnessed rapid development in recent years, there is as yet no established standard for evaluation, which is performed using several automatic metrics, lacking the possibility of always resorting to human judgement. We focus on the task of formality transfer, and on the three aspects that are usually evaluated: style strength, content preservation, and fluency. To cast light on how such aspects are assessed by common and new metrics, we run a human-based evaluation and perform a rich correlation analysis. We are then able to offer some recommendations on the use of such metrics in formality transfer, also with an eye to their generalisability (or not) to related tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
