小而强大：拆分和重组的新基准
Small but Mighty: New Benchmarks for Split and Rephrase
摘要：
拆分和重组是一个文本简化任务，将复杂的句子改写成简单的句子。作为一项相对较新的任务，确保其评价基准和衡量标准的健全性至关重要。我们发现，广泛使用的基准数据集普遍包含容易利用的句法线索，其自动生成过程。利用这些线索，我们表明，即使是一个简单的基于规则的模型可以执行与国家的最先进的模型。为了弥补这些限制，我们收集并发布了两个众包基准数据集。我们不仅要确保它们包含更加多样化的语法，而且还要根据一组定义良好的标准仔细控制它们的质量。虽然不存在令人满意的自动度量，但我们使用众包基于这些标准应用细粒度手动评估，这表明我们的数据集更好地代表了任务，并且对模型更具挑战性。
英文摘要：
Split and Rephrase is a text simplification task of rewriting a complex sentence into simpler ones. As a relatively new task, it is paramount to ensure the soundness of its evaluation benchmark and metric. We find that the widely used benchmark dataset universally contains easily exploitable syntactic cues caused by its automatic generation process. Taking advantage of such cues, we show that even a simple rule-based model can perform on par with the state-of-the-art model. To remedy such limitations, we collect and release two crowdsourced benchmark datasets. We not only make sure that they contain significantly more diverse syntax, but also carefully control for their quality according to a well-defined set of criteria. While no satisfactory automatic metric exists, we apply fine-grained manual evaluation based on these criteria using crowdsourcing, showing that our datasets better represent the task and are significantly more challenging for the models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
