WikiIns:一个用于通过自然语言指令进行受控文本编辑的高质量数据集
WikiIns: A High-Quality Dataset for Controlled Text Editing by Natural Language Instruction
AI总结:
针对现有受控文本编辑数据集指令信息不足的问题,研究者构建并发布了高质量数据集WikiIns,包含多类标注数据及自动生成的大规模银标训练集,并开展了相关分析以助力文本编辑研究。
摘要:
文本编辑,即修改或处理文本的过程,是人类写作流程中的关键步骤。本文研究了通过自然语言指令实现受控文本编辑的问题:根据传达编辑意图与必要信息的给定指令,需要将原始草稿文本修改为目标文本。现有的针对该任务的自动构建数据集存在局限性,因为它们不具备信息丰富的自然语言指令——而信息丰富性要求指令中包含的信息足以生成修改后的文本。为解决这一局限,我们构建并发布了WikiIns,这是一个信息丰富度更高的高质量受控文本编辑数据集。我们首先对维基百科编辑历史数据库进行预处理,提取原始数据(WikiIns-Raw);随后通过众包方式构建了高质量的验证集、测试集以及小规模训练集(WikiIns-Gold)。借助该高质量标注数据集,我们进一步提出了自动方法,用于生成大规模的“银标”训练集(WikiIns-Silver)。最后,我们对WikiIns数据集进行了多项有价值的分析,包括评估结果与编辑意图分析。我们在WikiIns上的分析与实验结果可为当前的文本编辑研究提供助力。该数据集、源代码与标注指南可在https://github.com/CasparSwift/WikiIns获取。
英文摘要:
Text editing, i.e., the process of modifying or manipulating text, is a crucial step in human writing process. In this paper, we study the problem of controlled text editing by natural language instruction. According to a given instruction that conveys the edit intention and necessary information, an original draft text is required to be revised into a target text. Existing automatically constructed datasets for this task are limited because they do not have informative natural language instruction. The informativeness requires the information contained in the instruction to be enough to produce the revised text. To address this limitation, we build and release WikiIns, a high-quality controlled text editing dataset with improved informativeness. We first preprocess the Wikipedia edit history database to extract the raw data (WikiIns-Raw). Then we crowdsource high-quality validation and test sets, as well as a small-scale training set (WikiIns-Gold). With the high-quality annotated dataset, we further propose automatic approaches to generate a large-scale ``silver'' training set (WikiIns-Silver). Finally, we provide some insightful analysis on our WikiIns dataset, including the evaluation results and the edit intention analysis. Our analysis and the experiment results on WikiIns may assist the ongoing research on text editing. The dataset, source code and annotation guideline are available at https://github.com/CasparSwift/WikiIns.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。