大规模收集高质量内隐推理的比较研究
A Comparative Study on Collecting High-Quality Implicit Reasonings at a Large-scale
摘要：
在论证中显式隐式推理（即，保证）是自然语言理解系统的长期挑战。虽然最近的方法集中在通过众包或专家注释来解释认股权证，但由于任务的极端复杂性和主观性，认股权证的质量一直受到质疑。在本文中，我们解决了权证解释的复杂任务，并制定了各种方法收集权证。我们与训练有素的专家进行了广泛的研究，以评估每种方法产生的认股权证，并发现我们的方法可以收集高质量的认股权证。我们构建了一个包含6,000个认股权证的初步数据集，注释了3个有争议的主题的600多个论点。为了促进相关下游任务的研究，我们发布了我们的指南和初步数据集。
英文摘要：
Explicating implicit reasoning (i.e. warrants) in arguments is a long-standing challenge for natural language understanding systems. While recent approaches have focused on explicating warrants via crowdsourcing or expert annotations, the quality of warrants has been questionable due to the extreme complexity and subjectivity of the task. In this paper, we tackle the complex task of warrant explication and devise various methodologies for collecting warrants. We conduct an extensive study with trained experts to evaluate the resulting warrants of each methodology and find that our methodologies allow for high-quality warrants to be collected. We construct a preliminary dataset of 6,000 warrants annotated over 600 arguments for 3 debatable topics. To facilitate research in related downstream tasks, we release our guidelines and preliminary dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
