arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.05129cs.CL
收藏

学术文档中的文档级定义检测:现有模型、错误分析和未来方向

Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions

University of California, Berkeley(加州大学伯克利分校);Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学)

Dongyeop Kang, Andrew Head, Risham Sidhu, Kyle Lo, Daniel S. Weld, Marti A. Hearst

摘要:

定义检测的任务对于学术论文很重要,因为论文经常使用读者可能不熟悉的技术术语。尽管之前的工作定义检测,目前的方法是远远不够准确,在现实世界中的应用程序。在本文中,我们首先进行了深入的错误分析,目前最好的清晰度检测系统,发现错误的主要原因。基于这种分析,我们开发了一个新的定义检测系统,HEDDEx,利用语法功能,Transformer编码器,和启发式过滤器,并评估它的一个标准的性能测试级基准。由于目前的基准评估随机采样的句子,我们提出了一种替代的评估,评估文件中的每一句话。这允许除了精确度之外还评估召回率。HEDDEx在文件级和文档级任务上的表现分别优于领先系统12.7 F1点和14.4 F1点。我们注意到,高召回率的文档级任务的性能是远远低于标准的评估方法,由于纳入文档结构的功能的必要性。我们讨论了在文档级定义检测,改进的想法,和阅读辅助应用程序的发展的潜在问题仍然存在的挑战。

英文摘要:

The task of definition detection is important for scholarly papers, because papers often make use of technical terminology that may be unfamiliar to readers. Despite prior work on definition detection, current approaches are far from being accurate enough to use in real-world applications. In this paper, we first perform in-depth error analysis of the current best performing definition detection system and discover major causes of errors. Based on this analysis, we develop a new definition detection system, HEDDEx, that utilizes syntactic features, transformer encoders, and heuristic filters, and evaluate it on a standard sentence-level benchmark. Because current benchmarks evaluate randomly sampled sentences, we propose an alternative evaluation that assesses every sentence within a document. This allows for evaluating recall in addition to precision. HEDDEx outperforms the leading system on both the sentence-level and the document-level tasks, by 12.7 F1 points and 14.4 F1 points, respectively. We note that performance on the high-recall document-level task is much lower than in the standard evaluation approach, due to the necessity of incorporation of document structure as features. We discuss remaining challenges in document-level definition detection, ideas for improvements, and potential issues for the development of reading aid applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑