arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11577cs.CLmath.HO
收藏

数学实体:语料库与基准

Mathematical Entities: Corpora and Benchmarks

National Institute of Standards and Technology(美国国家标准与技术研究院);Topos Institute(拓扑斯研究所);Carnegie Mellon University(卡内基梅隆大学)

Jacob Collard, Valeria de Paiva, Eswaran Subrahmanian

AI总结:

本文构建含182397句的三个数学标注语料库及元数据基准,评估神经与符号模型,发现术语和定义提取难以泛化,并提供基于搜索与实体链接的学习助手。

摘要:

数学是一个高度专业化的领域,具有其独特的一系列挑战。尽管如此,针对数学文本的自然语言处理研究相对较少,面向 NLP 的数学语言资源也很匮乏。本文旨在提供带标注的语料库,用于研究不同语境下的数学语言,范围从教科书中的基础概念到高等研究数学。我们使用神经解析模型并辅以一定的人工干预对语料库进行预处理,以提供词性标注、词元还原和依存句法树。我们总共提供了跨越三个语料库的 182397 个句子。随后,我们旨在使用这些语料库测试和评估若干值得关注的自然语言处理模型,以展示它们对数学领域的适应程度,并为探索数学语言提供有用的工具。我们依据从语料库元数据中提取的基准评估了若干神经模型和符号模型,结果表明术语提取和定义提取并不容易泛化到数学领域,需要进一步研究才能在这些指标上取得良好性能。最后,我们提供了一个学习助手,利用文本搜索和实体链接,以上下文敏感的方式访问这些语料库的内容。尽管我们的语料库和基准为评估数学语言处理提供了有用的指标,但仍需进一步工作使模型适应数学,从而提供更有效的学习助手,并将 NLP 方法应用于不同的数学领域。

英文摘要:

Mathematics is a highly specialized domain with its own unique set of challenges. Despite this, there has been relatively little research on natural language processing for mathematical texts, and there are few mathematical language resources aimed at NLP. In this paper, we aim to provide annotated corpora that can be used to study the language of mathematics in different contexts, ranging from fundamental concepts found in textbooks to advanced research mathematics. We preprocess the corpora with a neural parsing model and some manual intervention to provide part-of-speech tags, lemmas, and dependency trees. In total, we provide 182397 sentences across three corpora. We then aim to test and evaluate several noteworthy natural language processing models using these corpora, to show how well they can adapt to the domain of mathematics and provide useful tools for exploring mathematical language. We evaluate several neural and symbolic models against benchmarks that we extract from the corpus metadata to show that terminology extraction and definition extraction do not easily generalize to mathematics, and that additional work is needed to achieve good performance on these metrics. Finally, we provide a learning assistant that grants access to the content of these corpora in a context-sensitive manner, utilizing text search and entity linking. Though our corpora and benchmarks provide useful metrics for evaluating mathematical language processing, further work is necessary to adapt models to mathematics in order to provide more effective learning assistants and apply NLP methods to different mathematical domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑