The task of definition detection is important for scholarly papers, because papers often make use of technical terminology that may be unfamiliar to readers. Despite prior work on definition detection, current approaches are far from being accurate enough to use in real-world applications. In this paper, we first perform in-depth error analysis of the current best performing definition detection system and discover major causes of errors. Based on this analysis, we develop a new definition detection system, HEDDEx, that utilizes syntactic features, transformer encoders, and heuristic filters, and evaluate it on a standard sentence-level benchmark. Because current benchmarks evaluate randomly sampled sentences, we propose an alternative evaluation that assesses every sentence within a document. This allows for evaluating recall in addition to precision. HEDDEx outperforms the leading system on both the sentence-level and the document-level tasks, by 12.7 F1 points and 14.4 F1 points, respectively. We note that performance on the high-recall document-level task is much lower than in the standard evaluation approach, due to the necessity of incorporation of document structure as features. We discuss remaining challenges in document-level definition detection, ideas for improvements, and potential issues for the development of reading aid applications.
NFDI4DS Shared Tasks for Scholarly Document Processing
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)(德意志人工智能研究中心有限公司);Leuphana University of Lüneburg(吕贝克大学);TIB Leibniz Information Centre for Science and Technology(科学与技术信息中心);Fraunhofer Institute for Open Communication Systems FOKUS(开放通信系统研究所);GESIS – Leibniz Institute for the Social Sciences(社会科学莱比锡研究所);University Hospital Cologne, University of Cologne, Institute for Biomedical Informatics(科隆大学医院,生物医学信息学研究所);FIZ-Karlsruhe – Leibniz-Institute for Information Infrastructure(信息基础设施莱比锡研究所)
GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews
International Laboratory on Learning Systems(国际学习系统实验室);MILA - Quebec AI Institute(MILA - 魁北克人工智能研究所);McGill University(麦吉尔大学);Université Paris-Saclay(巴黎萨克雷大学);CNRS, CentraleSupélec(法国国家科学研究中心、中央高等电力学院)
Complex Mathematical Symbol Definition Structures: A Dataset and Model for Coordination Resolution in Definition Extraction
University of Minnesota(明尼苏达大学);University of Pennsylvania(宾夕法尼亚大学);Allen Institute for AI(艾伦人工智能研究所);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校)
NeuroSym-BioCAT: Leveraging Neuro-Symbolic Methods for Biomedical Scholarly Document Categorization and Question Answering
TIB—Leibniz Information Centre for Science and Technology(莱布尼茨科学与技术信息中心(TIB));L3S Research Center, Leibniz University Hannover(汉诺威莱布尼茨大学L3S研究中心);Daffodil International University(水仙国际大学)
Structure-Tags Improve Text Classification for Scholarly Document Quality Prediction
Center for Information Technology, University of Groningen(格罗宁根大学信息技术中心);Kapteyn Astronomical Institute, University of Groningen(格罗宁根大学卡普泰恩天文研究所)
在长文本(特别是学术文档)上训练递归神经网络会给学习带来问题。虽然层次注意力网络(HAN)在解决这些问题方面是有效的,但它们仍然丢失了关于文本结构的重要信息。为了解决这些问题,我们建议使用HAN与结构标签相结合,标记文档中句子的作用。为句子添加标签,将它们标记为与标题,摘要或主体文本相对应,从而改进了学术文档质量预测的最新技术。所提出的系统被应用到接受/拒绝的PeerRead数据集上的预测任务,并与最近的基于BiLSTM的模型和联合文本+视觉模型以及普通HAN进行比较。与普通HAN相比,在所有三个域上的准确性都有所提高。在计算和语言领域,我们的新模型总体上效果最好,比最佳文献结果提高了4.7%的准确性。我们还在引入用于预测我们从Allen AI S2 ORC数据集编译的88 k科学出版物的引用数量的标签时获得了改进。对于我们的HAN系统与结构标签,我们达到了28.5%的解释方差,比我们的基于BiLSTM的模型的重新实现提高了1.8%,比普通HAN提高了1.0%。
Falcon 7b for Software Mention Detection in Scholarly Documents
Fraunhofer Institute for Applied Information Technology(弗劳恩霍夫应用信息技术研究所);University of Koblenz(科布伦茨大学);School of Future Science and Engineering, Soochow University(苏州大学未来科学与工程学院);University Hospital of Cologne(科隆大学医院)
Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents
Fraunhofer Institute for Applied Information Technology FIT(弗劳恩霍夫应用信息技术研究所FIT);University Hospital of Cologne(科隆大学医院);University of Koblenz(科布伦茨大学);Soochow University(苏州大学)
Enhancing Document-level Argument Extraction with Definition-augmented Heuristic-driven Prompting for LLMs
School of Engineering and Informatics, University of Sussex(萨塞克斯大学工程与信息学院);International Business School Suzhou, Xi’an Jiaotong-Liverpool University(西交利物浦大学苏州国际商学院);Management School, University of Liverpool(利物浦大学管理学院)
Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications
University of Mannheim(曼海姆大学);GESIS – Leibniz Institute for the Social Sciences(GESIS——莱布尼茨社会科学研究所);Fraunhofer ISI(弗劳恩霍夫系统与创新研究所);Stuttgart Media University(斯图加特传媒大学)
The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces
Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);University of California, Berkeley(加州大学伯克利分校);University of Pennsylvania(宾夕法尼亚大学);Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);KAIST(韩国科学技术院);Johns Hopkins University(约翰斯·霍普金斯大学);University of Minnesota(明尼苏达大学);University of California, San Diego(加州大学圣地亚哥分校);Microsoft Research(微软研究院)
本文介绍 Semantic Reader 项目,利用 AI 与 HCI 将遗留 PDF 转化为智能、交互式、可访问的学术阅读界面,并通过十个原型和 300 多名用户验证其效果。
SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline
Monash University(莫纳什大学);Deakin University(迪肯大学)
学术文献处理(SDP)研讨会是为了鼓励更多的努力对科学任务的自然语言的理解。它包含三个共享任务,我们参与LongSumm共享任务。在本文中,我们描述了我们的文本摘要系统,SciSummPip,灵感来自SummPip(Zhao等人,2020),这是一个无监督的文本摘要系统的多文档在新闻领域。我们的SciSummPip包括基于转换器的语言模型SciBERT(Beltagy等人,2019)用于上下文句子表示,使用PageRank进行内容选择(Page等人,1999),具有深度和语言信息的句子图构造,句子图聚类和图内摘要生成。我们的工作不同于以前的方法,在内容选择和摘要长度约束适用于科学领域。在训练数据集和盲测试数据集上的实验结果表明了我们方法的有效性,并且我们用BERTScore(Zhang et al.,2019年a)。
Span-level Detection of AI-generated Scientific Text via Contrastive Learning and Structural Calibration
Beijing Renhe Information Technology Co., Ltd.(北京润和信息技术有限公司);Key Laboratory of Digital Publishing and Total Process Management of Scientific and Technical Journals(科技期刊数字出版与全流程管理重点实验室)