Metadata of scientific articles such as title, abstract, keywords or index terms, body text, conclusion, reference and others play a decisive role in collecting, managing and storing academic data in scientific databases, academic journals and digital libraries. An accurate extraction of these kinds of data from scientific papers is crucial to organize and retrieve important scientific information for researchers as well as librarians. Research social network systems and academic digital library systems provide academic data extracting, organizing and retrieving services. Mostly these types of services are not free or open source. They also have some performance problems and extracting limitations in the number of PDF (Portable Document Format) files that you can upload to the extraction systems. In this paper, a completely free and open source Java based high performance metadata extraction framework is proposed. This frameworks extraction speed is 9-10 times faster than existing metadata extraction systems. It is also flexible in that it allows uploading of unlimited number of PDF files. In this approach, titles of papers are extracted using layout features, font and size characteristics of text. Other metadata fields such as abstracts, body text, keywords, conclusions and references are extracted from PDF files using fixed rule sets. Extracted metadata are stored in both Oracle database and XML (Extensible Markup Language) file. This framework can be used to make scientific collections in digital libraries, online journals, online and offline scientific databases, government research agencies and research centers.
Kumoh National Institute of Technology（国立金乌工科大学）；Korea Institute of Science and Technology Information（韩国科学技术信息研究院）；Korea University of Technology and Education（韩国技术教育大学）
The volume of academic literature, such as academic conference papers and journals, has increased rapidly worldwide, and research on metadata extraction is ongoing. However, high-performing metadata extraction is still challenging due to diverse layout formats according to journal publishers. To accommodate the diversity of the layouts of academic journals, we propose a novel LAyout-aware Metadata Extraction (LAME) framework equipped with the three characteristics (e.g., design of an automatic layout analysis, construction of a large meta-data training set, and construction of Layout-MetaBERT)…
Nowadays, metadata information is often given by the authors themselves upon submission. However, a significant part of already existing research papers have missing or incomplete metadata information. German scientific papers come in a large variety of layouts which makes the extraction of metadata a non-trivial task that requires a precise way to classify the metadata extracted from the documents. In this paper, we propose a multimodal deep learning approach for metadata extraction from scientific papers in the German language. We consider multiple types of input data by combining natural la…
Extracting information from academic PDF documents is crucial for numerous indexing, retrieval, and analysis use cases. Choosing the best tool to extract specific content elements is difficult because many, technically diverse tools are available, but recent performance benchmarks are rare. Moreover, such benchmarks typically cover only a few content elements like header metadata or bibliographic references and use smaller datasets from specific academic disciplines. We provide a large and diverse evaluation framework that supports more extraction tasks than most related datasets. Our framewor…
Relation Extraction (RE) from tables is the task of identifying relations between pairs of columns of a table. Generally, RE models for this task require labelled tables for training. These labelled tables can also be generated artificially from a Knowledge Graph (KG), which makes the cost to acquire them much lower in comparison to manual annotations. However, unlike real tables, these synthetic tables lack associated metadata, such as, column-headers, captions, etc; this is because synthetic tables are created out of KGs that do not store such metadata. Meanwhile, previous works have shown t…
Clinical Document Metadata Extraction: A Scoping Review
University of Minnesota（明尼苏达大学）；Mayo Clinic（妙佑医疗国际）；University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Oregon Health & Science University（俄勒冈健康与科学大学）
Within the past few decades we have witnessed digital revolution, which moved scholarly communication to electronic media and also resulted in a substantial increase in its volume. Nowadays keeping track with the latest scientific achievements poses a major challenge for the researchers. Scientific information overload is a severe problem that slows down scholarly communication and knowledge propagation across the academia. Modern research infrastructures facilitate studying scientific literature by providing intelligent search tools, proposing similar and related documents, visualizing citati…
Automatic extraction of forum posts and metadata is a crucial but challenging task since forums do not expose their content in a standardized structure. Content extraction methods, therefore, often need customizations such as adaptations to page templates and improvements of their extraction code before they can be deployed to new forums. Most of the current solutions are also built for the more general case of content extraction from web pages and lack key features important for understanding forum content such as the identification of author metadata and information on the thread structure. …
Nanjing University of Science and Technology（南京理工大学）
With the development of Internet technology, the phenomenon of information overload is becoming more and more obvious. It takes a lot of time for users to obtain the information they need. However, keyphrases that summarize document information highly are helpful for users to quickly obtain and understand documents. For academic resources, most existing studies extract keyphrases through the title and abstract of papers. We find that title information in references also contains author-assigned keyphrases. Therefore, this article uses reference information and applies two typical methods of un…
Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents
Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所FIT）；University Hospital of Cologne（科隆大学医院）；University of Koblenz（科布伦茨大学）；Soochow University（苏州大学）
Extracting metadata from scientific papers can be considered a solved problem in NLP due to the high accuracy of state-of-the-art methods. However, this does not apply to German scientific publications, which have a variety of styles and layouts. In contrast to most of the English scientific publications that follow standard and simple layouts, the order, content, position and size of metadata in German publications vary greatly among publications. This variety makes traditional NLP methods fail to accurately extract metadata from these publications. In this paper, we present a method that ext…
Building a Multimodal Dataset of Academic Paper for Keyword Extraction
Department of Information Management, Nanjing University of Science and Technology（南京理工大学经济管理学院信息管理系）；Department of Archives and E-government, Soochow University（苏州大学社会学院档案与电子政务系）
CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets
ETH Zurich（苏黎世联邦理工学院）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）；Helmholtz Zentrum München（亥姆霍兹慕尼黑研究中心）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Google（谷歌）；Duke Kunshan University（昆山杜克大学）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
The amount of data managed in many academic institutions has increased in recent years, particularly in all the research work done by undergraduate students, who simply use empirical techniques for keyword selection, forgetting existing technical methods to assist their students in this process. Information and communication technologies, such as the platform for integrated research and academic work with responsibility (PILAR), which records information about research projects, such as titles, summaries, and keywords in their various modalities, have gained relevance and importance in the man…
This paper proposes OCR++, an open-source framework designed for a variety of information extraction tasks from scholarly articles including metadata (title, author names, affiliation and e-mail), structure (section headings and body text, table and figure headings, URLs and footnotes) and bibliography (citation instances and references). We analyze a diverse set of scientific articles written in English language to understand generic writing patterns and formulate rules to develop this hybrid framework. Extensive evaluations show that the proposed framework outperforms the existing state-of-t…
This paper studies the automated categorization and extraction of scientific concepts from titles of scientific articles, in order to gain a deeper understanding of their key contributions and facilitate the construction of a generic academic knowledgebase. Towards this goal, we propose an unsupervised, domain-independent, and scalable two-phase algorithm to type and extract key concept mentions into aspects of interest (e.g., Techniques, Applications, etc.). In the first phase of our algorithm we propose PhraseType, a probabilistic generative model which exploits textual features and limited …
Accurate extraction of body text from PDF-formatted academic documents is essential in text-mining applications for deeper semantic understandings. The objective is to extract complete sentences in the body text into a txt file with the original sentence flow and paragraph boundaries. Existing tools for extracting text from PDF documents would often mix body and nonbody texts. We devise and implement a system called PDFBoT to detect multiple-column layouts using a line-sweeping technique, remove nonbody text using computed text features and syntactic tagging in backward traversal, and align th…
Indexing learning documents using the Learning Object Metadata (LOM) is often carried out manually by archivists. Filling out the LOM fields is a long and difficult task, requiring a complete reading and a full knowledge on the topic dealt within the document. In this paper, we present an innovative model and method to assist the archivists in finding the important concepts and keywords of a learning document. The application is performed using wikipedia's category links.
Microsoft Research hosted the 2016 WSDM Cup Challenge based on the Microsoft Academic Graph. The goal was to provide static rankings for the articles that make up the graph, with the rankings to be evaluated against those of human judges. While the Microsoft Academic Graph provided metadata about many aspects of each scholarly document, we focused more narrowly on citation data and used this contest as an opportunity to test the Article Level Eigenfactor (ALEF), a novel citation-based ranking algorithm, and evaluate its performance against competing algorithms that drew upon multiple facets of…
Beijing University of Posts and Telecommunications（北京邮电大学）
Aiming at the problem that the current general-purpose semantic text similarity calculation methods are difficult to use the semantic information of scientific academic conference data, a semantic similarity calculation algorithm for scientific academic conferences by fusion with domain features is proposed. First, the domain feature information of the conference is obtained through entity recognition and keyword extraction, and it is input into the BERT network as a feature and the conference information. The structure of the Siamese network is used to solve the anisotropy problem of BERT. Th…
Drexel University（德雷塞尔大学）；Tulane University Biodiversity Research Institute（杜兰大学生物多样性研究所）；Carnegie Mellon University（卡内基梅隆大学）
Flexible metadata pipelines are crucial for supporting the FAIR data principles. Despite this need, researchers seldom report their approaches for identifying metadata standards and protocols that support optimal flexibility. This paper reports on an initiative targeting the development of a flexible metadata pipeline for a collection containing over 300,000 digital fish specimen images, harvested from multiple data repositories and fish collections. The images and their associated metadata are being used for AI-related scientific research involving automated species identification, segmentati…
Tabular PDF Information Extraction with Local LLMs and Layout-Aware Parsing: A Reliability Evaluation
Faculty of Engineering, Universitas Swadaya Gunung Jati（工程学院，Swadaya Gunung Jati大学）；University of California, Irvine（加州大学伊维奇分校）；UNIR, La Rioja（UNIR，拉里奥ja）；Universitas Diponegoro（迪波内戈罗大学）
"Keyword Extraction" refers to the task of automatically identifying the most relevant and informative phrases in natural language text. As we are deluged with large amounts of text data in many different forms and content - emails, blogs, tweets, Facebook posts, academic papers, news articles - the task of "making sense" of all this text by somehow summarizing them into a coherent structure assumes paramount importance. Keyword extraction - a well-established problem in Natural Language Processing - can help us here. In this report, we construct and test three different hypotheses (all relate…
In this paper, we describe the Lithium Natural Language Processing (NLP) system - a resource-constrained, high- throughput and language-agnostic system for information extraction from noisy user generated text on social media. Lithium NLP extracts a rich set of information including entities, topics, hashtags and sentiment from text. We discuss several real world applications of the system currently incorporated in Lithium products. We also compare our system with existing commercial and academic NLP systems in terms of performance, information extracted and languages supported. We show that L…
CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC). Buenos Aires, Argentina（CONICET-阿根廷布宜诺斯艾利斯大学。信息计算研究所（ICC）。布宜诺斯艾利斯，阿根廷）；École de bibliothéconomie et des sciences de l’information. Université de Montréal. Montréal, Canada（信息与图书馆科学学院。蒙特利尔大学。蒙特利尔，加拿大）
It is the most important way for researchers to acquire academic progress via reading scientific papers, most of which are in PDF format. However, existing PDF Readers like Adobe Acrobat Reader and Foxit PDF Reader are usually only for reading by rendering PDF files as a whole, and do not consider the multi-granularity content understanding of a paper itself. Specifically, taking a paper as a basic and separate unit, existing PDF Readers cannot access extended information about the paper, such as corresponding videos, blogs and codes. Meanwhile, they cannot understand the academic content of a…
Indian Institute of Technology Gandhinagar（甘地纳格尔印度理工学院）
Scientific documents contain tables that list important information in a concise fashion. Structure and content extraction from tables embedded within PDF research documents is a very challenging task due to the existence of visual features like spanning cells and content features like mathematical symbols and equations. Most existing table structure identification methods tend to ignore these academic writing features. In this paper, we adapt the transformer-based language modeling paradigm for scientific table structure and content extraction. Specifically, the proposed model converts a tabu…
Nanjing University of Science and Technology（南京理工大学）
In the era of big data, the advancement, improvement, and application of algorithms in academic research have played an important role in promoting the development of different disciplines. Academic papers in various disciplines, especially computer science, contain a large number of algorithms. Identifying the algorithms from the full-text content of papers can determine popular or classical algorithms in a specific field and help scholars gain a comprehensive understanding of the algorithms and even the field. To this end, this article takes the field of natural language processing (NLP) as …
LexNLP is an open source Python package focused on natural language processing and machine learning for legal and regulatory text. The package includes functionality to (i) segment documents, (ii) identify key text such as titles and section headings, (iii) extract over eighteen types of structured information like distances and dates, (iv) extract named entities such as companies and geopolitical entities, (v) transform text into features for model training, and (vi) build unsupervised and supervised models such as word embedding or tagging models. LexNLP includes pre-trained models based on …
Kansas State University（堪萨斯州立大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
This paper describes a machine learning and data science pipeline for structured information extraction from documents, implemented as a suite of open-source tools and extensions to existing tools. It centers around a methodology for extracting procedural information in the form of recipes, stepwise procedures for creating an artifact (in this case synthesizing a nanomaterial), from published scientific literature. From our overall goal of producing recipes from free text, we derive the technical objectives of a system consisting of pipeline stages: document acquisition and filtering, payload …
