This paper is devoted to the study of methods for information extraction (entity recognition and relation classification) from scientific texts on information technology. Scientific publications provide valuable information into cutting-edge scientific advances, but efficient processing of increasing amounts of data is a time-consuming task. In this paper, several modifications of methods for the Russian language are proposed. It also includes the results of experiments comparing a keyword extraction method, vocabulary method, and some methods based on neural networks. Text collections for these tasks exist for the English language and are actively used by the scientific community, but at present, such datasets in Russian are not publicly available. In this paper, we present a corpus of scientific texts in Russian, RuSERRC. This dataset consists of 1600 unlabeled documents and 80 labeled with entities and semantic relations (6 relation types were considered). The dataset and models are available at https://github.com/iis-research-team. We hope they can be useful for research purposes and development of information extraction systems.
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Vietnam Maritime University（越南海事大学）
Named Entity Recognition (NER) is one of the most common tasks of the natural language processing. The purpose of NER is to find and classify tokens in text documents into predefined categories called tags, such as person names, quantity expressions, percentage expressions, names of locations, organizations, as well as expression of time, currency and others. Although there is a number of approaches have been proposed for this task in Russian language, it still has a substantial potential for the better solutions. In this work, we studied several deep neural network models starting from vanill…
ITMO University（ITMO大学）；JetBrains Research（JetBrains研究院）；Ural Federal University（乌拉尔联邦大学）
The paper presents RuBQ, the first Russian knowledge base question answering (KBQA) dataset. The high-quality dataset consists of 1,500 Russian questions of varying complexity, their English machine translations, SPARQL queries to Wikidata, reference answers, as well as a Wikidata sample of triples containing entities with Russian labels. The dataset creation started with a large collection of question-answer pairs from online quizzes. The data underwent automatic filtering, crowd-assisted entity linking, automatic generation of SPARQL queries, and their subsequent in-house verification.
This paper presents several BERT-based models for Russian language biomedical text mining (RuBioBERT, RuBioRoBERTa). The models are pre-trained on a corpus of freely available texts in the Russian biomedical domain. With this pre-training, our models demonstrate state-of-the-art results on RuMedBench - Russian medical language understanding benchmark that covers a diverse set of tasks, including text classification, question answering, natural language inference, and named entity recognition.
Lomonosov Moscow State University（莫斯科国立大学）；HSE University（高等经济大学）；Novosibirsk State University（新西伯利亚国立大学）；Ural Federal University（乌拉尔联邦大学）；Innopolis University（因诺波利斯大学）；Wiseyak（Wiseyak公司）；Kazan Federal University（喀山联邦大学）；Sber AI
In this paper, we present NEREL, a Russian dataset for named entity recognition and relation extraction. NEREL is significantly larger than existing Russian datasets: to date it contains 56K annotated named entities and 39K annotated relations. Its important difference from previous datasets is annotation of nested named entities, as well as relations within nested entities and at the discourse level. NEREL can facilitate development of novel models that can extract relations between nested named entities, as well as relations on both sentence and document levels. NEREL also contains the annot…
University of Notre Dame（圣母大学）；HLTCOE, JHU（约翰斯·霍普金斯大学自然语言处理中心（HLTCOE））；UMBC, HLTCOE（马里兰大学巴尔的摩分校（UMBC）自然语言处理中心（HLTCOE））
The goal of this work is to improve the performance of a neural named entity recognition system by adding input features that indicate a word is part of a name included in a gazetteer. This article describes how to generate gazetteers from the Wikidata knowledge graph as well as how to integrate the information into a neural NER system. Experiments reveal that the approach yields performance gains in two distinct languages: a high-resource, word-based language, English and a high-resource, character-based language, Chinese. Experiments were also performed in a low-resource language, Russian on…
HSE University（俄罗斯国立高等经济大学）；DeepPavlov lab, MIPT（莫斯科物理技术学院 DeepPavlov 实验室）；AIRI（人工智能研究所）；The Institute of Linguistics RAS（俄罗斯科学院语言学研究所）；Lomonosov Moscow State University（莫斯科国立罗蒙诺索夫大学）；Sber AI Centre（Sber 人工智能中心）
The vast majority of existing datasets for Named Entity Recognition (NER) are built primarily on news, research papers and Wikipedia with a few exceptions, created from historical and literary texts. What is more, English is the main source for data for further labelling. This paper aims to fill in multiple gaps by creating a novel dataset "Razmecheno", gathered from the diary texts of the project "Prozhito" in Russian. Our dataset is of interest for multiple research lines: literary studies of diary texts, transfer learning from other domains, low-resource or cross-lingual named entity recogn…
RheinMain University of Applied Sciences（莱茵美因应用科学大学）
Entity linking, the task of mapping textual mentions to known entities, has recently been tackled using contextualized neural networks. We address the question whether these results -- reported for large, high-quality datasets such as Wikipedia -- transfer to practical business use cases, where labels are scarce, text is low-quality, and terminology is highly domain-specific. Using an entity linking model based on BERT, a popular transformer network in natural language processing, we show that a neural approach outperforms and complements hand-coded heuristics, with improvements of about 20% t…
The paper describes the open Russian medical language understanding benchmark covering several task types (classification, question answering, natural language inference, named entity recognition) on a number of novel text sets. Given the sensitive nature of the data in healthcare, such a benchmark partially closes the problem of Russian medical dataset absence. We prepare the unified format labeling, data split, and evaluation metrics for new tasks. The remaining tasks are from existing datasets with a few modifications. A single-number metric expresses a model's ability to cope with the benc…
NRC "Kurchatov institute"（国家研究中心“库尔恰托夫研究所”）；MEPhI National Research Nuclear University（莫斯科工程物理学院国家核研究大学）；I.M. Sechenov First Moscow State Medical University (Sechenov University)（谢切诺夫第一莫斯科国立医科大学（谢切诺夫大学））
We present the full-size Russian complexly NER-labeled corpus of Internet user reviews, along with an evaluation of accuracy levels reached on this corpus by a set of advanced deep learning neural networks to extract the pharmacologically meaningful entities from Russian texts. The corpus annotation includes mentions of the following entities: Medication (33005 mentions), Adverse Drug Reaction (1778), Disease (17403), and Note (4490). Two of them - Medication and Disease - comprise a set of attributes. A part of the corpus has the coreference annotation with 1560 coreference chains in 300 docu…
Beijing University of Posts and Telecommunications（北京邮电大学）
Patent texts contain a large amount of entity information. Through named entity recognition, intellectual property entity information containing key information can be extracted from it, helping researchers to understand the patent content faster. Therefore, it is difficult for existing named entity extraction methods to make full use of the semantic information at the word level brought about by professional vocabulary changes. This paper proposes a method for extracting intellectual property entities based on Transformer and technical word information , and provides accurate word vector repr…
Hefei University of Technology（合肥工业大学）；Umeå University（于默奥大学）
While Named Entity Recognition (NER) is a widely studied task, making inferences of entities with only a few labeled data has been challenging, especially for entities with nested structures. Unlike flat entities, entities and their nested entities are more likely to have similar semantic feature representations, drastically increasing difficulties in classifying different entity categories in the few-shot setting. Although prior work has briefly discussed nested structures in the context of few-shot learning, to our best knowledge, this paper is the first one specifically dedicated to studyin…
Università degli studi di Milano（米兰大学）；University of Milano–Bicocca（米兰比可卡大学）
We begin by introducing the Computer Science branch of Natural Language Processing, then narrowing the attention on its subbranch of Information Extraction and particularly on Named Entity Recognition, discussing briefly its main methodological approaches. It follows an introduction to state-of-the-art Conditional Random Fields under the form of linear chains. Subsequently, the idea of constrained inference as a way to model long-distance relationships in a text is presented, based on an Integer Linear Programming representation of the problem. Adding such relationships to the problem as autom…
While transfer learning has become a ubiquitous technique used across Natural Language Processing (NLP) tasks, it is often unable to replicate the performance of pre-trained models on text of niche domains like Automotive. In this paper we aim to understand the main characteristics of the distribution shift with automotive domain text (describing technical functionalities such as Cruise Control) and attempt to explain the potential reasons for the gap in performance. We focus on performing the Named Entity Recognition (NER) task as it requires strong lexical, syntactic and semantic understandi…
Korea University（高丽大学）；University of California, San Diego（加州大学圣迭戈分校）
This paper is a technical report on our system submitted to the chemical identification task of the BioCreative VII Track 2 challenge. The main feature of this challenge is that the data consists of full-text articles, while current datasets usually consist of only titles and abstracts. To effectively address the problem, we aim to improve tagging consistency and entity coverage using various methods such as majority voting within the same articles for named entity recognition (NER) and a hybrid approach that combines a dictionary and a neural model for normalization. In the experiments on the…
A Zipf's Law-based Text Generation Approach for Addressing Imbalance in Entity Extraction
School of Information Resource Management, Renmin University of China（中国人民大学信息资源管理学院）；College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）
Domain adaptation of Russian ModernBERT for long legal documents
Independent Researcher（独立研究者）；National Research Nuclear University MEPhI（国立核研究大学 MEPhI）；Moscow Center for Advanced Studies（莫斯科高级研究中心）；University of Waterloo（滑铁卢大学）
Specialized transformer-based models for encoding tabular data have gained interest in academia. Although tabular data is omnipresent in industry, applications of table transformers are still missing. In this paper, we study how these models can be applied to an industrial Named Entity Recognition (NER) problem where the entities are mentioned in tabular-structured spreadsheets. The highly technical nature of spreadsheets as well as the lack of labeled data present major challenges for fine-tuning transformer-based models. Therefore, we develop a dedicated table data augmentation strategy base…
This technical report presents a text-to-image neural network "Emojich" that generates emojis using captions in Russian language as a condition. We aim to keep the generalization ability of a pretrained big model ruDALL-E Malevich (XL) 1.3B parameters at the fine-tuning stage, while giving special style to the images generated. Here are presented some engineering methods, code realization, all hyper-parameters for reproducing results and a Telegram bot where everyone can create their own customized sets of stickers. Also, some newly generated emojis obtained by "Emojich" model are demonstrated…
We analyze some of the fundamental design challenges that impact the development of a multilingual state-of-the-art named entity transliteration system, including curating bilingual named entity datasets and evaluation of multiple transliteration methods. We empirically evaluate the transliteration task using traditional weighted finite state transducer (WFST) approach against two neural approaches: the encoder-decoder recurrent neural network method and the recent, non-sequential Transformer method. In order to improve availability of bilingual named entity transliteration datasets, we releas…
ABBYY（ABBYY公司）；Charles University（查理大学）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Sberbank（俄罗斯联邦储蓄银行）；National Research University Higher School of Economics（国家研究型高等经济大学）
This paper provides a comprehensive overview of the gapping dataset for Russian that consists of 7.5k sentences with gapping (as well as 15k relevant negative sentences) and comprises data from various genres: news, fiction, social media and technical texts. The dataset was prepared for the Automatic Gapping Resolution Shared Task for Russian (AGRR-2019) - a competition aimed at stimulating the development of NLP tools and methods for processing of ellipsis. In this paper, we pay special attention to the gapping resolution methods that were introduced within the shared task as well as an alter…
University of Amsterdam（阿姆斯特丹大学）；Facebook AI Research（脸书人工智能研究实验室）；ENS, PSL University（巴黎高等师范学院、PSL大学）；Inria（法国国家信息与自动化研究所）；University College London（伦敦大学学院）
Entities are at the center of how we represent and aggregate knowledge. For instance, Encyclopedias such as Wikipedia are structured by entities (e.g., one per Wikipedia article). The ability to retrieve such entities given a query is fundamental for knowledge-intensive tasks such as entity linking and open-domain question answering. Current approaches can be understood as classifiers among atomic labels, one for each entity. Their weight vectors are dense entity representations produced by encoding entity meta information such as their descriptions. This approach has several shortcomings: (i)…
AWED-PIPER: Agents, Web Applications & Expert Detectors for Personally Identifiable Information Protection & Fine-grained Named Entity Recognition across 36 languages for 6.6 Billion Speakers
Indian Institute of Technology Guwahati（印度理工学院古瓦哈蒂分校）
Extracting Structured Requirements from Unstructured Building Technical Specifications for Building Information Modeling
LISPEN EA 7515, Arts et Métiers Institute of Technology（LISPEN EA 7515，艺术与技术理工学院）；Information System and Applications Division, CSTB（信息系统与应用部门，CSTB）；Univ. Grenoble Alpes, CNRS, Grenoble INP, G-SCOP（格勒诺布尔阿尔卑斯大学，CNRS，格勒诺布尔INP，G-SCOP）
DFKI GmbH（德国人工智能研究中心）；pc GmbH Neue Kommunikation（3pc新传播有限公司）
Previous work of ours on Semantic Storytelling uses text analytics procedures including Named Entity Recognition and Event Detection. In this paper, we outline our longer-term vision on Semantic Storytelling and describe the current conceptual and technical approach. In the project that drives our research we develop AI-based technologies that are verified by partners from industry. One long-term goal is the development of an approach for Semantic Storytelling that has broad coverage and that is, furthermore, robust. We provide first results on experiments that involve discourse parsing, appli…
Recent years showed a strong increase in biomedical sciences and an inherent increase in publication volume. Extraction of specific information from these sources requires highly sophisticated text mining and information extraction tools. However, the integration of freely available tools into customized workflows is often cumbersome and difficult. We describe SIA (Scalable Interoperable Annotation Server), our contribution to the BeCalm-Technical interoperability and performance of annotation servers (BeCalm-TIPS) task, a scalable, extensible, and robust annotation service. The system current…
Nanjing University（南京大学）；Samsung Research America（三星美国研究中心）；Kno.e.sis Center, Wright State University（莱特州立大学Kno.e.sis研究中心）
The increasing availability of semantic data has substantially enhanced Web applications. Semantic data such as RDF data is commonly represented as entity-property-value triples. The magnitude of semantic data, in particular the large number of triples describing an entity, could overload users with excessive amounts of information. This has motivated fruitful research on automated generation of summaries for entity descriptions to satisfy users' information needs efficiently and effectively. We focus on this prominent topic of entity summarization, and our research objective is to present the…
NEREL-BIO: A Dataset of Biomedical Abstracts Annotated with Nested Named Entities
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；Madan Bhandari University of Science and Technology（马丹·班达里科技大学）；Ural Federal University（乌拉尔联邦大学）；HSE University（高等经济大学）；Innopolis University（因诺波利斯大学）；A.P. Ershov Institute of Informatics Systems（A.P. 埃尔绍夫信息学系统研究所）；Artificial Intelligence Research Institute（人工智能研究所）；Sber AI
MBZUAI, UAE（阿联酋MBZUAI）；Sofia University "St. Kliment Ohridski", Bulgaria（保加利亚索菲亚大学）；Athens University of Economics and Business, Greece（希腊雅典经济与商业大学）；University of Porto, Portugal（葡萄牙波尔图大学）；University of Helsinki, Finland（芬兰赫尔辛基大学）；Indian Institute of Technology Delhi, India（印度德里印度理工学院）；University of Padova, Italy（意大利帕多瓦大学）；European Commission Joint Research Centre, Italy（意大利欧盟联合研究中心）；Institute of Computer Science, Polish Academy of Sciences, Poland（波兰科学院计算机科学研究所）
L3S Research Center and TIB, University of Hannover（汉诺威大学L3S研究中心与TIB）；Cerence GmbH and Zerotha Research（Cerence有限公司与Zerotha研究）；TIB, University of Hannover（汉诺威大学TIB）
The Natural Language Processing (NLP) community has significantly contributed to the solutions for entity and relation recognition from the text, and possibly linking them to proper matches in Knowledge Graphs (KGs). Considering Wikidata as the background KG, still, there are limited tools to link knowledge within the text to Wikidata. In this paper, we present Falcon 2.0, first joint entity, and relation linking tool over Wikidata. It receives a short natural language text in the English language and outputs a ranked list of entities and relations annotated with the proper candidates in Wikid…
We introduce TechTrack, a new dataset for tracking entities in technical procedures. The dataset, prepared by annotating open domain articles from WikiHow, consists of 1351 procedures, e.g., "How to connect a printer", identifies more than 1200 unique entities with an average of 4.7 entities per procedure. We evaluate the performance of state-of-the-art models on the entity-tracking task and find that they are well below the human annotation performance. We describe how TechTrack can be used to take forward the research on understanding procedures from temporal texts.
Everyday activities performed by artificial assistants can potentially be executed naively and dangerously given their lack of common sense knowledge. This paper presents conceptual work towards obtaining prior knowledge on the usual modality (passive or active) of any given entity, and their affordance estimates, by extracting high-confidence ability modality semantic relations (X can Y relationship) from non-figurative texts, by analyzing co-occurrence of grammatical instances of subjects and verbs, and verbs and objects. The discussion includes an outline of the concept, potential and limit…
NVIDIA（英伟达）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Text normalization (TN) and inverse text normalization (ITN) are essential preprocessing and postprocessing steps for text-to-speech synthesis and automatic speech recognition, respectively. Many methods have been proposed for either TN or ITN, ranging from weighted finite-state transducers to neural networks. Despite their impressive performance, these methods aim to tackle only one of the two tasks but not both. As a result, in a complete spoken dialog system, two separate models for TN and ITN need to be built. This heterogeneity increases the technical complexity of the system, which in tu…
Revealing the Technology Development of Natural Language Processing: A Scientific Entity-Centric Perspective
Department of Information Management, Nanjing University of Science and Technology（南京理工大学信息管理学院）；Department of Management Science and Engineering, School of Management, Anhui University（安徽大学管理科学与工程学院）
DialogPII: A multilingual dataset of synthetic dialog transcripts to detect personal information
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Technical University Berlin（柏林技术大学）；Berlin Institute of Health (BIH)（柏林健康研究所（BIH））；Tilburg University（蒂尔堡大学）；Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Independent Researcher（独立研究者）；Bundesinstitut für Risikobewertung (BfR)（风险评估联邦机构（BfR））
We present an approach to extract measured information from text (e.g., a 1370 degrees C melting point, a BMI greater than 29.9 kg/m^2 ). Such extractions are critically important across a wide range of domains - especially those involving search and exploration of scientific and technical documents. We first propose a rule-based entity extractor to mine measured quantities (i.e., a numeric value paired with a measurement unit), which supports a vast and comprehensive set of both common and obscure measurement units. Our method is highly robust and can correctly recover valid measured quantiti…
