Accepted for Oral Presentation at LOUHI 2016 : EMNLP 2016 Workshop - The Seventh International Workshop on Health Text Mining and Information Analysis (LOUHI 2016)
Drug name recognition (DNR) is an essential step in the Pharmacovigilance (PV) pipeline. DNR aims to find drug name mentions in unstructured biomedical texts and classify them into predefined categories. State-of-the-art DNR approaches heavily rely on hand crafted features and domain specific resources which are difficult to collect and tune. For this reason, this paper investigates the effectiveness of contemporary recurrent neural architectures - the Elman and Jordan networks and the bidirectional LSTM with CRF decoding - at performing DNR straight from the text. The experimental results achieved on the authoritative SemEval-2013 Task 9.1 benchmarks show that the bidirectional LSTM-CRF ranks closely to highly-dedicated, hand-crafted systems.
GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries
School of Computer Science, The University of Manchester（曼彻斯特大学计算机学院）；Health eResearch Center, The University of Manchester（曼彻斯特大学健康电子研究中心）；The Alan Turing Institute（艾伦·图灵研究所）
CHU Necker, AP-HP（巴黎公共医疗救助机构内克尔大学医院）；INSERM UMR-S 872（法国国家健康与医学研究院 UMR-S 872）；Sorbonne Paris Cité, Faculté de Médecine（索邦巴黎西岱大学医学院）；Université de Paris（巴黎大学）
In this study, we evaluated the RNNG, a neural top-down transition based parser, for medication information extraction in clinical texts. We evaluated this model on a French clinical corpus. The task was to extract the name of a drug (or a drug class), as well as attributes informing its administration: frequency, dosage, duration, condition and route of administration. We compared the RNNG model that jointly identifies entities, events and their relations with separate BiLSTMs models for entities, events and relations as baselines. We call seq-BiLSTMs the baseline models for relations extract…
Do "New Snow Tablets" Contain Snow? Large Language Models Over-Rely on Names to Identify Ingredients of Chinese Drugs
Liaoning University（辽宁大学）；University of Queensland（昆士兰大学）；National University of Singapore（新加坡国立大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of California, Merced（加利福尼亚大学默塞德分校）
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
MIT（麻省理工学院）；Harvard（哈佛大学）；Mass General Brigham（麻省总医院布里格姆医疗系统）；Universitat Pompeu Fabra（庞培法布拉大学）；Aarhus University（奥胡斯大学）；Maastricht University（马斯特里赫特大学）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；University of Virginia（弗吉尼亚大学）
Terminologies augmented recurrent neural network model for clinical named entity recognition
Paris University（巴黎大学）；AP-HP（巴黎公共援助医院）；LIMSI（法国计算机科学与自动化研究所）；CNRS（法国国家科学研究中心）；Univ. Paris-Sud（巴黎南大学）；Université Paris-Saclay（巴黎萨克雷大学）；Sorbonne Université（索邦大学）；Inserm（法国国家健康与医学研究院）；Univ Paris 13（巴黎第十三大学）；LIMICS（医学信息学与电子健康知识工程实验室）
我们的目的是提高性能的监督模型的临床命名实体识别（NER）使用医学术语。为了评估我们的系统在法国，我们建立了一个语料库的5种类型的临床实体。我们使用了一个基于术语的系统作为基线，建立在UMLS和SNOMED。然后，我们评估了一个biGRU-CRF，和一个混合系统使用的预测的术语为基础的系统的biGRU-CRF的功能。在英语中，我们在i2 b2 -2009药物名称识别的药物挑战赛上评估了NER系统，其中包含268个文档的8，573个实体。在法语中，我们建立了APcNER，这是一个包含147个文档的语料库，注释了5个实体（药物名称，体征或症状，疾病或病症，诊断程序或实验室测试和治疗程序）。我们评估每个NER系统使用精确和部分匹配定义的F-测度的NER。APcNER包含4，837个实体，需要28小时进行注释，注释者间一致性对于精确匹配的药物名称是可接受的（85%），对于非精确匹配的其他实体类型是可接受的（>70%）。对于i2 b2 -2009和APcNER上的药物名称识别，biGRU-CRF比基于术语的系统表现更好，精确匹配F测量值分别为91.1%和73%，81.9%和75%。此外，混合系统优于biGRU-CRF，精确匹配F测量值为92.2% vs 91.1%（i2 b2 -2009）和88.4% vs 81.9%（APcNER）。在APcNER语料库上，混合系统对5个实体的微平均F-测…
In Track-1 of the BioCreative VII Challenge participants are asked to identify interactions between drugs/chemicals and proteins. In-context named entity annotations for each drug/chemical and protein are provided and one of fourteen different interactions must be automatically predicted. For this relation extraction task, we attempt both a BERT-based sentence classification approach, and a more novel text-to-text approach using a T5 model. We find that larger BERT-based models perform better in general, with our BioMegatron-based model achieving the highest scores across all metrics, achievin…
Kentucky Injury Prevention and Research Center（肯塔基伤害预防与研究中心）；University of Kentucky（肯塔基大学）；Center on Drug and Alcohol Research（药物与酒精研究中心）
Surveillance of drug overdose deaths relies on death certificates for identification of the substances that caused death. Drugs and drug classes can be identified through the International Classification of Diseases, 10th Revision (ICD-10) codes present on death certificates. However, ICD-10 codes do not always provide high levels of specificity in drug identification. To achieve more fine-grained identification of substances on a death certificate, the free-text cause of death section, completed by the medical certifier, must be analyzed. Current methods for analyzing free-text death certific…
To minimize the accelerating amount of time invested in the biomedical literature search, numerous approaches for automated knowledge extraction have been proposed. Relation extraction is one such task where semantic relations between the entities are identified from the free text. In the biomedical domain, extraction of regulatory pathways, metabolic processes, adverse drug reaction or disease models necessitates knowledge from the individual relations, for example, physical or regulatory interactions between genes, proteins, drugs, chemical, disease or phenotype. In this paper, we study the …
Concept normalization in free-form texts is a crucial step in every text-mining pipeline. Neural architectures based on Bidirectional Encoder Representations from Transformers (BERT) have achieved state-of-the-art results in the biomedical domain. In the context of drug discovery and development, clinical trials are necessary to establish the efficacy and safety of drugs. We investigate the effectiveness of transferring concept normalization from the general biomedical domain to the clinical trials domain in a zero-shot setting with an absence of labeled data. We propose a simple and effective…
Truecasing is the task of restoring the correct case (uppercase or lowercase) of noisy text generated either by an automatic system for speech recognition or machine translation or by humans. It improves the performance of downstream NLP tasks such as named entity recognition and language modeling. We propose a fast, accurate and compact two-level hierarchical word-and-character-based recurrent neural network model, the first of its kind for this problem. Using sequence distillation, we also address the problem of truecasing while ignoring token positions in the sentence, i.e. in a position-in…
Current approaches for clinical information extraction are inefficient in terms of computational costs and memory consumption, hindering their application to process large-scale electronic health records (EHRs). We propose an efficient end-to-end model, the Joint-NER-RE-Fourier (JNRF), to jointly learn the tasks of named entity recognition and relation extraction for documents of variable length. The architecture uses positional encoding and unitary batch sizes to process variable length documents and uses a weight-shared Fourier network layer for low-complexity token mixing. Finally, we reach…
Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contexts
University of Ghent（根特大学）；University of Udine（乌迪内大学）
Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search. We link this problem to the classic problem of Chinese word segmentation and show the effectiveness of a tagging model based on Recurrent Neural Networks (RNNs) using characters as input. To compensate for the lack of training data, we propose a pre-training method on concatenated entity names in a large knowledge database. Pre-training improves the model by 33% and brings the sequence accuracy to 85%.
University of Chicago（芝加哥大学）；Argonne National Laboratory（阿贡国家实验室）；Globus
Researchers worldwide are seeking to repurpose existing drugs or discover new drugs to counter the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A promising source of candidates for such studies is molecules that have been reported in the scientific literature to be drug-like in the context of coronavirus research. We report here on a project that leverages both human and artificial intelligence to detect references to drug-like molecules in free text. We engage non-expert humans to create a corpus of labeled text, use this labeled corpus to train a named enti…
Data Analytics Laboratory (DAL)（数据分析实验室（DAL））；Sharif University of Technology（谢里夫理工大学）
In this paper, we extend a class of celebrated regularization techniques originally proposed for feed-forward neural networks, namely Input Mixup (Zhang et al., 2017) and Manifold Mixup (Verma et al., 2018), to the realm of Recurrent Neural Networks (RNN). Our proposed methods are easy to implement and have a low computational complexity, while leverage the performance of simple neural architectures in a variety of tasks. We have validated our claims through several experiments on real-world datasets, and also provide an asymptotic theoretical analysis to further investigate the properties and…
Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Clinical Cohort Studies (CCS), such as randomized clinical trials, are a great source of documented clinical research. Ideally, a clinical expert inspects these articles for exploratory analysis ranging from drug discovery for evaluating the efficacy of existing drugs in tackling emerging diseases to the first test of newly developed drugs. However, more than 100 articles are published daily on a single prevalent disease like COVID-19 in PubMed. As a result, it can take days for a physician to find articles and extract relevant information. Can we develop a system to sift through the long list…
Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics of a variable is defined not only by its name but also by the contexts in which the variable occurs. In this work, we develop dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of the variable when it obtains more information about the variable's role in the program. We show that using the proposed dynamic embeddings significantly …
Avignon Université（阿维尼翁大学）；University of Cambridge（剑桥大学）
The light gated recurrent units (Li-GRU) is well-known for achieving impressive results in automatic speech recognition (ASR) tasks while being lighter and faster to train than a standard gated recurrent units (GRU). However, the unbounded nature of its rectified linear unit on the candidate recurrent gate induces an important gradient exploding phenomenon disrupting the training process and preventing it from being applied to famous datasets. In this paper, we theoretically and empirically derive the necessary conditions for its stability as well as engineering mechanisms to speed up by a fac…
