Clinical trials predicate subject eligibility on a diversity of criteria ranging from patient demographics to food allergies. Trials post their requirements as semantically complex, unstructured free-text. Formalizing trial criteria to a computer-interpretable syntax would facilitate eligibility determination. In this paper, we investigate an information extraction (IE) approach for grounding criteria from trials in ClinicalTrials(dot)gov to a shared knowledge base. We frame the problem as a novel knowledge base population task, and implement a solution combining machine learning and context free grammar. To our knowledge, this work is the first criteria extraction system to apply attention-based conditional random field architecture for named entity recognition (NER), and word2vec embedding clustering for named entity linking (NEL). We release the resources and core components of our system on GitHub at https://github.com/facebookresearch/Clinical-Trial-Parser. Finally, we report our per module and end to end performances; we conclude that our system is competitive with Criteria2Query, which we view as the current state-of-the-art in criteria extraction.
Clinical trial records are variable resources or the analysis of patients and diseases. Information extraction from free text such as eligibility criteria and summary of results and conclusions in clinical trials would better support computer-based eligibility query formulation and electronic patient screening. Previous research has focused on extracting information from eligibility criteria, with usually a single pair of medical entity and attribute, but seldom considering other kinds of free text with multiple entities, attributes and relations that are more complex for parsing. In this pape…
Electronic health records capture patient information using structured controlled vocabularies and unstructured narrative text. While structured data typically encodes lab values, encounters and medication lists, unstructured data captures the physician's interpretation of the patient's condition, prognosis, and response to therapeutic intervention. In this paper, we demonstrate that information extraction from unstructured clinical narratives is essential to most clinical applications. We perform an empirical study to validate the argument and show that structured data alone is insufficient i…
University of Washington（华盛顿大学）；Northeastern University（东北大学）；George Mason University（乔治梅森大学）
Identifying cohorts of patients based on eligibility criteria such as medical conditions, procedures, and medication use is critical to recruitment for clinical trials. Such criteria are often most naturally described in free-text, using language familiar to clinicians and researchers. In order to identify potential participants at scale, these criteria must first be translated into queries on clinical databases, which can be labor-intensive and error-prone. Natural language processing (NLP) methods offer a potential means of such conversion into database queries automatically. However they mu…
Data Science and AI, Novartis（诺华数据科学与人工智能部门）；Global Drug Development, Novartis Pharma AG（诺华制药全球药物开发部）
Natural language processing (NLP) of clinical trial documents can be useful in new trial design. Here we identify entity types relevant to clinical trial design and propose a framework called CT-BERT for information extraction from clinical trial text. We trained named entity recognition (NER) models to extract eligibility criteria entities by fine-tuning a set of pre-trained BERT models. We then compared the performance of CT-BERT with recent baseline methods including attention-based BiLSTM and Criteria2Query. The results demonstrate the superiority of CT-BERT in clinical trial NLP.
AD-CDO: A Lightweight Ontology for Representing Eligibility Criteria in Alzheimer's Disease Clinical Trials
University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Indiana University School of Medicine（印第安纳大学医学院）；Mayo Clinic in Florida（佛罗里达梅奥诊所）
Northwestern University Feinberg School of Medicine（西北大学范伯格医学院）
To reduce the large amount of time spent screening, identifying, and recruiting patients into clinical trials, we need prescreening systems that are able to automate the data extraction and decision-making tasks that are typically relegated to clinical research study coordinators. However, a major obstacle is the vast amount of patient data available as unstructured free-form text in electronic health records. Here we propose an information extraction-based approach that first automatically converts unstructured text into a structured form. The structured data are then compared against a list …
SatIR: Scalable High-Recall Constraint-Satisfaction-Based Information Retrieval for Clinical Trials Matching
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Samueli Electrical and Computer Engineering, UCLA（UCLA Samueli电气与计算机工程系）；Department of Computer Science and Informatics, Emory University（埃默里大学计算机科学与信息学系）；Mayo Clinic（梅奥诊所）
FDA has been promoting enrollment practices that could enhance the diversity of clinical trial populations, through broadening eligibility criteria. However, how to broaden eligibility remains a significant challenge. We propose an AI approach to Cohort Optimization (AICO) through transformer-based natural language processing of the eligibility criteria and evaluation of the criteria using real-world data. The method can extract common eligibility criteria variables from a large set of relevant trials and measure the generalizability of trial designs to real-world patients. It overcomes the sc…
Interventional cancer clinical trials are generally too restrictive, and some patients are often excluded on the basis of comorbidity, past or concomitant treatments, or the fact that they are over a certain age. The efficacy and safety of new treatments for patients with these characteristics are, therefore, not defined. In this work, we built a model to automatically predict whether short clinical statements were considered inclusion or exclusion criteria. We used protocols from cancer clinical trials that were available in public registries from the last 18 years to train word-embeddings, a…
COVID-19 clinical trial design is a critical task in developing therapeutics for the prevention and treatment of COVID-19. In this study, we apply a deep learning approach to extract eligibility criteria variables from COVID-19 trials to enable quantitative analysis of trial design and optimization. Specifically, we train attention-based bidirectional Long Short-Term Memory (Att-BiLSTM) models and use the optimal model to extract entities (i.e., variables) from the eligibility criteria of COVID-19 trials. We compare the performance of Att-BiLSTM with traditional ontology-based method. The resu…
Using Large Language Models to Generate Clinical Trial Tables and Figures
Novartis Pharmaceuticals Corporation（诺华制药公司）；School of Biomedical Informatics The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心生物医学信息学院）；Novartis Pharma AG（诺华制药公司）；The University of Texas MD Anderson Cancer Center（德克萨斯大学 MD 安德森癌症中心）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Clinical trials are essential for drug development but are extremely expensive and time-consuming to conduct. It is beneficial to study similar historical trials when designing a clinical trial. However, lengthy trial documents and lack of labeled data make trial similarity search difficult. We propose a zero-shot clinical trial retrieval method, Trial2Vec, which learns through self-supervision without annotating similar clinical trials. Specifically, the meta-structure of trial documents (e.g., title, eligibility criteria, target disease) along with clinical knowledge (e.g., UMLS knowledge ba…
The n2c2 2018 Challenge task 1 aimed to identify patients who meet lists of heterogeneous inclusion/exclusion criteria for a hypothetical clinical trial. We demonstrate a generic rule-based natural language pipeline can support this task with decent performance (the average F1 score on the test set is 0.89, ranked the 8th out of 45 teams ).
School of Medicine, Tsinghua University（清华大学医学院）；Alibaba DAMO Academy（阿里巴巴达摩院）；Alibaba Group（阿里巴巴集团）；Indiana University Bloomington（印第安纳大学伯明顿分校）
Clinical trials provide essential guidance for practicing Evidence-Based Medicine, though often accompanying with unendurable costs and risks. To optimize the design of clinical trials, we introduce a novel Clinical Trial Result Prediction (CTRP) task. In the CTRP framework, a model takes a PICO-formatted clinical trial proposal with its background as input and predicts the result, i.e. how the Intervention group compares with the Comparison group in terms of the measured Outcome in the studied Population. While structured clinical evidence is prohibitively expensive for manual collection, we …
The University of Melbourne（墨尔本大学）；RMIT University（皇家墨尔本理工大学）
This paper describes the submissions of the Natural Language Processing (NLP) team from the Australian Research Council Industrial Transformation Training Centre (ITTC) for Cognitive Computing in Medical Technologies to the TREC 2021 Clinical Trials Track. The task focuses on the problem of matching eligible clinical trials to topics constituting a summary of a patient's admission notes. We explore different ways of representing trials and topics using NLP techniques, and then use a common retrieval model to generate the ranked list of relevant trials for each topic. The results from all our s…
Retrieval-Reasoning Large Language Model-based Synthetic Clinical Trial Generation
Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎- Rocquencourt 国家信息与自动化研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒实验室）；University of Chicago（芝加哥大学）；Stanford University（斯坦福大学）；Purdue University（普渡大学）；University of Southern California（南加州大学）
Real-world validation of a multimodal LLM-powered pipeline for High-Accuracy Clinical Trial Patient Matching leveraging EHR data
Inato（伊纳托（公司名））；Center for Research in Epidemiology and Statistics (CRESS)（流行病学与统计学研究中心）；Université Paris Cité（巴黎城市大学）；Université Sorbonne Paris Nord（巴黎北索邦大学）；INSERM（法国国家健康与医学研究院）；INRAE（法国国家农业、食品与环境研究院）；AP-HP（巴黎公共医疗集团）；Hôpital Hôtel Dieu（主宫医院）
End-To-End Clinical Trial Matching with Large Language Models
National Center for Tumor Diseases (NCT), Heidelberg University Hospital（海德堡大学医院国家肿瘤疾病中心（NCT））；Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿工业大学Else Kroener Fresenius数字健康中心）；Heidelberg University（海德堡大学）；University Hospital Dresden（德累斯顿大学医院）；University Hospital RWTH Aachen（亚琛工业大学附属医学院）；University of St Andrews（圣安德鲁斯大学）；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））；Google Cloud（谷歌云）；National Center for Tumor Diseases (NCT/UCC), Dresden（德累斯顿国家肿瘤疾病中心（NCT/UCC））；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；TUD Dresden University of Technology（德累斯顿工业大学（TUD））；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心（HZDR））；Gustave Roussy（古斯塔夫·鲁西癌症中心）；Paris Saclay University（巴黎-萨克雷大学）；University Hospital Aache（亚琛大学医院）
Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology
Microsoft Research（微软研究院）；Providence Health & Services（普罗维登斯医疗服务机构）；Earle A. Chiles Research Institute（厄尔·A·奇尔斯研究所）；Providence Cancer Institute（普罗维登斯癌症研究所）
Georgia Institute of Technology（佐治亚理工学院）；Booz Allen Hamilton（博思艾伦汉密尔顿公司）
Due to increased access to clinical trial outcomes and analysis, researchers and scientists are able to iterate or improve upon relevant approaches more effectively. However, the metrics and related results of clinical trials typically do not follow any standardization in their reports, making it more difficult for researchers to parse the results of different trials. The objective of this paper is to describe an automated method utilizing natural language processing in order to describe the probable core outcomes of clinical trials, in order to alleviate the issues around disparate clinical t…
Clinical trials offer a fundamental opportunity to discover new treatments and advance the medical knowledge. However, the uncertainty of the outcome of a trial can lead to unforeseen costs and setbacks. In this study, we propose a new method to predict the effectiveness of an intervention in a clinical trial. Our method relies on generating an informative summary from multiple documents available in the literature about the intervention under study. Specifically, our method first gathers all the abstracts of PubMed articles related to the intervention. Then, an evidence sentence, which convey…
PRISM: Patient Records Interpretation for Semantic Clinical Trial Matching using Large Language Models
Triomics Research（Triomics研究院）；Medical College of Wisconsin（威斯康星医学院）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Cancer Informatics For Cancer Centers（癌症中心癌症信息学机构）；University of Pittsburgh（匹兹堡大学）
Language Interaction Network for Clinical Trial Approval Estimation
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Carle Illinois College of Medicine, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校卡尔伊利诺伊医学院）
Recommending Clinical Trials for Online Patient Cases using Artificial Intelligence
National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）；National Cancer Institute, National Institutes of Health（美国国立卫生研究院国家癌症研究所）
TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model
School of Public Health, Zhejiang University（浙江大学公共卫生学院）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
University of Wisconsin School of Medicine and Public Health（威斯康星大学医学院与公共卫生学院）
In the past decade, there has been much discussion about the issue of biased reporting in clinical research. Despite this attention, there have been limited tools developed for the systematic assessment of qualitative statements made in clinical research, with most studies assessing qualitative statements relying on the use of manual expert raters, which limits their size. Also, previous attempts to develop larger scale tools, such as those using natural language processing, were limited by both their accuracy and the number of categories used for the classification of their findings. With the…
