Background and Objectives: Clinical Practice Guidelines (CPGs) represent the foremost methodology for sharing state-of-the-art research findings in the healthcare domain with medical practitioners to limit practice variations, reduce clinical cost, improve the quality of care, and provide evidence based treatment. However, extracting relevant knowledge from the plethora of CPGs is not feasible for already burdened healthcare professionals, leading to large gaps between clinical findings and real practices. It is therefore imperative that state-of-the-art Computing research, especially machine learning is used to provide artificial intelligence based solution for extracting the knowledge from CPGs and reducing the gap between healthcare research/guidelines and practice. Methods: This research presents a novel methodology for knowledge extraction from CPGs to reduce the gap and turn the latest research findings into clinical practice. First, our system classifies the CPG sentences into four classes such as condition-action, condition-consequences, action, and not-applicable based on the information presented in a sentence. We use deep learning with state-of-the-art word embedding, improved word vectors technique in classification process. Second, it identifies qualifier terms in the classified sentences, which assist in recognizing the condition and action phrases in a sentence. Finally, the condition and action phrase are processed and transformed into plain rule If Condition(s) Then Action format. Results: We evaluate the methodology on three different domains guidelines including Hypertension, Rhinosinusitis, and Asthma. The deep learning model classifies the CPG sentences with an accuracy of 95%. While rule extraction was validated by user-centric approach, which achieved a Jaccard coefficient of 0.6, 0.7, and 0.4 with three human experts extracted rules, respectively.
A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations
Microsoft Research Asia（微软亚洲研究院）；Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；University of Pittsburgh Medical Center Western Maryland（匹兹堡大学医学中心西马里兰分院）
Imperial College London（伦敦帝国理工学院）；National Institute of Informatics（国立信息学研究所）
Automatically reasoning with conflicting generic clinical guidelines is a burning issue in patient-centric medical reasoning where patient-specific conditions and goals need to be taken into account. It is even more challenging in the presence of preferences such as patient's wishes and clinician's priorities over goals. We advance a structured argumentation formalism for reasoning with conflicting clinical guidelines, patient-specific information and preferences. Our formalism integrates assumption-based reasoning and goal-driven selection among reasoning outcomes. Specifically, we assume app…
Simon Fraser University（西蒙弗雷泽大学）；The University of British Columbia（不列颠哥伦比亚大学）
Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1)…
MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Albert Einstein College of Medicine（爱因斯坦医学院）
Institute for Computing and Information Sciences（计算与信息科学研究所）；Radboud University Nijmegen（拉德堡德大学奈梅亨分校）
Requirements about the quality of clinical guidelines can be represented by schemata borrowed from the theory of abductive diagnosis, using temporal logic to model the time-oriented aspects expressed in a guideline. Previously, we have shown that these requirements can be verified using interactive theorem proving techniques. In this paper, we investigate how this approach can be mapped to the facilities of a resolution-based theorem prover, Otter, and a complementary program that searches for finite models of first-order statements, Mace. It is shown that the reasoning required for checking t…
Université Paris 13（巴黎第十三大学）；Société de Formation Thérapeutique du Généraliste (SFTG)（全科医生治疗培训学会（SFTG））
Well-designed medical decision support system (DSS) have been shown to improve health care quality. However, before they can be used in real clinical situations, these systems must be extensively tested, to ensure that they conform to the clinical guidelines (CG) on which they are based. Existing methods cannot be used for the systematic testing of all possible test cases. We describe here a new exhaustive dynamic verification method. In this method, the DSS is considered to be a black box, and the Quinlan C4.5 algorithm is used to build a decision tree from an exhaustive set of DSS input vect…
From Continuous Predictors to Clinical Thresholds: Early Evidence on Performance Trade-offs of Guideline-Based Categorisation for Ischaemic Stroke Outcome Prediction
Technological University Dublin（都柏林理工大学）；Aalborg University（奥尔堡大学）；Health Management Institute（健康管理研究所）；Trinity College Dublin（都柏林圣三一学院）
Synthesized Annotation Guidelines are Knowledge-Lite Boosters for Clinical Information Extraction
McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心麦克威廉姆斯生物医学信息学院）；School of Dentistry, University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心牙科学院）；Enterprise Development and Integration, University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心企业开发与集成部）；Virginia Tech Carilion School of Medicine, Department of Health Systems & Implementation Science（弗吉尼亚理工大学卡里利恩医学院卫生系统与实施科学系）
Clinicians often do not sufficiently adhere to evidence-based clinical guidelines in a manner sensitive to the context of each patient. It is important to detect such deviations, typically including redundant or missing actions, even when the detection is performed retrospectively, so as to inform both the attending clinician and policy makers. Furthermore, it would be beneficial to detect such deviations in a manner proportional to the level of the deviation, and not to simply use arbitrary cut-off values. In this study, we introduce a new approach for automated guideline-based quality assess…
University of Trier（特里尔大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；University Hospital Münster（明斯特大学医院）；Heinrich Heine University（海因里希·海涅大学）
Conformance checking is a process mining technique that allows verifying the conformance of process instances to a given model. Thus, this technique is predestined to be used in the medical context for the comparison of treatment cases with clinical guidelines. However, medical processes are highly variable, highly dynamic, and complex. This makes the use of imperative conformance checking approaches in the medical domain difficult. Studies show that declarative approaches can better address these characteristics. However, none of the approaches has yet gained practical acceptance. Another cha…
LLM-Orchestrated Conformance Checking in Stroke Care Without Computer-Interpretable Guidelines
Computer Science Institute, DiSIT, University of Piemonte Orientale（皮埃蒙特东方大学计算机科学研究所）；Integrated Laboratory of AI and Medical Informatics, DAIRI, SS. Antonio e Biagio e Cesare Arrigo Hospital（圣安东尼奥、比亚焦与切萨雷·阿里戈医院DAIRI人工智能与医学信息学综合实验室）
Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines
Institute of Health Informatics, University College London（健康信息学研究所，伦敦大学学院）；Department of Biostatistics and Health Informatics, King’s College London（生物统计学与健康信息学系，伦敦国王学院）；EPSRC DRIVE-Health CDT, London, U.K.（EPSRC DRIVE-Health CDT，伦敦，英国）；Imperial College Healthcare NHS Trust（帝国学院医疗 NHS 信托）；Black Country Healthcare NHS Foundation Trust（黑斯廷斯地区医疗 NHS 基础信托）；Feldon Lane Surgery, The Dudley Group NHS Foundation Trust（费尔顿车道诊所，德比集团 NHS 基础信托）；The Cleveland Clinic, London, U.K.（克利夫兰诊所，伦敦，英国）；CogStack Limited（CogStack 有限公司）
Streamlining evidence based clinical recommendations with large language models
Engineering Research Center of EMR and Intelligent Expert System, Ministry of Education（教育部电磁辐射与智能专家系统工程研究中心）；National Clinical Research Center for Dermatologic and Immunologic Diseases (NCRC-DID), Ministry of Science and Technology（皮肤病与免疫性疾病临床研究中心，科学技术部）；State Key Laboratory of Complex Severe and Rare Diseases（复杂严重罕见疾病国家重点实验室）；Key Laboratory of Rheumatology and Clinical Immunology, Ministry of Education（风湿免疫学重点实验室，教育部）
Impact on clinical guideline adherence of Orient-COVID, a CDSS based on dynamic medical decision trees for COVID19 management: a randomized simulation trial
Lebanese Hospital Geitaoui-UMC（黎巴嫩盖塔乌伊医院大学医学中心）；INSERM（法国国家健康与医学研究院）；Université Sorbonne Paris Nord（巴黎第十三大学）；Sorbonne Université（索邦大学）；Université Paris Cité（巴黎西岱大学）；Centre de Recherche des Cordeliers（科德利埃研究所）；AP-HP（巴黎公共医疗集团）；Hôpital Européen Georges-Pompidou（欧洲乔治·蓬皮杜医院）
A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for Depression
National Institute of Technology Kurukshetra（库鲁克谢特拉国家技术学院）；Artificial Intelligence Institute at the University of South Carolina（南卡罗来纳大学人工智能研究所）；University of Maryland, Baltimore County（马里兰大学巴尔的摩分校）；Wright State University（莱特州立大学）
Propagating construction-time knowledge quality into medical question answering: A framework grounded in clinical guidelines
Nanjing Medical University（南京医科大学）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）；The Friendship Hospital of Ili Kazakh Autonomous Prefecture（伊犁哈萨克自治州友谊医院）；Nanjing University（南京大学）
CPGPrompt: Translating Clinical Guidelines into LLM-Executable Decision Support
Information Science (Health Tech), Cornell Tech, New York, NY, USA（信息科学（健康科技），康奈尔科技，纽约，纽约州）；Systems Engineering, Cornell University, Ithaca, NY, USA（系统工程，康奈尔大学，伊萨卡，纽约州）；Population Health Sciences, Weill Cornell Medicine, New York, NY, USA（人口健康科学，韦尔医学院，纽约，纽约州）；Computer and Information Science, Cornell University, Ithaca, NY, USA（计算机与信息科学，康奈尔大学，伊萨卡，纽约州）；Department of Biomedical Informatics, Columbia University, New York, NY, USA（生物医学信息学系，哥伦比亚大学，纽约，纽约州）；Department of Medicine, Weill Cornell Medicine, New York, NY, USA（医学系，韦尔医学院，纽约，纽约州）；Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA, USA（健康信息管理系，匹兹堡大学，匹兹堡，宾夕法尼亚州）；Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, USA（临床与转化科学研究所，匹兹堡大学，匹兹堡，宾夕法尼亚州）；Department of Neurology, UT Southwestern Medical Center, Dallas, TX, USA（神经病学系，德克萨斯西南医学中心，达拉斯，德克萨斯州）；Peter O’Donnell Jr. Brain Institute, UT Southwestern Medical Center, Dallas, TX, USA（彼得·奥·donnell Jr.脑研究所，德克萨斯西南医学中心，达拉斯，德克萨斯州）；Clinical Informatics Center, University of Texas Southwestern Medical Center, Dallas, USA（临床信息学中心，德克萨斯西南医学中心，达拉斯，美国）；Institute of Artificial Intelligence for Digital Health, Weill Cornell Medicine, New York, NY USA（数字健康人工智能研究所，韦尔医学院，纽约，纽约州）
Ben Gurion University（本·古里安大学）；Rambam Health Care Campus（拉姆巴姆医疗中心）；Technion - Israel Institute of Technology（以色列理工学院）；Clalit Health Care Services（克拉利特医疗服务机构）
We evaluated the DiscovErr system, in which we had previously implemented a new methodology for assessment of compliance to continuous application of evidence-based clinical guidelines, based on a bidirectional search from the guideline objectives to the patient's longitudinal data, and vice versa. We compared the system comments on 1584 transactions regarding the management, over a mean of 5.23 years, of 10 randomly selected Type 2 diabetes patients, to those of two diabetes experts and a senior family practitioner. After providing their own comments, the experts assessed both the correctness…
Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia
Inria Paris（法国国家信息与自动化研究所巴黎分部）；Centre de Recherche des Cordeliers（科尔德里耶研究中心）；Inserm（法国国家健康与医学研究院）；Université Paris Cité（巴黎城市大学）；Sorbonne Université（索邦大学）
Laboratoire d’Informatique de Paris-Nord（巴黎北方信息学实验室）；Université Paris 13（巴黎第十三大学）；CNRS（法国国家科学研究中心）
This paper describes a system capable of semi-automatically filling an XML template from free texts in the clinical domain (practice guidelines). The XML template includes semantic information not explicitly encoded in the text (pairs of conditions and actions/recommendations). Therefore, there is a need to compute the exact scope of conditions over text sequences expressing the required actions. We present a system developed for this task. We show that it yields good performance when applied to the analysis of French practice guidelines.
This paper describes a system capable of semi-automatically filling an XML template from free texts in the clinical domain (practice guidelines). The XML template includes semantic information not explicitly encoded in the text (pairs of conditions and actions/recommendations). Therefore, there is a need to compute the exact scope of conditions over text sequences expressing the required actions. We present in this paper the rules developed for this task. We show that the system yields good performance when applied to the analysis of French practice guidelines.
A collective capability boundary in frontier large language models on guideline-conformant and case-specific oncology decision-making
City University of Hong Kong（香港城市大学）；Fudan University Shanghai Cancer Center（复旦大学附属肿瘤医院）；Affiliated Hangzhou First People’s Hospital, Westlake University School of Medicine（西湖大学医学院附属杭州市第一人民医院）；Zhejiang University（浙江大学）；Shaoxing Vocational & Technical College（绍兴职业技术学院）
AI in radiological imaging of soft-tissue and bone tumours: a systematic review evaluating against CLAIM and FUTURE-AI guidelines
Erasmus MC Cancer Institute, University Medical Center Rotterdam（伊拉斯姆斯医学中心癌症研究所，鹿特丹大学医学中心）；Leeds Teaching Hospitals NHS Trust（利兹教学医院 NHS 信托基金）；Leeds Biomedical Research Centre, University of Leeds（利兹大学生物医学研究中心）；Leeds Institute of Medical Research, University of Leeds（利兹大学医学研究所）；Amsterdam UMC（阿姆斯特丹大学医学中心）；Leiden University Medical Center（莱顿大学医学中心）；Faculty of Medical Sciences, University of Groningen（格罗宁根大学医学科学学院）
Université Paris 13（巴黎第十三大学）；AP-HP（巴黎公立医院集团）；INSERM（法国国家健康与医学研究院）；Silk Informatique（Silk Informatique公司）；Université Paris Descartes（巴黎笛卡尔大学）；Société de Formation Thérapeutique du Généraliste（全科医生治疗培训协会）
Clinical decision support systems have been developed to help physicians to take clinical guidelines into account during consultations. The ASTI critiquing module is one such systems; it provides the physician with automatic criticisms when a drug prescription does not follow the guidelines. It was initially developed for hypertension and type 2 diabetes, but is designed to be generic enough for application to all chronic diseases. We present here the results of usability and satisfaction evaluations for the ASTI critiquing module, obtained with GPs for a newly implemented guideline concerning…
Foundation Models in Biomedical Imaging: Turning Hype into Reality
Department of Imaging Physics, The University of Texas MD Anderson Cancer Center（影像物理系，德克萨斯大学MD安德森癌症中心）；Center for Secure Artificial Intelligence for Healthcare (SAFE), McWilliams School of Biomedical Informatics, UTHealth Houston（安全人工智能用于医疗保健中心（SAFE），麦威廉斯生物医学信息学学院，UTHealth休斯顿）；Female Medicine in Machine Learning, Massachusetts Institute of Technology（机器学习中的女性医学，麻省理工学院）；Department of Computer Science, Salim Habib University（计算机科学系，Salim Habib大学）；School of Computer Science and Digital Technologies, Aston Centre for Artificial Intelligence Research and Application, Aston University（计算机科学与数字技术学院，阿斯顿人工智能研究与应用中心，阿斯顿大学）；Division of Computing Science and Mathematics, University of Stirling（计算科学与数学系，斯特灵大学）；School of Medicine and Health Sciences, George Washington University（医学与健康科学学院，乔治·华盛顿大学）；Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital（谢赫扎耶德儿童外科创新研究所，儿童医院）；Department of Thoracic/Head and Neck Medical Oncology, The University of Texas MD Anderson Cancer Center（胸腔/头颈医学肿瘤科，德克萨斯大学MD安德森癌症中心）
AD-CARE: A Guideline-grounded, Modality-agnostic LLM Agent for Real-world Alzheimer's Disease Diagnosis with Multi-cohort Assessment, Fairness Analysis, and Reader Study
The Hong Kong Polytechnic University（香港理工大学）；Xuanwu Hospital, Capital Medical University（首都医科大学宣武医院）；Amazon（亚马逊）；Zhejiang University（浙江大学）；Sany AI（三一人工智能）；Yantai Yuhuangding Hospital, Qingdao University（青岛大学烟台毓璜顶医院）；The University of Hong Kong（香港大学）；The Third Affiliated Hospital of Sun Yat-sen University（中山大学附属第三医院）
Management of chronic diseases such as heart failure, diabetes, and chronic obstructive pulmonary disease (COPD) is a major problem in health care. A standard approach that the medical community has devised to manage widely prevalent chronic diseases such as chronic heart failure (CHF) is to have a committee of experts develop guidelines that all physicians should follow. These guidelines typically consist of a series of complex rules that make recommendations based on a patient's information. Due to their complexity, often the guidelines are either ignored or not complied with at all, which c…
Demo: Guide-RAG: Evidence-Driven Corpus Curation for Retrieval-Augmented Generation in Long COVID
Department of Computer Science University of Texas at Austin（计算机科学系得克萨斯大学奥斯汀分校）；School of Information University of Texas at Austin（信息学院得克萨斯大学奥斯汀分校）；McCombs School of Business University of Texas at Austin（麦库姆斯商学院得克萨斯大学奥斯汀分校）；Dell Medical School University of Texas at Austin（德克萨斯大学奥斯汀分校戴尔医学学院）
RWTH Aachen University（亚琛工业大学）；University of Pisa（比萨大学）；RWTH University Hospital（亚琛工业大学医院）
The discipline of process mining has a solid track record of successful applications to the healthcare domain. Within such research space, we conducted a case study related to the Intensive Care Unit (ICU) ward of the Uniklinik Aachen hospital in Germany. The aim of this work is twofold: developing a normative model representing the clinical guidelines for the treatment of COVID-19 patients, and analyzing the adherence of the observed behavior (recorded in the information system of the hospital) to such guidelines. We show that, through conformance checking techniques, it is possible to analyz…
Université Sorbonne Paris Nord（索邦巴黎西岱大学）；Sorbonne Université（索邦大学）；INSERM（法国国家健康与医学研究院）；Laboratoire de Recherche en Informatique, CNRS/Université Paris-Sud/Université Paris-Saclay（计算机研究实验室，法国国家科学研究中心/巴黎南大学/巴黎萨克雷大学）
Clinical trials are the basis of Evidence-Based Medicine. Trial results are reviewed by experts and consensus panels for producing meta-analyses and clinical practice guidelines. However, reviewing these results is a long and tedious task, hence the meta-analyses and guidelines are not updated each time a new trial is published. Moreover, the independence of experts may be difficult to appraise. On the contrary, in many other domains, including medical risk analysis, the advent of data science, big data and visual analytics allowed moving from expert-based to fact-based knowledge. Since 12 yea…
University of Leeds（利兹大学）；Leeds Teaching Hospitals NHS Trust（利兹教学医院NHS信托）；Linköping University（林雪平大学）
The application of new artificial intelligence (AI) discoveries is transforming healthcare research. However, the standards of reporting are variable in this still evolving field, leading to potential research waste. The aim of this work is to highlight resources and reporting guidelines available to researchers working in computational pathology. The EQUATOR Network library of reporting guidelines and extensions was systematically searched up to August 2022 to identify applicable resources. Inclusion and exclusion criteria were used and guidance was screened for utility at different stages of…
Clinical Reasoning AI for Oncology Treatment Planning: A Multi-Specialty Case-Based Evaluation
Moffitt Cancer Center and Research Institute（莫菲特癌症中心与研究学院）；Innova Montréal Inc.（蒙特利尔创新公司）；Oncobrain, Inc.（Oncobrain公司）；Advanced Cancer Treatment Centers（先进癌症治疗中心）
