Objective: To assess the performance of the OpenAI GPT API in accurately and efficiently identifying relevant titles and abstracts from real-world clinical review datasets and compare its performance against ground truth labelling by two independent human reviewers.
Methods: We introduce a novel workflow using the OpenAI GPT API for screening titles and abstracts in clinical reviews. A Python script was created to make calls to the GPT API with the screening criteria in natural language and a corpus of title and abstract datasets that have been filtered by a minimum of two human reviewers. We compared the performance of our model against human-reviewed papers across six review papers, screening over 24,000 titles and abstracts.
Results: Our results show an accuracy of 0.91, a sensitivity of excluded papers of 0.91, and a sensitivity of included papers of 0.76. On a randomly selected subset of papers, the GPT API demonstrated the ability to provide reasoning for its decisions and corrected its initial decision upon being asked to explain its reasoning for a subset of incorrect classifications.
Conclusion: The GPT API has the potential to streamline the clinical review process, save valuable time and effort for researchers, and contribute to the overall quality of clinical reviews. By prioritizing the workflow and acting as an aid rather than a replacement for researchers and reviewers, the GPT API can enhance efficiency and lead to more accurate and reliable conclusions in medical research.
Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practice
Euler Institute, Università della Svizzera italiana(欧拉研究所,瑞士大学);National Institute of Informatics(国家信息研究所);Brown University(布朗大学);University of Tübingen(图宾根大学);Insubria University(因斯布里亚大学)
Enhancing Hepatopathy Clinical Trial Efficiency: A Secure, Large Language Model-Powered Pre-Screening Pipeline
The First Affiliated Hospital of Guangxi University of Traditional Chinese Medicine(广西中医药大学第一附属医院);Institute of Biointellgence Technology, BGI Research(华大研究院生物智能技术研究所);Guangdong Bigdata Engineering Technology Research Center for Life Sciences, BGI Research(华大研究院广东省生命科学大数据工程技术研究中心)
MedRLM: Recursive Multimodal Health Intelligence for Long-Context Clinical Reasoning, Sensor-Guided Screening, Evidence-Grounded Decision Support, and Community-to-Tertiary Referral Optimization
School of Information, Computer;Communication Technology Sirindhorn International Institute of Technology, Thammasat University Pathum Thani, Thailand 1
Automated clinical coding using off-the-shelf large language models
Canon Medical Research Europe(佳能医疗研究欧洲);Queen Mary University of London(伦敦玛丽女王大学);University of Edinburgh(爱丁堡大学);Anglia Ruskin University(安格利亚鲁斯金大学);The Alan Turing Institute(艾伦·图灵研究所);University of Warwick(华威大学)
Structured Visual Evidence Decomposition for Evidence-Grounded Multimodal Screening of Obstructive Sleep Apnea-Hypopnea Syndrome
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science(上海工程技术大学电子与电气工程学院);Tencent Youtu Lab(腾讯云视频实验室);ENT Institute and Department of Otorhinolaryngology, Eye & ENT Hospital of Fudan University(复旦大学耳鼻喉科医院耳鼻喉科研究所);National University of Singapore(新加坡国立大学)
Accelerating Clinical Evidence Synthesis with Large Language Models
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National Center for Biotechnology Information(国家生物技术信息中心);National Library of Medicine(国家医学图书馆);Carle Illinois College of Medicine(卡尔伊利诺伊医学院)
A Scoping Review of AI-Driven Digital Interventions in Mental Health Care: Mapping Applications Across Screening, Support, Monitoring, Prevention, and Clinical Education
School of International and Public Affairs(国际与公共事务学院);Columbia University(哥伦比亚大学);Department of Psychology(心理学系);Seton Hall University(塞顿·霍尔大学)
A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice
Wuhan University(武汉大学);Union Hospital, Tongji Medical College, Huazhong University of Science and Technology(华中科技大学同济医学院附属协和医院);Hubei Provincial Clinical Research Center for Precision Radiology & Interventional Medicine(湖北省精准放射与介入医学临床医学研究中心);The First Affiliated Hospital of Zhengzhou University(郑州大学第一附属医院);Huazhong University of Science and Technology(华中科技大学);The First Affiliated Hospital of Henan University of Science and Technology(河南科技大学第一附属医院)
AeroSpectra Sentinel: An Auditable LLM Prompt-Chaining Decision-Support Workflow for Acute Asthma Risk Assessment from Respiratory Sounds and Clinical Signals
School of Information, Computer, and Communication Technology(信息、计算机与通信技术学院);Sirindhorn International Institute of Technology, Thammasat University(泰国朱拉隆梭国际技术学院)
Specialty-Oriented Generalist Medical AI for Chest CT Screening
Rensselaer Polytechnic Institute(伦斯勒理工学院);Wake Forest University School of Medicine(维克森林大学医学院);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)
Automated Clinical Coding: What, Why, and Where We Are?
University of Edinburgh(爱丁堡大学);University College London(伦敦大学学院);Edinburgh Futures Institute(爱丁堡未来学院);Epic Systems Corporation(Epic系统公司);University College London Hospitals NHS Foundation Trust(伦敦大学学院医院NHS基金会信托);Aalto University(阿尔托大学);University of Oxford(牛津大学)
Reliable Automated Triage in Spanish Clinical Notes: A Hybrid Framework for Risk-Aware HIV Suspicion Identification
Dept. of Lenguajes y Sistemas Informáticos, Escuela Técnica Superior de Ingeniería Informática, Universidad Nacional de Educación a Distancia (UNED)(语言与系统信息学系,信息工程技术学校,国家远程教育大学(UNED));Dept. Informática y Estadística, Escuela Técnica Superior de Ingeniería Informática, Universidad Rey Juan Carlos (URJC)(信息与统计学系,信息工程技术学校,皇家胡安·卡洛斯大学(URJC))
AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
Yonsei University College of Medicine(延世大学医学院);Yonsei Institute for Digital Health(延世大学数字健康研究所);Yonsei University(延世大学);Institute of Behavioral Sciences in Medicine(医学行为科学研究所)
Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation
College of Information Sciences and Technology, Pennsylvania State University(宾夕法尼亚州立大学信息科学与技术学院);School of Software Technology, Dalian University of Technology(大连理工大学软件学院);Relativity(相对论公司)
Automated SNOMED CT Concept Annotation in Clinical Text Using Bi-GRU Neural Networks
Informatics and Analytics, University of North Carolina Greensboro(信息学与分析,北卡罗来纳大学格林斯堡分校);Department of Computer Science, University of Nebraska Omaha(计算机科学系,内布拉斯加大学奥马哈分校)
NeuraHealth: An Automated Screening Pipeline to Detect Undiagnosed Cognitive Impairment in Electronic Health Records with Deep Learning and Natural Language Processing
Massachusetts General Hospital(麻省总医院);McCance Center for Brain Health(麦坎斯大脑健康中心)
痴呆相关的认知障碍(CI)是一种神经退行性疾病,影响全球超过5500万人,并以每3秒一个新病例的速度快速增长。全球75%的病例未被诊断,其中90%发生在低收入和中等收入国家,估计全球每年的成本为1.3万亿美元,预计到2030年将达到2.8万亿美元。由于无法治愈,临床试验反复失败,缺乏早期诊断,死亡率为100%。电子健康记录(EHR)中的信息可以为CI的早期检测提供重要线索,但专家的手动审查是繁琐且容易出错的。已经提出了几种计算方法,然而,他们缺乏对复杂的语言结构的EHR的语言上下文的增强的理解。因此,我提出了一个新的,更准确的框架,神经健康,以确定谁没有早期诊断的患者。在NeuraHealth中,我使用来自Mass General Brigham BioBank的患者EHR,微调了一个基于双向注意力的深度学习自然语言处理模型来对序列进行分类。序列预测用于生成结构化特征作为患者水平正则化逻辑回归模型的输入。这种两步框架创建了高维度,优于所有现有的最先进的计算方法以及临床方法。此外,我将这些模型集成到一个现实世界的产品,一个网络应用程序,以创建一个自动化的EHR筛查管道,用于可扩展和高速发现EHR中未检测到的CI,使早期诊断在医疗设施和医疗服务稀缺的地区变得可行。
Evaluating Spoken Language as a Biomarker for Automated Screening of Cognitive Impairment
Imperial College London(帝国理工学院);UK Dementia Research Institute(英国痴呆症研究所);Imperial College Healthcare NHS Trust(帝国理工学院医疗保健国民保健服务信托基金);University of Surrey(萨里大学);University of Southern California(南加州大学)