评估生成式AI作为放射科住院医师报告起草的教育工具
Evaluating Generative AI as an Educational Tool for Radiology Resident Report Drafting
AI总结:
本研究评估了符合HIPAA的GPT-4o系统在真实临床环境中对放射科住院医师起草的乳腺影像报告提供自动反馈的能力,发现其能可靠识别关键教育错误且反馈具有高帮助率,可作为支持放射科教育的可扩展工具。
摘要:
目的:放射科住院医师需要及时、个性化的反馈,以培养准确的图像分析和报告技能。日益增加的临床工作量通常限制了主治医师提供指导的能力。本研究评估了一个符合HIPAA的GPT-4o系统,该系统在真实临床环境中对住院医师起草的乳腺影像报告提供自动反馈。方法:我们分析了来自美国某多站点医疗系统常规实践中的5000对住院医师-主治医师报告。GPT-4o被输入临床指令以识别常见错误并提供反馈。进行了一项使用100对报告的阅读研究。四名主治放射科医师和四名住院医师独立审查每对报告,确定是否存在预定义的错误类型,并将GPT-4o的反馈评为有帮助或无帮助。使用百分比匹配评估GPT与读者之间的一致性。使用Krippendorff's alpha测量读者间信度。教育价值以被评为有帮助的病例比例来衡量。结果:识别出三种常见错误类型:(1)遗漏或添加关键发现,(2)技术描述符的不正确使用或遗漏,以及(3)最终评估与发现不一致。GPT-4o与主治医师共识显示出高度一致性:各错误类型分别为90.5%、78.3%和90.4%。读者间信度显示出中等变异性(α = 0.767, 0.595, 0.567),并且用GPT-4o替换人类读者并未显著影响一致性(Δ = -0.004至0.002)。GPT的反馈在大多数病例中被评为有帮助:89.8%、83.0%和92.0%。讨论:ChatGPT-4o能够可靠地识别关键教育错误。它可以作为支持放射科教育的可扩展工具。
英文摘要:
Objective: Radiology residents require timely, personalized feedback to develop accurate image analysis and reporting skills. Increasing clinical workload often limits attendings' ability to provide guidance. This study evaluates a HIPAA-compliant GPT-4o system that delivers automated feedback on breast imaging reports drafted by residents in real clinical settings. Methods: We analyzed 5,000 resident-attending report pairs from routine practice at a multi-site U.S. health system. GPT-4o was prompted with clinical instructions to identify common errors and provide feedback. A reader study using 100 report pairs was conducted. Four attending radiologists and four residents independently reviewed each pair, determined whether predefined error types were present, and rated GPT-4o's feedback as helpful or not. Agreement between GPT and readers was assessed using percent match. Inter-reader reliability was measured with Krippendorff's alpha. Educational value was measured as the proportion of cases rated helpful. Results: Three common error types were identified: (1) omission or addition of key findings, (2) incorrect use or omission of technical descriptors, and (3) final assessment inconsistent with findings. GPT-4o showed strong agreement with attending consensus: 90.5%, 78.3%, and 90.4% across error types. Inter-reader reliability showed moderate variability (α = 0.767, 0.595, 0.567), and replacing a human reader with GPT-4o did not significantly affect agreement (Δ = -0.004 to 0.002). GPT's feedback was rated helpful in most cases: 89.8%, 83.0%, and 92.0%. Discussion: ChatGPT-4o can reliably identify key educational errors. It may serve as a scalable tool to support radiology education.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。