生成式AI在腰痛患者教育中尚未达到临床使用标准,即使采用检索增强生成
Generative AI Is Not Ready for Clinical Use in Patient Education for Lower Back Pain Patients, Even With Retrieval-Augmented Generation
AI总结:
本研究提出利用RAG和少样本学习的LLMs为腰痛患者生成教育材料,实验表明其优于传统LLMs,但生成内容的临床相关性和粒度仍存挑战,尚未达到临床使用标准。
摘要:
腰痛(LBP)是全球致残的主要原因。在LBP发作及随后的治疗中,充分的患者教育对于改善功能和长期结果至关重要。尽管患者教育策略取得了进展,但在向LBP患者提供个性化、循证信息方面仍存在显著差距。大型语言模型(LLMs)和生成式人工智能(GenAI)的最新进展已展现出增强患者教育的潜力。然而,它们在向LBP患者提供教育内容方面的应用和疗效仍未得到充分探索,值得进一步研究。在本研究中,我们引入了一种新方法,利用带有检索增强生成(RAG)和少样本学习的LLMs,为LBP患者生成量身定制的教育材料。物理治疗师使用Likert量表手动评估了模型响应的冗余度、准确性和完整性。此外,使用Flesch Reading Ease评分评估了生成教育材料的可读性。研究结果表明,基于RAG的LLMs优于传统LLMs,能提供更准确、完整且可读性更高的患者教育材料,且冗余更少。尽管如此,我们的分析显示,生成的材料尚未准备好用于临床实践。本研究强调了利用RAG的AI驱动模型在改善LBP患者教育方面的潜力;然而,在确保这些模型生成内容的临床相关性和粒度方面,仍存在重大挑战。
英文摘要:
Low back pain (LBP) is a leading cause of disability globally. Following the onset of LBP and subsequent treatment, adequate patient education is crucial for improving functionality and long-term outcomes. Despite advancements in patient education strategies, significant gaps persist in delivering personalized, evidence-based information to patients with LBP. Recent advancements in large language models (LLMs) and generative artificial intelligence (GenAI) have demonstrated the potential to enhance patient education. However, their application and efficacy in delivering educational content to patients with LBP remain underexplored and warrant further investigation. In this study, we introduce a novel approach utilizing LLMs with Retrieval-Augmented Generation (RAG) and few-shot learning to generate tailored educational materials for patients with LBP. Physical therapists manually evaluated our model responses for redundancy, accuracy, and completeness using a Likert scale. In addition, the readability of the generated education materials is assessed using the Flesch Reading Ease score. The findings demonstrate that RAG-based LLMs outperform traditional LLMs, providing more accurate, complete, and readable patient education materials with less redundancy. Having said that, our analysis reveals that the generated materials are not yet ready for use in clinical practice. This study underscores the potential of AI-driven models utilizing RAG to improve patient education for LBP; however, significant challenges remain in ensuring the clinical relevance and granularity of content generated by these models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。