适配与评估多模态大语言模型用于青少年特发性脊柱侧弯自我管理：一种分而治之框架
Adapting and Evaluating Multimodal Large Language Models for Adolescent Idiopathic Scoliosis Self-Management: A Divide and Conquer Framework
AI总结：
本研究首次全面评估多模态大语言模型在青少年特发性脊柱侧弯（AIS）自我管理中的表现，提出“分而治之”框架，并引入脊柱关键点提示和检索增强生成技术以提升模型性能，但发现当前模型在脊柱畸形检测准确率上仍面临巨大挑战。
摘要：
本研究首次全面评估了多模态大语言模型（MLLMs）在青少年特发性脊柱侧弯（AIS）自我管理中的表现。我们构建了一个包含约3000张带有诊断文本的脊柱前后位X光片的数据库，并通过一个由视觉问答任务、领域知识评估任务和患者教育咨询评估任务组成的“分而治之”框架评估了五个MLLMs。我们的调查揭示了MLLMs在解释复杂脊柱影像和理解AIS护理知识方面的局限性。为了解决这些问题，我们率先使用脊柱关键点提示增强MLLMs，并分别编制了AIS知识库用于检索增强生成（RAG）。结果显示，视觉提示在不同架构中的效果各不相同，而RAG显著提升了模型在知识评估任务上的表现。研究结果表明，当前的MLLMs在实现AIS护理个性化助手方面还有很大差距。最大的挑战在于它们准确检测脊柱畸形位置（最佳准确率：0.55）和方向（最佳准确率：0.13）的能力。
英文摘要：
This study presents the first comprehensive evaluation of Multimodal Large Language Models (MLLMs) for Adolescent Idiopathic Scoliosis (AIS) self-management. We constructed a database of approximately 3,000 anteroposterior X-rays with diagnostic texts and evaluated five MLLMs through a `Divide and Conquer' framework consisting of a visual question-answering task, a domain knowledge assessment task, and a patient education counseling assessment task. Our investigation revealed limitations of MLLMs' ability in interpreting complex spinal radiographs and comprehending AIS care knowledge. To address these, we pioneered enhancing MLLMs with spinal keypoint prompting and compiled an AIS knowledge base for retrieval augmented generation (RAG), respectively. Results showed varying effectiveness of visual prompting across different architectures, while RAG substantially improved models' performances on the knowledge assessment task. Our findings indicate current MLLMs are far from capable in realizing personalized assistant in AIS care. The greatest challenge lies in their abilities to obtain accurate detections of spinal deformity locations (best accuracy: 0.55) and directions (best accuracy: 0.13).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
