arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.19330cs.CVcs.AI
收藏

3D-CT-GPT:通过大型视觉-语言模型集成生成3D放射学报告

3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models

Northwestern Polytechnical University(西北工业大学);The Second Xiangya Hospital, Central South University(中南大学湘雅二医院);Clinical Research Center for Medical Imaging in Hunan Province(湖南省医学影像临床研究中心);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Georgia(佐治亚大学)

Hao Chen, Wei Zhao, Yingli Li, Tianyang Zhong, Yisong Wang, Youlan Shang, Lei Guo, Junwei Han, Tianming Liu, Jun Liu, Tuo Zhang

AI总结:

本文提出3D-CT-GPT,一种基于视觉问答的医学视觉语言模型,用于从3D CT扫描自动生成放射学报告,实验证明其在报告准确性和质量上优于现有方法,为临床放射诊断提供了稳健的自动化报告生成方案。

摘要:

医学图像分析在现代放射诊断中至关重要,尤其是在医学成像数据呈指数级增长的背景下。对自动化报告生成系统的需求已变得日益迫切。尽管先前的研究主要集中于将机器学习和多模态语言模型应用于2D医学图像,但由于数据稀缺和计算复杂性,针对3D医学图像的报告生成探索较少。本文介绍了3D-CT-GPT,这是一种基于视觉问答(VQA)的医学视觉语言模型,专门设计用于从3D CT扫描(尤其是胸部CT)生成放射学报告。在公共和私有数据集上的大量实验表明,3D-CT-GPT在报告准确性和质量方面显著优于现有方法。尽管当前方法数量有限,包括部分开源的CT2Rep和开源的M3D,我们通过适当的数据转换和评估方法确保了公平比较。实验结果表明,3D-CT-GPT提高了诊断准确性和报告连贯性,确立了其作为临床放射学报告生成的稳健解决方案的地位。未来工作将侧重于扩展数据集并进一步优化模型,以提升其性能和适用性。

英文摘要:

Medical image analysis is crucial in modern radiological diagnostics, especially given the exponential growth in medical imaging data. The demand for automated report generation systems has become increasingly urgent. While prior research has mainly focused on using machine learning and multimodal language models for 2D medical images, the generation of reports for 3D medical images has been less explored due to data scarcity and computational complexities. This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model specifically designed for generating radiology reports from 3D CT scans, particularly chest CTs. Extensive experiments on both public and private datasets demonstrate that 3D-CT-GPT significantly outperforms existing methods in terms of report accuracy and quality. Although current methods are few, including the partially open-source CT2Rep and the open-source M3D, we ensured fair comparison through appropriate data conversion and evaluation methodologies. Experimental results indicate that 3D-CT-GPT enhances diagnostic accuracy and report coherence, establishing itself as a robust solution for clinical radiology report generation. Future work will focus on expanding the dataset and further optimizing the model to enhance its performance and applicability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑