arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2603.26952cs.CVcs.LG
收藏

多模态深度学习在结合RGB和热成像的糖尿病足溃疡分期中的应用

Multimodal Deep Learning for Diabetic Foot Ulcer Staging Using Integrated RGB and Thermal Imaging

Gulengul Mermer, Mustafa Furkan Aksu, Gozde Ozsezer, Sevki Cetinkalp, Orhan Er, Mehmet Kemal Gullu

AI总结:

本文研究了多模态图像对深度学习模型在糖尿病足溃疡分期中的影响,通过RGB和热成像数据结合提升模型性能,VGG16模型在RGB+热成像数据集上达到93.25%的准确率。

摘要:

糖尿病足溃疡(DFU)是糖尿病的严重并发症,可能导致截肢和高额医疗费用。本研究旨在探讨多模态图像对深度学习模型在DFU分期分类中的影响。我们开发了一种基于Raspberry Pi的便携成像系统,同时捕获RGB和热成像数据。在医院环境中收集了1,205个样本的数据集,并由专家标注为六个阶段。为评估模型性能,我们准备了三种不同的训练集:仅RGB、仅热成像和RGB+热成像(热成像作为第四个通道)。这些训练集在DenseNet121、EfficientNetV2、InceptionV3、ResNet50和VGG16模型上进行训练。结果表明,多模态训练数据集在四个通道中结合RGB和热数据,优于单模态方法。VGG16模型在RGB+热成像数据集上表现最佳,达到93.25%的准确率、92.53%的F1分数和91.03%的MCC。Grad-CAM热图可视化显示,热通道通过突出溃疡区域的温度异常帮助模型聚焦正确位置,而RGB通道通过补充的结构和纹理信息支持决策过程。

英文摘要:

Diabetic foot ulcers (DFU) are one of the serious complications of diabetes that can lead to amputations and high healthcare costs. Regular monitoring and early diagnosis are critical for reducing the clinical burden and the risk of amputation. The aim of this study is to investigate the impact of using multimodal images on deep learning models for the classification of DFU stages. To this end, we developed a Raspberry Pi-based portable imaging system capable of simultaneously capturing RGB and thermal images. Using this prototype, a dataset consisting of 1,205 samples was collected in a hospital setting. The dataset was labeled by experts into six distinct stages. To evaluate the models performance, we prepared three different training sets: RGB-only, thermal-only, and RGB+Thermal (with the thermal image added as a fourth channel). We trained these training sets on the DenseNet121, EfficientNetV2, InceptionV3, ResNet50, and VGG16 models. The results show that the multimodal training dataset, in which RGB and thermal data are combined across four channels, outperforms single-modal approaches. The highest performance was observed in the VGG16 model trained on the RGB+Thermal dataset. The model achieved an accuracy of 93.25%, an F1-score of 92.53%, and an MCC of 91.03%. Grad-CAM heatmap visualizations demonstrated that the thermal channel helped the model focus on the correct location by highlighting temperature anomalies in the ulcer region, while the RGB channel supported the decision-making process with complementary structural and textural information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑