arXivDaily arXiv每日学术速递 周一至周五更新
看起来今天可以划水了
arXiv 2402.07595eess.IVcs.LG
收藏

ImageNet预训练深度学习模型与DINOv2在医学图像分类中的比较分析

Comparative Analysis of ImageNet Pre-Trained Deep Learning Models and DINOv2 in Medical Imaging Classification

School of Software Engineering, Beijing University of Technology(北京工业大学软件学院);Beijing Neurosurgical Institute(北京神经外科研究所);Beijing Tiantan Hospital, Capital Medical University(首都医科大学附属北京天坛医院);Sorbonne Université(索邦大学);Institut du Cerveau – Paris Brain Institute - ICM(巴黎脑研究所(ICM));CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);Inserm(法国国家健康与医学研究院);AP-HP(巴黎公共医疗集团);Hôpital de la Pitié-Salpêtrière(皮蒂-萨尔皮特里耶医院)

Yuning Huang, Jingchen Zou, Lanxi Meng, Xin Yue, Qing Zhao, Jianqiang Li, Changwei Song, Gabriel Jimenez, Shaowu Li, Guanghui Fu

AI总结:

本文比较了ImageNet预训练模型与DINOv2在医学图像分类中的表现,发现DINOv2在类似自然图像的公开数据集上表现更优,但在MRI临床数据上不及ImageNet模型,且较小模型即可满足需求。

摘要:

医学图像分析经常面临数据稀缺的挑战。迁移学习在解决这一问题的同时节省计算资源方面一直很有效。近期,像DINOv2这样采用vision transformer架构的基础模型的出现,为该领域带来了新机遇并引起了广泛关注。然而,DINOv2在临床数据上的性能仍需验证。本文中,我们使用三种临床模态的脑MRI数据执行了胶质瘤分级任务。我们在迁移学习背景下比较了各种预训练深度学习模型的性能,包括基于ImageNet和DINOv2的模型。我们的重点是理解冻结机制对性能的影响。我们还在其他三种公开数据集上验证了我们的发现:胸部放射摄影、眼底放射摄影和皮肤镜检查。我们的发现表明,在我们的临床数据集中,DINOv2的性能不如基于ImageNet的预训练模型;而在公开数据集中,DINOv2通常优于其他模型,特别是在使用冻结机制时。在不同任务中,各种规模的DINOv2模型表现出相似的性能。总之,DINOv2适用于医学图像分类任务,尤其是处理类似于自然图像的数据。然而,对于与自然图像差异显著的数据(如MRI),其有效性可能有所不同。此外,采用较小版本的模型即可满足医学任务需求,从而节省资源。我们的代码可在https://github.com/GuanghuiFU/medical_DINOv2_eval获取。

英文摘要:

Medical image analysis frequently encounters data scarcity challenges. Transfer learning has been effective in addressing this issue while conserving computational resources. The recent advent of foundational models like the DINOv2, which uses the vision transformer architecture, has opened new opportunities in the field and gathered significant interest. However, DINOv2's performance on clinical data still needs to be verified. In this paper, we performed a glioma grading task using three clinical modalities of brain MRI data. We compared the performance of various pre-trained deep learning models, including those based on ImageNet and DINOv2, in a transfer learning context. Our focus was on understanding the impact of the freezing mechanism on performance. We also validated our findings on three other types of public datasets: chest radiography, fundus radiography, and dermoscopy. Our findings indicate that in our clinical dataset, DINOv2's performance was not as strong as ImageNet-based pre-trained models, whereas in public datasets, DINOv2 generally outperformed other models, especially when using the frozen mechanism. Similar performance was observed with various sizes of DINOv2 models across different tasks. In summary, DINOv2 is viable for medical image classification tasks, particularly with data resembling natural images. However, its effectiveness may vary with data that significantly differs from natural images such as MRI. In addition, employing smaller versions of the model can be adequate for medical task, offering resource-saving benefits. Our codes are available at https://github.com/GuanghuiFU/medical_DINOv2_eval.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑