用于处理医学影像深度学习中小数据集的对比学习与掩码自编码器自监督预训练方法
Self-Supervised Pre-Training with Contrastive and Masked Autoencoder Methods for Dealing with Small Datasets in Deep Learning for Medical Imaging
AI总结：
针对医学影像标注数据集小的问题，本文比较了对比学习与掩码自编码器SparK两种自监督预训练方法，发现SparK在小数据集微调时鲁棒性更强，推荐用于小样本医学影像任务。
摘要：
医学影像中的深度学习有望将诊断错误的风险降至最低，减轻放射科医生的工作量，并加快诊断速度。训练此类深度学习模型需要大型且准确的数据集，并为所有训练样本提供标注。然而，在医学影像领域，由于标注的高复杂性、获取受限或疾病的罕见性，针对特定任务的标注数据集通常很小。为了应对这一挑战，可以使用自监督学习领域的方法，在无标注的大型图像数据集上对深度学习模型进行预训练。预训练完成后，小型标注数据集足以针对特定任务对模型进行微调。医学影像中最受欢迎的自监督预训练方法基于对比学习。然而，最近在自然图像处理中的研究表明，掩码自编码器方法具有巨大潜力。我们的工作在医学图像上比较了最先进的对比学习方法与最近引入的用于卷积神经网络（CNN）的掩码自编码器方法“SparK”。因此，我们在一个大型无标注CT图像数据集上进行预训练，并在几个CT分类任务上进行微调。由于在医学影像中获取足够的标注训练数据具有挑战性，因此评估自监督预训练方法在小型数据集上微调时的表现具有特别的意义。通过实验逐渐减少用于微调的训练数据集大小，我们发现这种减少产生的影响因所选预训练类型而异。SparK预训练方法对训练数据集大小的鲁棒性优于对比方法。基于我们的结果，我们建议在仅有小型标注数据集的医学影像任务中使用SparK预训练。
英文摘要：
Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis. Training such deep learning models requires large and accurate datasets, with annotations for all training samples. However, in the medical imaging domain, annotated datasets for specific tasks are often small due to the high complexity of annotations, limited access, or the rarity of diseases. To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning. After pre-training, small annotated datasets are sufficient to fine-tune the models for a specific task. The most popular self-supervised pre-training approaches in medical imaging are based on contrastive learning. However, recent studies in natural image processing indicate a strong potential for masked autoencoder approaches. Our work compares state-of-the-art contrastive learning methods with the recently introduced masked autoencoder approach "SparK" for convolutional neural networks (CNNs) on medical images. Therefore we pre-train on a large unannotated CT image dataset and fine-tune on several CT classification tasks. Due to the challenge of obtaining sufficient annotated training data in medical imaging, it is of particular interest to evaluate how the self-supervised pre-training methods perform when fine-tuning on small datasets. By experimenting with gradually reducing the training dataset size for fine-tuning, we find that the reduction has different effects depending on the type of pre-training chosen. The SparK pre-training method is more robust to the training dataset size than the contrastive methods. Based on our results, we propose the SparK pre-training for medical imaging tasks with only small annotated datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
