arXiv 2401.062782024-05-28cs.CVcs.LG

胃肠内窥镜视觉问题中自监督预训练研究

A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy

University of Central Lancashire（中央兰开夏大学）

Edward Sanderson, Bogdan J. Matuszewski

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究比较了自监督与监督预训练在胃肠内窥镜视觉任务中的效果，发现自监督预训练通常更优，且ImageNet-1k预训练优于专用数据集，但深度估计例外，并指出不同骨干架构适用于不同任务。

摘要：

胃肠内窥镜（GIE）中的视觉任务解决方案通常使用以ImageNet-1k为骨干、以监督方式预训练的图像编码器。然而，使用现代自监督预训练算法以及最近发布的包含10万张未标记GIE图像的数据集（Hyperkvasir-unlabelled）可能带来改进。在本研究中，我们考察了以ResNet50和ViT-B为骨干、分别通过自监督和监督方式在ImageNet-1k及Hyperkvasir-unlabelled（仅自监督）上预训练的模型，在一系列GIE视觉任务中的微调性能。除了为每项任务确定所考虑方案中最合适的预训练流程和骨干架构外，我们的结果还提出了三项一般性原则。首先，自监督预训练通常比监督预训练产生更适合GIE视觉任务的骨干。其次，使用ImageNet-1k进行自监督预训练通常比使用Hyperkvasir-unlabelled预训练更合适，但结肠镜中的单目深度估计是一个显著例外。第三，ViT-B在息肉分割和结肠镜单目深度估计中更合适，ResNet50在息肉检测中更合适，而两种架构在解剖标志识别和病理发现表征方面表现相似。我们希望这项工作能引起人们对GIE视觉任务预训练复杂性的关注，为开发比常规方法更合适的方案提供信息，并激发对此主题的进一步研究以推动这一发展。代码可在以下网址获取：\underline{github.com/ESandML/SSL4GIE}

英文摘要：

Solutions to vision tasks in gastrointestinal endoscopy (GIE) conventionally use image encoders pretrained in a supervised manner with ImageNet-1k as backbones. However, the use of modern self-supervised pretraining algorithms and a recent dataset of 100k unlabelled GIE images (Hyperkvasir-unlabelled) may allow for improvements. In this work, we study the fine-tuned performance of models with ResNet50 and ViT-B backbones pretrained in self-supervised and supervised manners with ImageNet-1k and Hyperkvasir-unlabelled (self-supervised only) in a range of GIE vision tasks. In addition to identifying the most suitable pretraining pipeline and backbone architecture for each task, out of those considered, our results suggest three general principles. Firstly, that self-supervised pretraining generally produces more suitable backbones for GIE vision tasks than supervised pretraining. Secondly, that self-supervised pretraining with ImageNet-1k is typically more suitable than pretraining with Hyperkvasir-unlabelled, with the notable exception of monocular depth estimation in colonoscopy. Thirdly, that ViT-Bs are more suitable in polyp segmentation and monocular depth estimation in colonoscopy, ResNet50s are more suitable in polyp detection, and both architectures perform similarly in anatomical landmark recognition and pathological finding characterisation. We hope this work draws attention to the complexity of pretraining for GIE vision tasks, informs this development of more suitable approaches than the convention, and inspires further research on this topic to help advance this development. Code available: \underline{github.com/ESandML/SSL4GIE}

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文