arXiv 2402.143002024-03-08cs.CV

一种结合视觉上下文学习与掩码图像建模以改进超声分割的简单框架

A Simple Framework Uniting Visual In-context Learning with Masked Image Modeling to Improve Ultrasound Segmentation

University of Alberta（阿尔伯塔大学）

Yuyue Zhou, Banafshe Felfeliyan, Shrimanti Ghosh, Jessica Knight, Fatima Alves-Pereira, Christopher Keen, Jessica Küpper, Abhilash Rakkunedeth Hareendranathan, Jacob L. Jaremko

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对医学超声图像标注成本高、模型泛化受限的问题，提出结合视觉上下文学习与掩码图像建模的SimICL方法，在腕部超声骨分割任务上以有限标注实现远超现有模型的精度，降低标注成本并提升AI辅助实用性。

摘要：

传统深度学习模型逐张处理图像，在医学成像领域需要成本高昂且耗时的专家标注，且领域特定限制会制约模型的泛化能力。视觉上下文学习（ICL）是计算机视觉中一个新兴且备受关注的研究方向。与传统深度学习不同，ICL强调模型基于给定示例快速适应新任务的能力。受MAE-VQGAN启发，我们提出了一种名为SimICL的新型简单视觉ICL方法，将图像配对的视觉ICL与为自监督学习设计的掩码图像建模（MIM）相结合。我们在标注有限的腕部超声（US）数据集的骨结构分割任务上验证了该方法，其临床目标是分割骨结构以辅助后续的骨折检测。我们使用包含18名患者的3822张图像的测试集进行骨区域分割。SimICL实现了0.96的极高Dice系数（DC）和0.92的Jaccard指数（IoU），超越了现有最优的分割模型和视觉ICL模型（其最高DC为0.86、IoU为0.76），SimICL的DC和IoU最高提升了0.10和0.16。在人工标注有限的情况下取得如此高的一致性表明，SimICL甚至可用于小型超声数据集上的AI模型训练。与传统方法相比，这能大幅减少专家进行图像标注所需的时间，提升AI辅助在超声图像分析中的实际应用价值。

英文摘要：

Conventional deep learning models deal with images one-by-one, requiring costly and time-consuming expert labeling in the field of medical imaging, and domain-specific restriction limits model generalizability. Visual in-context learning (ICL) is a new and exciting area of research in computer vision. Unlike conventional deep learning, ICL emphasizes the model's ability to adapt to new tasks based on given examples quickly. Inspired by MAE-VQGAN, we proposed a new simple visual ICL method called SimICL, combining visual ICL pairing images with masked image modeling (MIM) designed for self-supervised learning. We validated our method on bony structures segmentation in a wrist ultrasound (US) dataset with limited annotations, where the clinical objective was to segment bony structures to help with further fracture detection. We used a test set containing 3822 images from 18 patients for bony region segmentation. SimICL achieved an remarkably high Dice coeffient (DC) of 0.96 and Jaccard Index (IoU) of 0.92, surpassing state-of-the-art segmentation and visual ICL models (a maximum DC 0.86 and IoU 0.76), with SimICL DC and IoU increasing up to 0.10 and 0.16. This remarkably high agreement with limited manual annotations indicates SimICL could be used for training AI models even on small US datasets. This could dramatically decrease the human expert time required for image labeling compared to conventional approaches, and enhance the real-world use of AI assistance in US image analysis.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。