arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.08259cs.CV
收藏

BROW:基于自蒸馏的全切片图像更好特征提取

BROW: Better featuRes fOr Whole slide image based on self-distillation

Zhejiang Lab(之江实验室)

Yuanfeng Wu, Shaojie Li, Zhiqiang Du, Wentao Zhu

AI总结:

针对超高分辨率全切片图像处理难题,提出基于自蒸馏和transformer架构的基础模型BROW,利用多尺度金字塔全局视图和图像块打乱技术,在超1.1万张切片数据上预训练,下游任务验证了其有效性与泛化能力。

摘要:

全切片图像(WSI)处理正逐渐成为多种疾病标准临床诊断的关键组成部分。然而,由于WSI具有超高分辨率这一显著特性,将常规图像处理算法直接应用于WSI面临一定障碍。大多数与WSI相关任务的性能依赖于提取WSI图像块特征表示的主干网络的有效性。因此,我们提出了BROW,一个用于为WSI提取更好特征表示的基础模型,该模型可方便地适应于下游任务,无需微调或仅需轻微微调。该模型采用transformer架构,使用自蒸馏框架进行预训练。为提高模型的鲁棒性,采用了图像块打乱等技术。此外,该模型利用WSI的独特性质,使用WSI的多尺度金字塔来引入额外的全局视图,从而进一步提升其性能。我们使用私有和公开数据构建了一个大型预训练数据集,包含超过11000张切片,超过180M个提取的图像块,涵盖与各种器官和组织相关的WSI。为评估所提模型的有效性,我们运行了广泛的下游任务,包括切片级亚型分类、图像块级分类和细胞核实例分割。结果证实了所提模型的有效性、鲁棒性和良好的泛化能力。这证实了其作为WSI特征提取基础模型的潜力,并突出了其在WSI处理中的应用前景。

英文摘要:

Whole slide image (WSI) processing is becoming part of the key components of standard clinical diagnosis for various diseases. However, the direct application of conventional image processing algorithms to WSI faces certain obstacles because of WSIs' distinct property: the super-high resolution. The performance of most WSI-related tasks relies on the efficacy of the backbone which extracts WSI patch feature representations. Hence, we proposed BROW, a foundation model for extracting better feature representations for WSIs, which can be conveniently adapted to downstream tasks without or with slight fine-tuning. The model takes transformer architecture, pretrained using self-distillation framework. To improve model's robustness, techniques such as patch shuffling have been employed. Additionally, the model leverages the unique properties of WSIs, utilizing WSI's multi-scale pyramid to incorporate an additional global view, thereby further enhancing its performance. We used both private and public data to make up a large pretraining dataset, containing more than 11000 slides, over 180M extracted patches, encompassing WSIs related to various organs and tissues. To assess the effectiveness of \ourmodel, we run a wide range of downstream tasks, including slide-level subtyping, patch-level classification and nuclei instance segmentation. The results confirmed the efficacy, robustness and good generalization ability of the proposed model. This substantiates its potential as foundation model for WSI feature extraction and highlights promising prospects for its application in WSI processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑