arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.19802cs.CV
收藏

评估显微镜中用于像素和物体分类的视觉基础模型

Evaluating Vision Foundation Models for Pixel and Object Classification in Microscopy

Georg-August-University Göttingen, Institute of Computer Science Department of Tissue Dynamics;Regeneration, Max Planck Institute for Multidisciplinary Sciences, Göttingen Georg-August-University Göttingen, Faculty of Biology;Psychology CAIMed - Lower Saxony Center for AI \& Causal Methods in Medicine, Göttingen Cluster of Excellence Multiscale Bioimaging (MBExC), Georg-August-University Göttingen

Carolin Teuber, Anwai Archit, Tobias Boothe, Peter Ditte, Jochen Rink, Constantin Pape

AI总结:

本文评估了视觉基础模型在显微镜像素和物体分类中的应用,探讨了其在提升分类性能方面的潜力,并建立了相关基准。

摘要:

深度学习构成了计算机视觉大多数现代方法和工具的基础,包括生物医学成像。然而,在交互式语义分割(在此称为像素分类)和交互式物体级别分类(物体分类)中,基于特征的浅层学习仍被广泛使用。这归因于该领域数据的多样性、缺乏大规模预训练数据集以及对计算和标签效率的需求。相比之下,许多其他显微镜视觉任务(特别是细胞实例分割)的最新工具已经依赖深度学习,并最近显著受益于视觉基础模型(VFMs),特别是SAM。本文研究了VFMs是否能比当前方法在像素和物体分类中取得改进。为此,我们评估了几种VFMs,包括通用模型(SAM、SAM2、DINOv3)和领域特定的模型(μSAM、PathoSAM),结合浅层学习和注意力探测在五个多样且具有挑战性的数据集上进行评估。我们的结果表明,与手工特征相比,有持续的改进,并为实际改进提供了明确的路径。此外,本研究为显微镜中的VFMs建立了基准,并为该领域的未来发展提供了指导。

英文摘要:

Deep learning underlies most modern approaches and tools in computer vision, including biomedical imaging. However, for interactive semantic segmentation (often called pixel classification in this context) and interactive object-level classification (object classification), feature-based shallow learning remains widely used. This is due to the diversity of data in this domain, the lack of large pretraining datasets, and the need for computational and label efficiency. In contrast, state-of-the-art tools for many other vision tasks in microscopy - most notably cellular instance segmentation - already rely on deep learning and have recently benefited substantially from vision foundation models (VFMs), particularly SAM. Here, we investigate whether VFMs can also improve pixel and object classification compared to current approaches. To this end, we evaluate several VFMs, including general-purpose models (SAM, SAM2, SAM3, DINOv3) and domain-specific ones ($μ$SAM, PathoSAM, KRONOS), in combination with shallow learning and attentive probing on five diverse and challenging datasets. Our results demonstrate consistent improvements over hand-crafted features and provide a clear pathway toward practical improvements. Furthermore, our study establishes a benchmark for VFMs in microscopy and informs future developments in this area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑