arXiv 2604.178562026-04-20cs.CV

PlankFormer：通过MAE预训练视觉变换器和伪社区图像生成实现鲁棒的浮游生物实例分割

PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation

Graduate School of Information Sciences, Tohoku University（东京大学信息科学研究生院）；The Center for Ecological Research, Kyoto University（京都大学生态研究中心）；Graduate School of Life Sciences, Tohoku University（东京大学生命科学研究生院）

Masaharu Miyazaki, Yurie Otake, Koichi Ito, Wataru Makino, Jotaro Urabe, Takafumi Aoki

备注：: Accepted to ICPR2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PlankFormer框架，通过生成伪社区图像和MAE预训练视觉变换器解决浮游生物实例分割中的数据不足和区分困难问题，实验表明其在高杂质密度环境下表现优异。

摘要：

浮游生物监测对评估水生生态系统至关重要，但手动显微镜分析耗时费力。自动化分割密集图像中的浮游生物至关重要，但面临两个主要挑战：（i）像素级标注数据集稀缺；（ii）传统CNN方法难以区分浮游生物与碎片和重叠个体。为此，我们提出PlankFormer，一种新的浮游生物实例分割框架。首先，为克服数据不足，我们引入一种方法生成标记的伪社区图像（PCI），通过在多样化的背景上合成单个浮游生物图像，包括由生成模型创建的背景。其次，我们提出使用视觉变换器（ViT）主干和Mask2Former解码器的分割模型。为了稳健地捕捉浮游生物的全局结构特征以应对遮挡和碎片，我们采用MAE对未标记的单个图像进行自监督预训练。在真实世界数据集上的实验结果表明，我们的方法显著优于传统方法，如Mask R-CNN，特别是在高碎片密度的挑战环境中。我们证明了我们的合成训练策略和基于MAE的架构能够实现高精度分割，并且需要较少的单个浮游生物图像的手动标注。

英文摘要：

Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automating the segmentation of plankton from crowded images is crucial, however, it faces two major challenges: (i) the scarcity of pixel-level annotated datasets and (ii) the difficulty of distinguishing plankton from debris and overlapping individuals using conventional CNN-based methods. To address these issues, we propose PlankFormer, a novel framework for plankton instance segmentation. First, to overcome the data shortage, we introduce a method to generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto diverse backgrounds, including those created by generative models. Second, we propose a segmentation model utilizing a Vision Transformer (ViT) backbone with a Mask2Former decoder. To robustly capture the global structural features of plankton against occlusion and debris, we employ a Masked Autoencoder (MAE) for self-supervised pre-training on unlabeled individual images. Experimental results on real-world datasets demonstrate that our method significantly outperforms conventional methods, such as Mask R-CNN, particularly in challenging environments with high debris density. We demonstrate that our synthetic training strategy and MAE-based architecture enable high-precision segmentation with requiring less manual annotations for individual plankton images.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。