用于单分支多器官多类别细胞语义分割的渐进式视觉-语言提示
Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch
AI总结：
针对多器官多类别细胞分割中架构复杂、视觉信息依赖限制性能的问题，提出单分支视觉-语言融合方法MONCH，通过分层特征提取和渐进式提示解码器提升分割性能，实验优于现有方法。
摘要：
病理细胞语义分割是计算病理学中的基础技术，对癌症诊断和有效治疗等应用至关重要。鉴于不同器官存在多种细胞类型，且细胞大小和形状差异细微，多器官、多类别细胞分割极具挑战性。现有多数方法采用多分支框架增强特征提取，但常导致架构复杂。此外，依赖视觉信息会因复杂纹理细节限制多类别分析性能。为应对这些挑战，我们提出单分支多器官多类别细胞语义分割方法MONCH，利用视觉-语言输入。具体而言，我们设计分层特征提取机制，提供从粗到细的特征以分割不同形状的细胞，包括高频、卷积和拓扑特征。受文本和多粒度视觉特征协同作用的启发，我们引入渐进式提示解码器协调多模态信息，从细到粗整合特征以更好地捕捉上下文。在具有显著类别不平衡和细微细胞大小形状差异的PanNuke数据集上的大量实验表明，MONCH优于最先进的细胞分割方法和视觉-语言模型。代码和实现将公开。
英文摘要：
Pathological cell semantic segmentation is a fundamental technology in computational pathology, essential for applications like cancer diagnosis and effective treatment. Given that multiple cell types exist across various organs, with subtle differences in cell size and shape, multi-organ, multi-class cell segmentation is particularly challenging. Most existing methods employ multi-branch frameworks to enhance feature extraction, but often result in complex architectures. Moreover, reliance on visual information limits performance in multi-class analysis due to intricate textural details. To address these challenges, we propose a Multi-OrgaN multi-Class cell semantic segmentation method with a single brancH (MONCH) that leverages vision-language input. Specifically, we design a hierarchical feature extraction mechanism to provide coarse-to-fine-grained features for segmenting cells of various shapes, including high-frequency, convolutional, and topological features. Inspired by the synergy of textual and multi-grained visual features, we introduce a progressive prompt decoder to harmonize multimodal information, integrating features from fine to coarse granularity for better context capture. Extensive experiments on the PanNuke dataset, which has significant class imbalance and subtle cell size and shape variations, demonstrate that MONCH outperforms state-of-the-art cell segmentation methods and vision-language models. Codes and implementations will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
