arXiv 2408.112892024-08eess.IVcs.CV

HMT-UNet：用于医学图像分割的混合 Mamba-Transformer 视觉 UNet

HMT-UNet: A hybird Mamba-Transformer Vision UNet for Medical Image Segmentation

Nanjing University（南京大学）；BISTU（北京信息科技大学）

Mingya Zhang, Zhihao Chen, Yiyuan Ge, Xianping Tao

备注：: arXiv admin note: text overlap with arXiv:2403.09157; text overlap with arXiv:2407.08083 by other authors

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出融合 Mamba SSM 与 Transformer 自注意力的 U 形 HTM-UNet，以线性复杂度建模长程空间依赖，并在多个公共及私有医学分割数据集上验证其竞争力。

摘要：

在医学图像分割领域，基于 CNN 和 Transformer 的模型均已得到深入研究。然而，CNN 对长程依赖关系的建模能力有限，使其难以充分利用图像中的语义信息；另一方面，二次计算复杂度也给 Transformer 带来挑战。诸如 Mamba 之类的状态空间模型（SSM）已被视为一种有前景的方法，它们不仅在建模长程交互方面表现出优越性能，还能保持线性计算复杂度。经过精心设计的 SSM（状态空间模型）与 Transformer 混合机制，可增强其高效建模视觉特征的能力。大量实验表明，将自注意力机制集成到 Mamba 架构各层之后的混合部分中，可显著提升捕获长程空间依赖关系的建模能力。本文利用 SSM 的混合机制，提出一种用于医学图像分割的 U 形架构模型，命名为 Hybird Transformer vision Mamba UNet（HTM-UNet）。我们在 ISIC17、ISIC18、CVC-300、CVC-ClinicDB、Kvasir、CVC-ColonDB、ETIS-Larib PolypDB 公共数据集及 ZD-LCI-GIM 私有数据集上进行了全面实验。结果表明，HTM-UNet 在医学图像分割任务中展现出具有竞争力的性能。代码可在 https://github.com/simzhangbest/HMT-Unet 获取。

英文摘要：

In the field of medical image segmentation, models based on both CNN and Transformer have been thoroughly investigated. However, CNNs have limited modeling capabilities for long-range dependencies, making it challenging to exploit the semantic information within images fully. On the other hand, the quadratic computational complexity poses a challenge for Transformers. State Space Models (SSMs), such as Mamba, have been recognized as a promising method. They not only demonstrate superior performance in modeling long-range interactions, but also preserve a linear computational complexity. The hybrid mechanism of SSM (State Space Model) and Transformer, after meticulous design, can enhance its capability for efficient modeling of visual features. Extensive experiments have demonstrated that integrating the self-attention mechanism into the hybrid part behind the layers of Mamba's architecture can greatly improve the modeling capacity to capture long-range spatial dependencies. In this paper, leveraging the hybrid mechanism of SSM, we propose a U-shape architecture model for medical image segmentation, named Hybird Transformer vision Mamba UNet (HTM-UNet). We conduct comprehensive experiments on the ISIC17, ISIC18, CVC-300, CVC-ClinicDB, Kvasir, CVC-ColonDB, ETIS-Larib PolypDB public datasets and ZD-LCI-GIM private dataset. The results indicate that HTM-UNet exhibits competitive performance in medical image segmentation tasks. Our code is available at https://github.com/simzhangbest/HMT-Unet.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。