整张全切片图像能告诉我们什么？亚型引导的掩码Transformer用于病理图像描述
What a Whole Slide Image Can Tell? Subtype-guided Masked Transformer for Pathological Image Captioning
AI总结：
本文提出亚型引导掩码Transformer（SGMT），将全切片图像视为稀疏图像块序列进行病理描述生成，并通过亚型预测和非对称掩码机制提升训练效果，在PatchGastricADC22数据集上优于传统RNN方法。
摘要：
全切片图像（WSIs）的病理描述虽然在计算机辅助病理诊断中至关重要，但由于数据集和模型训练效果的限制，相关研究很少。本文提出了一种新的基于Transformer的亚型引导掩码Transformer（SGMT）病理描述范式，该范式将WSI视为稀疏图像块序列，并从该序列生成整体描述句子。SGMT中引入了伴随的亚型预测，以指导训练过程并提高描述准确性。我们还提出了一种非对称掩码机制方法，以应对病理图像描述的大尺寸约束，其中SGMT中序列图像块的数量在训练和推理阶段分别以不同方式进行采样。在PatchGastricADC22数据集上的实验表明，我们的方法通过基于Transformer的模型有效地适应了该任务，并取得了优于传统基于RNN方法的性能。我们的代码将公开以供进一步研究和开发。
英文摘要：
Pathological captioning of Whole Slide Images (WSIs), though is essential in computer-aided pathological diagnosis, has rarely been studied due to the limitations in datasets and model training efficacy. In this paper, we propose a new paradigm Subtype-guided Masked Transformer (SGMT) for pathological captioning based on Transformers, which treats a WSI as a sequence of sparse patches and generates an overall caption sentence from the sequence. An accompanying subtype prediction is introduced into SGMT to guide the training process and enhance the captioning accuracy. We also present an Asymmetric Masked Mechansim approach to tackle the large size constraint of pathological image captioning, where the numbers of sequencing patches in SGMT are sampled differently in the training and inferring phases, respectively. Experiments on the PatchGastricADC22 dataset demonstrate that our approach effectively adapts to the task with a transformer-based model and achieves superior performance than traditional RNN-based methods. Our codes are to be made available for further research and development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
