Structural Attention：重新思考用于非配对医学图像合成的Transformer
Structural Attention: Rethinking Transformer for Unpaired Medical Image Synthesis
AI总结：
本文提出UNest架构，通过结合结构归纳偏置和利用Segment-Anything Model提取前景结构并执行结构注意力，解决了Transformer在非配对医学图像合成中因缺乏归纳偏置而收敛到非最优解的问题，在六项任务上提升高达19.30%。
摘要：
非配对医学图像合成旨在为准确的临床诊断提供互补信息，并解决获取对齐的多模态医学扫描的挑战。基于Transformer的模型凭借其捕获长距离依赖的能力，在图像转换任务中表现出色。尽管在监督训练设置中有效，但其在非配对图像合成中的表现却显得力不从心，特别是在合成结构细节方面。本文通过实证表明，由于缺乏强归纳偏置，Transformer在缺乏配对数据时会收敛到非最优解。为了解决这一问题，我们引入了UNet Structured Transformer（UNest），这是一种为非配对医学图像合成结合了结构归纳偏置的新颖架构。我们利用基础模型Segment-Anything Model精确提取前景结构，并在主要解剖结构内执行结构注意力。这引导模型学习关键解剖区域，从而改善非配对训练中由于缺乏监督而导致的结构合成问题。在两个公共数据集上进行了评估，涵盖三种模态，即MR、CT和PET，UNest在六项医学图像合成任务中较近期方法提升了高达19.30%。我们的代码已在 https://github.com/HieuPhan33/MICCAI2024-UNest 发布。
英文摘要：
Unpaired medical image synthesis aims to provide complementary information for an accurate clinical diagnostics, and address challenges in obtaining aligned multi-modal medical scans. Transformer-based models excel in imaging translation tasks thanks to their ability to capture long-range dependencies. Although effective in supervised training settings, their performance falters in unpaired image synthesis, particularly in synthesizing structural details. This paper empirically demonstrates that, lacking strong inductive biases, Transformer can converge to non-optimal solutions in the absence of paired data. To address this, we introduce UNet Structured Transformer (UNest), a novel architecture incorporating structural inductive biases for unpaired medical image synthesis. We leverage the foundational Segment-Anything Model to precisely extract the foreground structure and perform structural attention within the main anatomy. This guides the model to learn key anatomical regions, thus improving structural synthesis under the lack of supervision in unpaired training. Evaluated on two public datasets, spanning three modalities, i.e., MR, CT, and PET, UNest improves recent methods by up to 19.30% across six medical image synthesis tasks. Our code is released at https://github.com/HieuPhan33/MICCAI2024-UNest.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
