arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.11909eess.IVcs.CVq-bio.NC
收藏

多尺度表面 Vision Transformer

The Multiscale Surface Vision Transformer

King’s College London(伦敦国王学院);Technical University of Munich(慕尼黑工业大学)

Simon Dahan, Logan Z. J. Williams, Daniel Rueckert, Emma C. Robinson

AI总结:

本文提出 MS-SiT,通过局部网格窗口、移位窗口和 patch 合并实现表面数据的分层高效建模,并在新生儿表型预测与皮层分区任务中验证其优势。

摘要:

表面网格是表示人类皮层结构和功能信息的常用域,但其复杂的拓扑结构和几何形态给深度学习分析带来了重大挑战。尽管 Transformer 已作为适用于序列到序列学习的领域无关架构表现出色,但自注意力操作的二次成本仍是许多密集预测任务的障碍。受视觉 Transformer 分层建模最新进展的启发,我们提出多尺度表面 Vision Transformer(Multiscale Surface Vision Transformer,MS-SiT),作为表面深度学习的骨干架构。自注意力机制在局部网格窗口内应用,以实现对底层数据的高分辨率采样,同时移位窗口策略改善了窗口之间的信息共享。相邻 patch 被相继合并,使 MS-SiT 能够学习适用于任何预测任务的分层表示。结果表明,在使用 Developing Human Connectome Project(dHCP)数据集进行新生儿表型预测任务时,MS-SiT 优于现有表面深度学习方法。此外,将 MS-SiT 骨干构建为用于表面分割的 U 形架构后,在使用 UK Biobank(UKB)和人工标注 MindBoggle 数据集进行皮层分区时取得了具有竞争力的结果。代码和训练模型公开发布于 https://github.com/metrics-lab/surface-vision-transformers。

英文摘要:

Surface meshes are a favoured domain for representing structural and functional information on the human cortex, but their complex topology and geometry pose significant challenges for deep learning analysis. While Transformers have excelled as domain-agnostic architectures for sequence-to-sequence learning, the quadratic cost of the self-attention operation remains an obstacle for many dense prediction tasks. Inspired by some of the latest advances in hierarchical modelling with vision transformers, we introduce the Multiscale Surface Vision Transformer (MS-SiT) as a backbone architecture for surface deep learning. The self-attention mechanism is applied within local-mesh-windows to allow for high-resolution sampling of the underlying data, while a shifted-window strategy improves the sharing of information between windows. Neighbouring patches are successively merged, allowing the MS-SiT to learn hierarchical representations suitable for any prediction task. Results demonstrate that the MS-SiT outperforms existing surface deep learning methods for neonatal phenotyping prediction tasks using the Developing Human Connectome Project (dHCP) dataset. Furthermore, building the MS-SiT backbone into a U-shaped architecture for surface segmentation demonstrates competitive results on cortical parcellation using the UK Biobank (UKB) and manually-annotated MindBoggle datasets. Code and trained models are publicly available at https://github.com/metrics-lab/surface-vision-transformers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑