Semantic segmentation of brain tumors is a fundamental medical image analysis task involving multiple MRI imaging modalities that can assist clinicians in diagnosing the patient and successively studying the progression of the malignant entity. In recent years, Fully Convolutional Neural Networks (FCNNs) approaches have become the de facto standard for 3D medical image segmentation. The popular "U-shaped" network architecture has achieved state-of-the-art performance benchmarks on different 2D and 3D semantic segmentation tasks and across various imaging modalities. However, due to the limited kernel size of convolution layers in FCNNs, their performance of modeling long-range information is sub-optimal, and this can lead to deficiencies in the segmentation of tumors with variable sizes. On the other hand, transformer models have demonstrated excellent capabilities in capturing such long-range information in multiple domains, including natural language processing and computer vision. Inspired by the success of vision transformers and their variants, we propose a novel segmentation model termed Swin UNEt TRansformers (Swin UNETR). Specifically, the task of 3D brain tumor semantic segmentation is reformulated as a sequence to sequence prediction problem wherein multi-modal input data is projected into a 1D sequence of embedding and used as an input to a hierarchical Swin transformer as the encoder. The swin transformer encoder extracts features at five different resolutions by utilizing shifted windows for computing self-attention and is connected to an FCNN-based decoder at each resolution via skip connections. We have participated in BraTS 2021 segmentation challenge, and our proposed model ranks among the top-performing approaches in the validation phase. Code: https://monai.io/research/swin-unetr
Fully Convolutional Neural Networks (FCNNs) with contracting and expanding paths have shown prominence for the majority of medical image segmentation applications since the past decade. In FCNNs, the encoder plays an integral role by learning both global and local features and contextual representations which can be utilized for semantic output prediction by the decoder. Despite their success, the locality of convolutional layers in FCNNs, limits the capability of learning long-range spatial dependencies. Inspired by the recent success of transformers for Natural Language Processing (NLP) in l…
SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images
Philips Research North America（飞利浦北美研究院）；Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院）；The Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院高级医学计算与分析中心）
Robert Bosch Center of Data Science and AI, IIT Madras（印度理工学院马德拉斯分校罗伯特·博世数据科学与人工智能中心）；Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）
In this work, we present our proposed method to segment the pulmonary arteries from the CT scans using Swin UNETR and U-Net-based deep neural network architecture. Six models, three models based on Swin UNETR, and three models based on 3D U-net with residual units were ensemble using a weighted average to make the final segmentation masks. Our team achieved a multi-level dice score of 84.36 percent through this method. The code of our work is available on the following link: https://github.com/akansh12/parse2022. This work is part of the MICCAI PARSE 2022 challenge.
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation
Shanghai Jiao Tong University（上海交通大学）；University of Washington（华盛顿大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；University of California, Santa Cruz（加利福尼亚大学圣克鲁兹分校）；Tsinghua University（清华大学）；Stanford University（斯坦福大学）
University Hospital Essen（埃森大学医院）；Cancer Research Center Cologne Essen（科隆埃森癌症研究中心）；University Medicine Essen（埃森大学医学院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；HIDSS4Health - Helmholtz Information and Data Science School for Health（亥姆霍兹健康信息与数据科学学院）；University of Minho（米尼奥大学）；Computer Algorithms for Medicine Laboratory（医学计算机算法实验室）
Tumor volume and changes in tumor characteristics over time are important biomarkers for cancer therapy. In this context, FDG-PET/CT scans are routinely used for staging and re-staging of cancer, as the radiolabeled fluorodeoxyglucose is taken up in regions of high metabolism. Unfortunately, these regions with high metabolism are not specific to tumors and can also represent physiological uptake by normal functioning organs, inflammation, or infection, making detailed and reliable tumor segmentation in these scans a demanding task. This gap in research is addressed by the AutoPET challenge, wh…
Weakly Supervised Intracranial Hemorrhage Segmentation using Head-Wise Gradient-Infused Self-Attention Maps from a Swin Transformer in Categorical Learning
Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis
Institute of Computer Science, University of Tartu（塔尔图大学计算机科学研究所）；Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学人工智能中心）；STACC
Duke University（杜克大学）；Vanderbilt University（范德堡大学）
Lesion detection in digital breast tomosynthesis (DBT) is an important and a challenging problem characterized by a low prevalence of images containing tumors. Due to the label scarcity problem, large deep learning models and computationally intensive algorithms are likely to fail when applied to this task. In this paper, we present a practical yet lightweight backbone to improve the accuracy of tumor detection. Specifically, we propose a novel modification of visual transformer (ViT) on image feature patches to connect the feature patches of a tumor with healthy backgrounds of breast images a…
Frequency-Aware Ensemble Learning for BraTS 2025 Pediatric Brain Tumor Segmentation
Institute of Natural Sciences, School of Mathematical Sciences, MOE-LSC, Shanghai Jiao Tong University（自然科学院、数学科学学院、MOE-LSC、上海交通大学）；Department of Radiology, Huashan Hospital, Fudan University（放射科、华山医院、复旦大学）；Department of Neurosurgery, Huashan Hospital, Fudan University（神经外科、华山医院、复旦大学）
Xi’an University of Posts&Telecommunications（西安邮电大学）；Xidian University（西安电子科技大学）
The Resolution of feature maps is critical for medical image segmentation. Most of the existing Transformer-based networks for medical image segmentation are U-Net-like architecture that contains an encoder that utilizes a sequence of Transformer blocks to convert the input medical image from high-resolution representation into low-resolution feature maps and a decoder that gradually recovers the high-resolution representation from low-resolution feature maps. Unlike previous studies, in this paper, we utilize the network design style from the High-Resolution Network (HRNet), replace the convo…
Background and objective: Parotid gland tumors account for approximately 2% to 10% of head and neck tumors. Preoperative tumor localization, differential diagnosis, and subsequent selection of appropriate treatment for parotid gland tumors are critical. However, the relative rarity of these tumors and the highly dispersed tissue types have left an unmet need for a subtle differential diagnosis of such neoplastic lesions based on preoperative radiomics. Recently, deep learning methods have developed rapidly, especially Transformer beats the traditional convolutional neural network in computer v…
Instance-level quantitative saliency in multiple sclerosis lesion segmentation
Translational Imaging in Neurology (ThINk)（神经学转化成像（ThINk））；Department of Medicine and Biomedical Engineering, University Hospital Basel and University of Basel（医学与生物医学工程系，巴塞尔大学医院和巴塞尔大学）；Department of Neurology, University Hospital Basel（神经病学系，巴塞尔大学医院）；Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB)（巴塞尔临床神经免疫学和神经科学研究中心（RC2NB））；MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland（MedGIFT，信息学院，管理学院，瓦莱-迪-瓦尔省应用科学与艺术西方瑞士大学）；CIBM Center for Biomedical Imaging, Lausanne, Switzerland（洛桑瑞士生物医学成像中心）；Radiology Department, Lausanne University Hospital (CHUV) and University of Lausanne（放射科，洛桑大学医院（CHUV）和洛桑大学）
Department of Computer Science, Vanderbilt University（范德堡大学计算机科学系）
Histopathology has played an essential role in cancer diagnosis. With the rapid advances in convolutional neural networks (CNN). Various CNN-based automated pathological image segmentation approaches have been developed in computer-assisted pathological image analysis. In the past few years, Transformer neural networks (Transformer) have shown the unique merit of capturing the global long-distance dependencies across the entire image as a new deep learning paradigm. Such merit is appealing for exploring spatially heterogeneous pathological images. However, there have been very few, if any, stu…
Tumor-aware augmentation with task-guided attention analysis improves rectal cancer segmentation from magnetic resonance images
Department of Medical Physics, Memorial Sloan Kettering Cancer Center（医学物理系，纪念斯隆凯特琳癌症中心）；Department of Radiology, Memorial Sloan Kettering Cancer Center（放射学系，纪念斯隆凯特琳癌症中心）；Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center（放射肿瘤学系，纪念斯隆凯特琳癌症中心）；Department of Surgery, Memorial Sloan Kettering Cancer Center（外科系，纪念斯隆凯特琳癌症中心）；Department of Medicine, Memorial Sloan Kettering Cancer Center（医学系，纪念斯隆凯特琳癌症中心）
Numerous oncology indications have extensively quantified metabolically active tumors using positron emission tomography (PET) and computed tomography (CT). F-fluorodeoxyglucose-positron emission tomography (FDG-PET) is frequently utilized in clinical practice and clinical drug research to detect and measure metabolically active malignancies. The assessment of tumor burden using manual or computer-assisted tumor segmentation in FDG-PET images is widespread. Deep learning algorithms have also produced effective solutions in this area. However, there may be a need to improve the performance of a…
Integrating AI for Human-Centric Breast Cancer Diagnostics: A Multi-Scale and Multi-View Swin Transformer Framework
Concordia Institute for Information Systems Engineering (CIISE)（康科迪亚信息系统工程研究所）；Concordia University（康科迪亚大学）；Thomas Jefferson University Hospital（泰勒斯·杰弗里斯大学医院）；College of Computing & Informatics（计算与信息学院）；Drexel University（德雷塞尔大学）
HBFormer: A Hybrid-Bridge Transformer for Microtumor and Miniature Organ Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；Guangdong University of Technology（广东技术大学）；University of Macau（澳门大学）；Huizhou University（惠州大学）
Model Ensemble for Brain Tumor Segmentation in Magnetic Resonance Imaging
Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital（谢赫·扎耶德儿科外科创新研究所，儿童国家医院）；CIBER-BBN, ISCIII（西班牙生物医学研究中心-生物工程、生物材料与纳米医学分部，卡洛斯三世卫生研究所）；Princeton University（普林斯顿大学）；School of Medicine and Health Sciences, George Washington University（乔治·华盛顿大学医学与健康科学学院）
Progressive Growing of Patch Size: Curriculum Learning for Accelerated and Improved Medical Image Segmentation
School of Computation, Information and Technology, Technical University Munich（技术大学慕尼黑计算、信息与技术学院）；Department of Radiation Oncology, TUM School of Medicine, TUM University Hospital rechts der Isar, Technical University of Munich（肿瘤放射治疗科，TUM医学院，TUM大学医院rechts der Isar，技术大学慕尼黑）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（生物医学影像中的机器学习研究所，海德堡慕尼黑）；Institute of Radiation Medicine, Helmholtz Center Munich（放射医学研究所，海德堡慕尼黑）；Munich Center of Machine Learning (MCML)（慕尼黑机器学习中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（生物医学工程与成像科学学院，伦敦国王学院）；Departament de Matemàtiques i Informàtica, Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM), Universitat de Barcelona（数学与计算机科学系，巴塞罗那人工智能在医学实验室（BCN-AIM），巴塞罗那大学）
NeuroVascU-Net: A Unified Multi-Scale and Cross-Domain Adaptive Feature Fusion U-Net for Precise 3D Segmentation of Brain Vessels in Contrast-Enhanced T1 MRI
School of Electrical and Computer Engineering, College of Engineering, University of Tehran（电信工程学院，工程学院，德黑兰大学）；Department of EECS, Lassonde School of Engineering, York University（电子工程系，拉索nde工程学院，约克大学）；Department of Neurosurgery, School of Medicine, Iran University of Medical Sciences（神经外科系，医学院，伊朗医学科学大学）；Department of Electrical and Computer Engineering, University of Waterloo（电子工程系，滑铁库大学）
Self-Supervised Learning for Organs At Risk and Tumor Segmentation with Uncertainty Quantification
University of Central Florida（中佛罗里达大学）；Northwestern University（西北大学）；KnowledgeVis LLC（KnowledgeVis有限责任公司）；Orlando Health（奥兰多健康）；University of Chicago（芝加哥大学）
An Ensemble Approach for Brain Tumor Segmentation and Synthesis
Paul G. Allen School of Computer Science, University of Washington（华盛顿大学保罗G.艾伦计算机科学学院）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Department of Bioengineering, University of Washington（华盛顿大学生物工程系）；Department of Electrical and Computer Engineering, University of Washington（华盛顿大学电气与计算机工程系）；Department of Mechanical Engineering, University of Washington（华盛顿大学机械工程系）；Watauga High School（沃托加高中）；Department of Neurological Surgery, University of Washington（华盛顿大学神经外科系）
BraTS-PEDs: Results of the Multi-Consortium International Pediatric Brain Tumor Segmentation Challenge 2023
Center for Data-Driven Discovery in Biomedicine (D3b)（数据驱动生物医学发现中心）；The Children’s Hospital of Philadelphia（费城儿童医院）；Sheikh Zayed Institute for Pediatric Surgical Innovation（谢赫扎伊德儿童外科创新研究所）；Children’s National Hospital（儿童国家医院）；Department of Neurosurgery（神经外科部门）；Dana-Farber Brigham Cancer Center & Boston Children’s Hospital（达纳-法伯布里弗癌症中心及波士顿儿童医院）；Department of Paediatric Radiology（儿科放射科部门）；University of Utah（犹他大学）；Cincinnati Children’s Hospital Medical Center（辛辛那提儿童医学中心）；Department of Radiology（放射科部门）；Division of Neuroradiology, Department of Radiology, Boston Children’s Hospital, Harvard Medical School（神经放射科，放射科，波士顿儿童医院，哈佛医学院）；College of Arts and Sciences, University of Pennsylvania（艺术与科学学院，宾夕法尼亚大学）；Department of Diagnostic and Interventional Radiology, All India Institute of Medical Sciences（诊断与介入放射科，全印度医学科学院）；Department of Diagnostic & Interventional Imaging, Neuroradiology section, The University of Texas Health at Houston, Texas, USA（诊断与介入成像部门，神经放射科，德克萨斯大学健康科学中心，德克萨斯，美国）；Beth Israel Deaconess Medical Center, Harvard Medical School（贝斯以色列医疗中心，哈佛医学院）；UNIDAD DE PATOLOGIA CLINICA（临床病理部门）
DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation
Department of Mechanical and Mechatronics Engineering（机械与机电工程系）；University of Waterloo（滑铁卢大学）；Department of Radiology Withsim Clinic Seongnam, Korea（Seongnam诊所放射科）
Here Comes the Explanation: A Shapley Perspective on Multi-contrast Medical Image Segmentation
University of Washington（华盛顿大学）；Paul G. Allen School of Computer Science, University of Washington（华盛顿大学保罗·G·艾伦计算机科学学院）；Department of Neurological Surgery, University of Washington（华盛顿大学神经外科（注：此处按要求提取到学院/大学级别，实际原文该机构属于华盛顿大学，故提取为华盛顿大学））
How we won BraTS 2023 Adult Glioma challenge? Just faking it! Enhanced Synthetic Data Augmentation and Model Ensemble for brain tumour segmentation
University of Minho（米尼奥大学）；Institute for Artificial Intelligence in Medicine（人工智能医学研究所）；Graz University of Technology（格拉茨工业大学）；University Medicine Essen（埃森大学医学中心）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心）；University Hospital Essen（埃森大学医院）；German Cancer Consortium (DKTK)（德国癌症联盟）
KTH Royal Institute of Technology（KTH皇家理工学院）；Science for Life Laboratory（生命科学实验室）
Artifacts on magnetic resonance scans are a serious challenge for both radiologists and computer-aided diagnosis systems. Most commonly, artifacts are caused by motion of the patients, but can also arise from device-specific abnormalities such as noise patterns. Irrespective of the source, artifacts can not only render a scan useless, but can potentially induce misdiagnoses if left unnoticed. For instance, an artifact may masquerade as a tumor or other abnormality. Retrospective artifact correction (RAC) is concerned with removing artifacts after the scan has already been taken. In this work, …
