In the field of medical images, although various works find Swin Transformer has promising effectiveness on pixelwise dense prediction, whether pre-training these models without using extra dataset can further boost the performance for the downstream semantic segmentation remains unexplored.Applications of previous representation learning methods are hindered by the limited number of 3D volumes and high computational cost. In addition, most of pretext tasks designed specifically for Transformer are not applicable to hierarchical structure of Swin Transformer. Thus, this work proposes a token-level representation learning loss that maximizes agreement between token embeddings from different augmented views individually instead of volume-level global features. Moreover, we identify a potential representation collapse exclusively caused by this new loss. To prevent collapse, we invent a simple "rotate-and-restore" mechanism, which rotates and flips one augmented view of input volume, and later restores the order of tokens in the feature maps. We also modify the contrastive loss to address the discrimination between tokens at the same position but from different volumes. We test our pre-training scheme on two public medical segmentation datasets, and the results on the downstream segmentation task show more improvement of our methods than other state-of-the-art pre-trainig methods.
SAM-DCE: Addressing Token Uniformity and Semantic Over-Smoothing in Medical Segmentation
Mohamed bin Zayed University of AI（Mohamed bin Zayed人工智能大学）；Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Liverpool（利物浦大学）；Monash University（墨尔本大学）
What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning, Technical University of Munich（慕尼黑工业大学慕尼黑机器学习中心）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学伊萨尔河右岸医院）；ELLIS Institute Finland（芬兰ELLIS研究所）；Aalto University（阿尔托大学）；Imperial College London（伦敦帝国理工学院）
The visual analysis of retina and of its vascular characteristics is important in the diagnosis and monitoring of diseases of visual perception. In the related medical diagnoses, the digital processing of the fundus images is used to obtain the segmentation of retinal vessels. However, an image segmentation is often requiring methods based on peculiar or complex algorithms: in this paper we will show some alternative approaches obtained by applying freely available tools to enhance, without a specific segmentation, the images of the fundus of the eye. We will see in particular, that combining …
VinBigData JSC（VinBigData股份公司）；VinUniversity（VinUniversity（温纳大学））；VinUni-Illinois Smart Health Center（VinUni-伊利诺伊智慧健康中心）
Building an accurate computer-aided diagnosis system based on data-driven approaches requires a large amount of high-quality labeled data. In medical imaging analysis, multiple expert annotators often produce subjective estimates about "ground truth labels" during the annotation process, depending on their expertise and experience. As a result, the labeled data may contain a variety of human biases with a high rate of disagreement among annotators, which significantly affect the performance of supervised machine learning algorithms. To tackle this challenge, we propose a simple yet effective a…
Enhancing Medical Imaging with GANs Synthesizing Realistic Images from Limited Data
Columbia University（哥伦比亚大学）；Texas Tech University（德克萨斯理工大学）；University of California, Davis（加利福尼亚大学戴维斯分校）；Johns Hopkins University（约翰斯·霍普金斯大学）；North Carolina State University（北卡罗来纳州立大学）
PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification
Dalian University of Technology（大连理工大学）；School of Computer Science and Technology（计算机科学与技术学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Dalian Minzu University（大连民族大学）；School of Information and Communication Engineering（信息与通信工程学院）
Random Token Fusion for Multi-View Medical Diagnosis
KTH Royal Institute of Technology（皇家理工学院）；Science for Life Laboratory（生命科学实验室）；Karolinska Institutet（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Convolution Neural Networks (CNNs) are widely used in medical image analysis, but their performance degrade when the magnification of testing images differ from the training images. The inability of CNNs to generalize across magnification scales can result in sub-optimal performance on external datasets. This study aims to evaluate the robustness of various deep learning architectures in the analysis of breast cancer histopathological images with varying magnification scales at training and testing stages. Here we explore and compare the performance of multiple deep learning architectures, inc…
Although supervised deep-learning has achieved promising performance in medical image segmentation, many methods cannot generalize well on unseen data, limiting their real-world applicability. To address this problem, we propose a deep learning-based Bayesian framework, which jointly models image and label statistics, utilizing the domain-irrelevant contour of a medical image for segmentation. Specifically, we first decompose an image into components of contour and basis. Then, we model the expected label as a variable only related to the contour. Finally, we develop a variational Bayesian fra…
Enhancing Multimodal Medical Image Classification using Cross-Graph Modal Contrastive Learning
Department of Systems Engineering, Stevens Institute of Technology, Hoboken, New Jersey, USA（系统工程系，史蒂文斯理工学院）；Department of Nuclear Medicine, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung, Taiwan（高雄长庚纪念医院核医学部）；Institute of Statistics, National University of Kaohsiung, Kaohsiung, Taiwan（国立高雄大学统计研究所）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（深圳先进技术研究院，中国科学院）
Transformer has achieved impressive successes for various computer vision tasks. However, most of existing studies require to pretrain the Transformer backbone on a large-scale labeled dataset (e.g., ImageNet) for achieving satisfactory performance, which is usually unavailable for medical images. Additionally, due to the gap between medical and natural images, the improvement generated by the ImageNet pretrained weights significantly degrades while transferring the weights to medical image processing tasks. In this paper, we propose Bootstrap Own Latent of Transformer (BOLT), a self-supervise…
Enhancing Transfer Learning for Medical Image Classification with SMOTE: A Comparative Study
University of Dhaka（达卡大学）；Utah State University（犹他州立大学）；Noakhali Science and Technology University（诺阿卡利科学技术大学）；Daffodil International University（水仙花国际大学）
AGA: An adaptive group alignment framework for structured medical cross-modal representation learning
School of Computing and Artificial Intelligence（计算机与人工智能学院）；Southwest Jiaotong University（西南交通大学）；Department of Gastroenterology（消化内科部）；The Third People’s Hospital of Chengdu（成都第三人民医院）
Enhancing Network Initialization for Medical AI Models Using Large-Scale, Unlabeled Natural Images
University Hospital RWTH Aachen（亚琛工业大学附属医院）；Technische Universität Dresden（德累斯顿工业大学）；University Hospital Carl Gustav Carus Dresden（德累斯顿卡尔·古斯塔夫·卡鲁斯大学医院）；Else Kröner Fresenius Center for Digital Health（Else Kröner Fresenius 数字健康中心）；University Hospital Heidelberg（海德堡大学医院）；National Center for Tumor Diseases（国家肿瘤疾病中心）
TM-UNet: Token-Memory Enhanced Sequential Modeling for Efficient Medical Image Segmentation
Dalian University of Technology（大连理工大学）；University of Lincoln（林肯大学）；University of Nottingham（诺丁汉大学）；Waseda University（早稻田大学）；University of Nottingham Ningbo China（宁波诺丁汉大学）；Yale University（耶鲁大学）
Can we use sparse tokens for dense prediction, e.g., segmentation? Although token sparsification has been applied to Vision Transformers (ViT) to accelerate classification, it is still unknown how to perform segmentation from sparse tokens. To this end, we reformulate segmentation as a sparse encoding -> token completion -> dense decoding (SCD) pipeline. We first empirically show that naively applying existing approaches from classification token pruning and masked image modeling (MIM) leads to failure and inefficient training caused by inappropriate sampling algorithms and the low quality of …
SemiSAM: Enhancing Semi-Supervised Medical Image Segmentation via SAM-Assisted Consistency Regularization
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）
Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models
Chinese University of Hong Kong（香港中文大学）；Westlake University（西湖大学）；Southern Medical University（南方医科大学）；Jiangnan University（江南大学）；Dalian University of Technology（大连理工大学）；Shanghai Jiao Tong University（上海交通大学）；University of Copenhagen（哥本哈根大学）
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）
