Visual task adaptation has been demonstrated to be effective in adapting pre-trained Vision Transformers (ViTs) to general downstream visual tasks using specialized learnable layers or tokens. However, there is yet a large-scale benchmark to fully explore the effect of visual task adaptation on the realistic and important medical domain, particularly across diverse medical visual modalities, such as color images, X-ray, and CT. To close this gap, we present Med-VTAB, a large-scale Medical Visual Task Adaptation Benchmark consisting of 1.68 million medical images for diverse organs, modalities, and adaptation approaches. Based on Med-VTAB, we explore the scaling law of medical prompt tuning concerning tunable parameters and the generalizability of medical visual adaptation using non-medical/medical pre-train weights. Besides, we study the impact of patient ID out-of-distribution on medical visual adaptation, which is a real and challenging scenario. Furthermore, results from Med-VTAB indicate that a single pre-trained model falls short in medical task adaptation. Therefore, we introduce GMoE-Adapter, a novel method that combines medical and general pre-training weights through a gated mixture-of-experts adapter, achieving state-of-the-art results in medical visual task adaptation.
AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Models
Carnegie Mellon University（卡内基梅隆大学）；Brown University（布朗大学）；National Institute of Technology Karnataka（印度卡纳塔克国家理工学院）；University of Pittsburgh（匹兹堡大学）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of British Columbia（不列颠哥伦比亚大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
COSMOS: Cross-Modality Unsupervised Domain Adaptation for 3D Medical Image Segmentation based on Target-aware Domain Translation and Iterative Self-Training
The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN
Fudan University（复旦大学）；Ruijin Hospital, Shanghai Jiaotong University（上海交通大学医学院附属瑞金医院）；Tongji Hospital, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院）；Leiden University Medical Center（莱顿大学医学中心）
MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosis
ShanghaiTech University（上海科技大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
LE-UDA: Label-efficient unsupervised domain adaptation for medical image segmentation
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research（新加坡科技研究局）；National University of Singapore（新加坡国立大学）；Shanghai University（上海大学）；University of Science and Technology of China（中国科学技术大学）；Chinese Academy of Sciences（中国科学院）
Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans
Queen Mary University of London（伦敦玛丽女王大学）；National Technical University of Athens（雅典国家技术大学）；National Infrastructures for Research and Technology（国家研究与技术基础设施机构）
Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learning
The University of Texas Arlington（德克萨斯大学阿灵顿分校）；RIKEN（理化学研究所）；University of Tokyo（东京大学）；National Cancer Center Research Institute（国立癌症中心研究所）；National Library of Medicine - National Institutes of Health（美国国立卫生研究院国家医学图书馆）；Temple University（天普大学）
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Chinese University of Hong Kong（香港中文大学）；Lehigh University（理海大学）；Harvard University（哈佛大学）；The University of Georgia（佐治亚大学）
Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Nankai University（南开大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Southern California（南加州大学）；China University of Mining and Technology（中国矿业大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
Zhejiang University（浙江大学）；University of Electronic Science and Technology of China（电子科技大学）；Alibaba（阿里巴巴）；The Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（新加坡国立大学）
Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma（人工智能与计算机系统单位，工程系，罗马生物医学大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University（诊断与干预系，辐射物理，生物医学工程，乌梅大学）
Deep-based quality assessment of medical images through domain adaptation
Université d'Orléans（奥尔良大学）；Institut National Des Télécommunications et TIC（国家电信与信息通信技术学院）
预测多媒体内容的质量在不同的领域经常需要。在某些应用中，质量指标是至关重要的，具有很高的影响力，并可能影响决策，如医疗多媒体诊断。在本文中，我们专注于这样的应用程序，提出了一个有效的和浅层的模型，用于预测质量的医学图像，而无需参考少量的注释数据。我们的模型基于卷积自注意力，旨在从图像的相关局部特征中对复杂表示进行建模，图像本身在图像上滑动以插值全局质量分数。我们还以无监督和半监督的方式应用领域自适应学习。该模型是通过一个数据集组成的几个图像及其相应的主观评分进行评估。所得到的结果表明，所提出的方法的效率，而且，应用域自适应概括不同的多媒体域的感知质量预测的下游任务的相关性。\footnote{Funded by the TIC-ART project, Regional fund (Region Centre-Val de Loire)}
