Developing artificial intelligence (AI) and machine learning (ML) models for medical imaging typically involves extensive training and testing on large datasets, consuming significant computational time, energy, and resources. There is a need for more efficient methods that can achieve comparable or superior diagnostic performance without the associated resource burden. We investigated the feasibility of replacing conventional training procedures with an embedding-based approach that leverages concise and semantically meaningful representations of medical images. Using pre-trained foundational models-specifically, convolutional neural networks (CNN) like ResNet and multimodal models like Contrastive Language-Image Pre-training (CLIP)-we generated image embeddings for multi-class classification tasks. Simple linear classifiers were then applied to these embeddings. The approach was evaluated across diverse medical imaging modalities, including retinal images, mammography, dermatoscopic images, and chest radiographs. Performance was compared to benchmark models trained and tested using traditional methods. The embedding-based models surpassed the benchmark area under the receiver operating characteristic curve (AUC-ROC) scores by up to 87 percentage in multi-class classification tasks across the various medical imaging modalities. Notably, CLIP embedding models achieved the highest AUC-ROC scores, demonstrating superior classification performance while significantly reducing computational demands. Our study indicates that leveraging embeddings from pre-trained foundational models can effectively replace conventional, resource-intensive training and testing procedures in medical image analysis. This embedding-based approach offers a more efficient alternative for image segmentation, classification, and prediction, potentially accelerating AI technology integration into clinical practice.
Multimodal Medical Image Binding via Shared Text Embeddings
The Hong Kong Polytechnic University（香港理工大学）；Emory University（埃默里大学）；The University of Hong Kong（香港大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Science and Technology of China（中国科学技术大学）；Shanghai AI Laboratory（上海人工智能实验室）；Johns Hopkins University（约翰霍普金斯大学）
MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging
Microsoft Health and Life Sciences（微软健康与生命科学部门）；Microsoft Research（微软研究院）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Microsoft Azure AI（微软Azure人工智能部门）
Linear Algebraic Approaches to Neuroimaging Data Compression: A Comparative Analysis of Matrix and Tensor Decomposition Methods for High-Dimensional Medical Images
UAE: Universal Anatomical Embedding on Multi-modality Medical Images
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机科学与工程学院，空天地海一体化大数据应用技术国家工程实验室）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Beijing United Family Hospital（北京和睦家医院）；The First Affiliated Hospital, Zhejiang University（浙江大学医学院附属第一医院）；Peking Union Medical College Hospital（北京协和医院）
Towards Fair Medical AI: Adversarial Debiasing of 3D CT Foundation Embeddings
Rice University（莱斯大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）；McGovern Medical School, UTHealth Houston（德克萨斯大学健康科学中心休斯顿分校麦戈文医学院）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；Sidney Kimmel Comprehensive Cancer Center, The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院西德尼·基梅尔综合癌症中心）；Google Research（谷歌研究院）；Department of Neurosurgery, The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院神经外科）
SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Alibaba Group（阿里巴巴集团）；Johns Hopkins University（约翰斯·霍普金斯大学）；The First Affiliated Hospital of Zhejiang University（浙江大学附属第一医院）
FairDomain: Achieving Fairness in Cross-Domain Medical Image Segmentation and Classification
Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（纽约大学阿布扎比分校人工智能与机器人中心）；New York University（纽约大学）；Harvard University（哈佛大学）
HYDEN: Hyperbolic Density Representations for Medical Images and Reports
Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging（北京联合智能影像研究院智能诊断研究所）；School of Physics, Peking University（北京大学物理学院）
Early Intervention for VFM-based Multimodal Medical Image Classification
Renmin University of China（中国人民大学）；Beijing Key Laboratory for Intelligent Diagnosis of Fundus Diseases and Drug-Device R&D and Translation（眼底疾病智能诊断与药械研发转化北京市重点实验室）
本文提出EI框架，通过早期干预提升多模态影像疾病识别性能，引入MoR方法优化Vision Foundation Models适应，验证了在三个公开数据集上的有效性。
Embedding Radiomics into Vision Transformers for Multimodal Medical Image Classification
Duke Kunshan University（昆山杜克大学）；Jiangsu Provincial University Key (Construction) Laboratory for Smart Diagnosis and Treatment of Lung Cancer（江苏省高校重点（建设）实验室肺癌智能诊疗实验室）；The First People’s Hospital of Kunshan（昆山市第一人民医院）；Duke University（杜克大学）
Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation
Wuhan University of Technology（武汉理工大学）；Huazhong University of Science and Technology（华中科技大学）；Changzhou United lmaging Healthcare Surgical Technology Co. Ltd.（常州联影医疗手术技术有限公司）
提出Mask to Concept (M2C)框架，无需外部模块或重训练，仅用少量标注图像，通过可学习概念嵌入搜索和混合不确定性估计，实现SAM3在医学图像中的自动少样本标注，达到SOTA性能。
Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation
Global College, Shanghai Jiao Tong University（上海交通大学全球学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）
DiffuSAM: Diffusion-Based Prompt-Free SAM2 for Few-Shot and Source-Free Medical Image Segmentation
School of Electrical Engineering, Tel Aviv University, Israel（特拉维夫大学电气工程学院）；School of Biomedical Engineering, Tel Aviv University, Israel（特拉维夫大学生物医学工程学院）
Feature learning based on visual similarity triplets in medical image analysis: A case study of emphysema in chest CT scans
University of Copenhagen（哥本哈根大学）；Eindhoven University of Technology（埃因霍温理工大学）；Hvidovre Hospital（维兹奥勒医院）；Bispebjerg Hospital（比斯佩比约医院）；Erasmus MC - University Medical Center Rotterdam（伊拉斯谟医学中心——鹿特丹大学医学中心）
Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models
Computer Science Department, University of California, Los Angeles（计算机科学系，加州大学洛杉矶分校）；Department of Computational Medicine, David Geffen School of Medicine, University of California, Los Angeles（计算医学系，大卫·格芬医学院，加州大学洛杉矶分校）；Bioinformatics Interdepartmental Program, University of California, Los Angeles（生物信息学跨学科项目，加州大学洛杉矶分校）；Department of Human Genetics, University of California, Los Angeles（人类遗传学系，加州大学洛杉矶分校）
Content-Based Medical Image Retrieval with Opponent Class Adaptive Margin Loss
Amasya University（阿马西亚大学）；Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center, Bilkent University（比尔肯特大学国家磁共振研究中心）；Sabuncu Brain Research Center, Bilkent University（比尔肯特大学萨本库脑研究中心）
On Differentially Private 3D Medical Image Synthesis with Controllable Latent Diffusion Models
Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；Universitat de Barcelona（巴塞罗那大学）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Helmholtz AI（亥姆霍兹人工智能研究所）
本文提出基于医学属性条件的 Latent Diffusion Models，通过差分隐私训练生成 3D 心脏 MRI 图像，并首次在 3D 医学图像生成中量化差分隐私，发现预训练显著提升图像质量。
