Recent advances in multimodal large language models have enabled unified processing of visual and textual inputs, offering promising applications in general-purpose medical AI. However, their ability to generalize compositionally across unseen combinations of imaging modality, anatomy, and task type remains underexplored. We introduce CrossMed, a benchmark designed to evaluate compositional generalization (CG) in medical multimodal LLMs using a structured Modality-Anatomy-Task (MAT) schema. CrossMed reformulates four public datasets, CheXpert (X-ray classification), SIIM-ACR (X-ray segmentation), BraTS 2020 (MRI classification and segmentation), and MosMedData (CT classification) into a unified visual question answering (VQA) format, resulting in 20,200 multiple-choice QA instances. We evaluate two open-source multimodal LLMs, LLaVA-Vicuna-7B and Qwen2-VL-7B, on both Related and Unrelated MAT splits, as well as a zero-overlap setting where test triplets share no Modality, Anatomy, or Task with the training data. Models trained on Related splits achieve 83.2 percent classification accuracy and 0.75 segmentation cIoU, while performance drops significantly under Unrelated and zero-overlap conditions, demonstrating the benchmark difficulty. We also show cross-task transfer, where segmentation performance improves by 7 percent cIoU even when trained using classification-only data. Traditional models (ResNet-50 and U-Net) show modest gains, confirming the broad utility of the MAT framework, while multimodal LLMs uniquely excel at compositional generalization. CrossMed provides a rigorous testbed for evaluating zero-shot, cross-task, and modality-agnostic generalization in medical vision-language models.
CompDiff enables fair and zero shot medical image generation across demographic intersections through compositional diffusion
Institute of Data Science, Faculty of Science and Engineering, Maastricht University, Maastricht, The Netherlands（数据科学研究所，科学与工程学院，马斯特里赫特大学，马斯特里赫特，荷兰）；Department of Advanced Computing Sciences, Faculty of Science and Engineering, Maastricht University, Maastricht, The Netherlands（先进计算科学系，科学与工程学院，马斯特里赫特大学，马斯特里赫特，荷兰）；VITO, Belgium（比利时VITO研究院）
University of Edinburgh（爱丁堡大学）；Canon Medical Research Europe Ltd.（佳能医疗欧洲研究有限公司）；AC Codewheel Ltd（AC Codewheel有限公司）；The Alan Turing Institute（艾伦·图灵研究所）
Training medical image segmentation models usually requires a large amount of labeled data. By contrast, humans can quickly learn to accurately recognise anatomy of interest from medical (e.g. MRI and CT) images with some limited guidance. Such recognition ability can easily generalise to new images from different clinical centres. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in medical image segmentation. In this paper, we model the compositional components (i.e. patterns) of human a…
Enhancing Cross-Modal Medical Image Segmentation through Compositionality
University of Amsterdam（阿姆斯特丹大学）；The Netherlands Cancer Institute（荷兰癌症研究所）；Utrecht University（乌得勒支大学）；Maastricht University Medical Center（马斯特里赫特大学医学中心）
本文提出一种端到端跨模态医学图像分割框架，通过引入可学习的 von Mises-Fisher 核实现组合性归纳偏置与内容-风格解耦，有效提升了分割性能、可解释性并降低计算成本。
ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团）；Hupan Laboratory（湖畔实验室）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Department of Radiology, The Affiliated Yangming Hospital of Ningbo University（宁波大学附属阳明医院放射科）；Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University（浙江大学伊利诺伊大学厄巴纳香槟校区联合学院，浙江大学）；Hepato-Pancreato-Biliary Center, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University（清华长庚医院肝胆胰中心，清华大学临床医学院，清华医学，清华大学）；School of Software, Tsinghua University（清华大学软件学院）；Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学北京信息科学与技术国家研究中心）
Stanford AIMI（斯坦福大学医学人工智能实验室）；Stanford University（斯坦福大学）；University of California, Davis（加州大学戴维斯分校）；Stability AI
Multimodal models trained on large natural image-text pair datasets have exhibited astounding abilities in generating high-quality images. Medical imaging data is fundamentally different to natural images, and the language used to succinctly capture relevant details in medical data uses a different, narrow but semantically rich, domain-specific vocabulary. Not surprisingly, multi-modal models trained on natural image-text pairs do not tend to generalize well to the medical domain. Developing generative imaging models faithfully representing medical concepts while providing compositional divers…
Compositional Reward Models for Conditional Medical Image Generation
Yardi School of Artificial Intelligence（亚尔迪人工智能学院）；Indian Institute Of Technology, Delhi（德里印度理工学院）；Indian Institute of Science（印度科学学院）；Department of Computer Science and Engineering（计算机科学与工程系）
National Cancer Center Research Institute（国立癌症研究中心研究所）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息理工学研究生院）；Research Center for Advanced Science and Technology, The University of Tokyo（东京大学先端科学技术研究中心）
Medical images can be decomposed into normal and abnormal features, which is considered as the compositionality. Based on this idea, we propose an encoder-decoder network to decompose a medical image into two discrete latent codes: a normal anatomy code and an abnormal anatomy code. Using these latent codes, we demonstrate a similarity retrieval by focusing on either normal or abnormal features of medical images.
Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching
Università Campus Bio-Medico di Roma（罗马 Campus Bio-Medico 大学）；Umeå University（于默奥大学）；UniCamillus – Saint Camillus International University of Health Sciences（UniCamillus – 圣卡米勒斯国际健康科学大学）
Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）；Endoscopy Center, Zhongshan Hospital, Fudan University（复旦大学中山医院内窥镜中心）；Shanghai Collaborative Innovation Center of Endoscopy（上海内窥镜协同创新中心）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医疗研究中心）；Shanghai Key Laboratory of MICCAI（上海MICCAI重点实验室）；Alliance Manchester Business School, The University of Manchester（曼彻斯特大学联盟商学院）；Data Science Institute, Imperial College London（伦敦帝国学院数据科学研究所）
MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
South China University of Technology（华南理工大学）；Sun Yat-sen University（中山大学）；Hangzhou Dianzi University（杭州电子科技大学）；Zhejiang University of Finance & Economics（浙江财经大学）；National University of Singapore（新加坡国立大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；City University of Hong Kong（香港城市大学）
