A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
The City University of Hong Kong(香港城市大学);The Chinese University of Hong Kong(香港中文大学);Shenzhen University(深圳大学);National Yang Ming Chiao Tung University(国立阳明交通大学);Taipei Veterans General Hospital(台北荣民总医院);Stanford University(斯坦福大学)
Jie Liu, Wenxuan Wang, Yihang Su, Jingyuan Huan, Wenting Chen, Yudi Zhang, Cheng-Yi Li, Kao-Jung Chang, Xiaohan Xin, Linlin Shen, Michael R. Lyu
The significant breakthroughs of Medical Multi-Modal Large Language Models (Med-MLLMs) renovate modern healthcare with robust information synthesis and medical decision support. However, these models are often evaluated on benchmarks that are unsuitable for the Med-MLLMs due to the complexity of real-world diagnostics across diverse specialties. To address this gap, we introduce Asclepius, a novel Med-MLLM benchmark that comprehensively assesses Med-MLLMs in terms of: distinct medical specialties (cardiovascular, gastroenterology, etc.) and different diagnostic capacities (perception, disease analysis, etc.). Grounded in 3 proposed core principles, Asclepius ensures a comprehensive evaluation by encompassing 15 medical specialties, stratifying into 3 main categories and 8 sub-categories of clinical tasks, and exempting overlap with existing VQA dataset. We further provide an in-depth analysis of 6 Med-MLLMs and compare them with 3 human specialists, providing insights into their competencies and limitations in various medical contexts. Our work not only advances the understanding of Med-MLLMs' capabilities but also sets a precedent for future evaluations and the safe deployment of these models in clinical environments.
NEARL: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding
MoE Key Lab of Artificial Intelligence(人工智能MOE实验室);AI Institute(人工智能研究院);School of Computer Science(计算机科学学院);Shanghai Jiao Tong University(上海交通大学);Department of Radiology(放射科);The First Affiliated Hospital(第一附属医院);School of Medicine(医学院);Zhejiang University(浙江大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);State Key Laboratory of Infrared Physics(红外物理国家重点实验室);Shanghai Institute of Technical Physics(上海技术物理研究所);Chinese Academy of Science(中国科学院)
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);The Hong Kong University of Science and Technology(香港科技大学);Sun Yat-sen University(中山大学)
Toward Robust Medical Fairness: Debiased Dual-Modal Alignment via Text-Guided Attribute-Disentangled Prompt Learning for Vision-Language Models
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology(集成空天地海大数据应用技术国家工程实验室);Northwestern Polytechnical University(西北工业大学);Huiying Medical Technology Company Ltd.(慧影医疗技术有限公司);The School of Computer Science(计算机学院);The University of Sydney(悉尼大学);Department of Computer Science and Engineering(计算机科学与工程系);The Chinese University of Hong Kong(香港中文大学)
Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);City University of Hong Kong(香港城市大学);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);School of Biomedical Engineering, ShanghaiTech University(上海科技大学生物医学工程学院);Shanghai United Imaging Intelligence Co., Ltd.(上海联影智能医疗科技有限公司);Shanghai Clinical Research and Trial Center(上海临床研究与试验中心);University of Georgia(佐治亚大学);Massachusetts General Hospital(麻省总医院)
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training
Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Sun Yat-sen University(中山大学);Pazhou Lab(琶洲实验室)
Overview of the NLPCC 2025 Shared Task 4: Multi-modal, Multilingual, and Multi-hop Medical Instructional Video Question Answering Challenge
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究所,中国科学院);School of Computer Science and Technology, Beijing Institute of Technology(计算机科学与技术学院,北京理工大学);School of Engineering, Westlake University(工程学院,西湖大学)