6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
6根手指,1个肾脏:自然对抗性医学图像揭示视觉语言模型的关键弱点
Leon Mayer, Piotr Kalinowski, Caroline Ebersbach, Marcel Knopp, Tim Rädsch, Evangelia Christodoulou, Annika Reinke, Fiona R. Kolbinger, Lena Maier-Hein
机构
*
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems(德国癌症研究中心(DKFZ)海德堡,智能医学系统部门)
;
Medical Faculty, Heidelberg University(海德堡大学医学院)
;
Faculty of Mathematics and Computer Science, Heidelberg University(海德堡大学数学与计算机科学学院)
;
HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg(HIDSS4Health - 哈勃-马克斯信息与数据科学健康学院,卡尔斯鲁厄/海德堡)
;
Helmholtz Imaging, German Cancer Research Center (DKFZ)(哈勃-马克斯成像,德国癌症研究中心(DKFZ))
;
Engineering Faculty, Heidelberg University(海德堡大学工程学院)
;
School of Computation, Information and Technology, TUM(技术大学(TUM)计算、信息与技术学院)
;
Weldon School of Biomedical Engineering, Purdue University(普渡大学韦尔登生物医学工程学院)
;
Department of Visceral, Thoracic and Vascular Surgery, University Hospital and Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology(visceral、胸腔和血管外科部门,技术大学(TUD)德累斯顿大学医院和医学院)
;
National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Hospital Heidelberg(肿瘤疾病国家中心(NCT),海德堡NCT,DKFZ与海德堡大学医院之间的合作)
;
Heidelberg University Hospital, Surgical Clinic, Surgical AI Research Group(海德堡大学医院,外科诊所,外科人工智能研究组)
;
Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE(Mohamed Bin Zayed人工智能大学(MBZUAI),阿布扎赫,阿拉伯联合酋长国)
机构
*
Academy for Engineering and Technology, Fudan University(复旦大学工程与技术学院)
;
Tencent Youtu Lab(腾讯优图实验室)
;
Cognition and Intelligent Technology Laboratory(认知与智能技术实验室)
专题命中
领域大模型
:language model(title,abstract);large language model(abstract)
A multimodal and temporal foundation model for virtual patient representations at healthcare system scale
面向医疗系统规模的多模态与时间基础模型:虚拟患者表示
Andrew Zhang, Tong Ding, Sophia J. Wagner, Caiwei Tian, Ming Y. Lu, Rowland Pettit, Joshua E. Lewis, Alexandre Misrahi, Dandan Mo, Long Phi Le, Faisal Mahmood
机构
*
Department of Pathology, Mass General Brigham, Harvard Medical School(病理学系,马萨诸塞州总医院与哈佛医学院)
;
Cancer Program, Broad Institute of Harvard and MIT(癌症计划,哈佛-麻省理工Broad研究所)
;
Data Science Program, Dana-Farber Cancer Institute(数据科学计划,达纳-法伯癌症研究所)
;
Health Sciences and Technology, Harvard-MIT(健康科学与技术,哈佛-麻省理工)
;
Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛约翰·A·保罗森工程与应用科学学院,哈佛大学)
;
Department of Biomedical Informatics, Harvard Medical School(生物医学信息学系,哈佛医学院)
;
Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT)(电气工程与计算机科学,麻省理工学院(MIT))
;
School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland(计算机与通信科学学院,EPFL,瑞士洛桑)
机构
*
Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco(加利福尼亚大学旧金山分校放射与生物医学影像系智能成像中心)
;
Department of Bioengineering, University of California, Berkeley(加利福尼亚大学伯克利分校生物工程系)
;
Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco(加利福尼亚大学旧金山分校生物工程与治疗科学系)
;
Department of Medical Sciences and Technology, Indian Institute of Technology, Madras(印度理工学院马德拉斯分校医学科学与技术系)
;
Bay Area Institute of Computation, Altos Labs(Altos Labs湾区计算研究所)
SOTA: Self-adaptive Optimal Transport for Zero-Shot Classification with Multiple Foundation Models
SOTA: 自适应最优传输用于多基础模型的零样本分类
Zhanxuan Hu, Qiyu Xu, Yu Duan, Yonghang Tai, Huafeng Li
机构
*
Yunnan Normal University(云南师范大学)
;
Xidian University(西安电子科技大学)
;
Xi’an University of Posts and Telecommunications(西安邮电大学)
;
Kunming University of Science and Technology(昆明理工大学)
Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis
通过混合低秩专家专门化基础模型以实现全面头部CT分析
Youngjin Yoo, Han Liu, Bogdan Georgescu, Yanbo Zhang, Sasa Grbic, Michael Baumgartner, Thomas J. Re, Jyotipriya Das, Poikavila Ullaskrishnan, Eva Eibenberger, Andrei Chekkoury, Uttam K. Bodanapally, Savvas Nicolaou, Pina C. Sanelli, Thomas J. Schroeppel, Yvonne W. Lui, Eli Gibson
机构
*
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA(西门子医疗数字化技术与创新部)
;
Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany(西门子医疗计算机断层扫描部)
;
Department of Radiology, New York University, New York, NY USA(纽约大学放射科部)
;
Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA(马里兰大学医学中心放射科部)
;
Department of Radiology, Vancouver General Hospital, Vancouver, BC Canada(温哥华总医院放射科部)
;
Department of Radiology, Northwell Health, New York, NY USA(北well医疗放射科部)
;
Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, CO USA(UCHealth纪念医院外科部)
See-in-Pairs: Reference Image-Guided Comparative Vision-Language Models for Medical Diagnosis
See-in-Pairs: 用于医学诊断的参考图像引导的比较视觉-语言模型
Ruinan Jin, Gexin Huang, Xinwei Shen, Qiong Zhang, Yan Shuo Tan, Xiaoxiao Li
机构
*
Electrical and Computer Engineering Department, The University of British Columbia(不列颠哥伦比亚大学电气与计算机工程系)
;
Vector Institute(向量研究所)
;
ETH Zurich(苏黎世联邦理工学院)
;
Renmin University of China(中国人民大学)
;
National University of Singapore(新加坡国立大学)