医学 AI - arXivDaily 专题

2501.03939 2025-01-14 cs.CV cs.MM 70%

Visual question answering: from early developments to recent advances -- a survey

Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Sunil Aryal, Imran Razzak, Hakim Hacid

专题命中医疗多模态：medical image(abstract)；diagnosis(abstract)；分类 cs.CV

Comments 20 papers

2408.16343 2025-01-06 cs.CV cs.AI 70%

Toward Robust Early Detection of Alzheimer's Disease via an Integrated Multimodal Learning Approach

Yifei Chen, Shenghao Zhu, Zhaojie Fang, Chang Liu, Binfeng Zou, Yuhe Wang, Shuo Chang, Fan Jia, Feiwei Qin, Jin Fan, Yong Peng, Changmiao Wang

专题命中医疗多模态：MRI(abstract)；diagnosis(abstract)；分类 cs.CV

Comments 5 pages, 2 figures

Journal ref ICASSP 2025

2412.05876 2024-12-10 cs.CV cs.AI 70%

MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training

Xuefeng Ni, Linshan Wu, Jiaxin Zhuang, Qiong Wang, Mingxiang Wu, Varut Vardhanabhuti, Lihai Zhang, Hanyu Gao, Hao Chen

专题命中医疗多模态：medical image(abstract)；radiology(abstract)；分类 cs.CV

Comments 10 Pages

2412.02978 2024-12-05 cs.CV 70%

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Qing Zhang, Hang Guo, Siyuan Yang, Qingli Li, Yan Wang

专题命中医疗多模态：pathology(abstract)；diagnosis(abstract)；分类 cs.CV

2401.11421 2024-09-05 cs.CV 70%

Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinement

Weijian Huang, Cheng Li, Hao Yang, Jiarun Liu, Yong Liang, Hairong Zheng, Shanshan Wang

专题命中医疗多模态：medical image(abstract)；radiology(abstract)；分类 cs.CV

2408.10703 2024-08-21 cs.CV 70%

Large Language Models for Multimodal Deformable Image Registration

Mingrui Ma, Weijie Wang, Jie Ning, Jianfeng He, Nicu Sebe, Bruno Lepri

专题命中医疗多模态：medical image(abstract)；CT(abstract)；分类 cs.CV

2406.18146 2024-07-01 cs.CV 70%

A Refer-and-Ground Multimodal Large Language Model for Biomedicine

Xiaoshuang Huang, Haifeng Huang, Lingdong Shen, Yehui Yang, Fangxin Shang, Junwei Liu, Jia Liu

专题命中医疗多模态：medical image(abstract)；biomedical(abstract)；分类 cs.CV

Comments Accepted by MICCAI2024

2406.07078 2024-06-12 cs.CV cs.AI 70%

Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology

Huahui Yi, Xiaofei Wang, Kang Li, Chao Li

专题命中医疗多模态：pathology(abstract)；diagnosis(abstract)；分类 cs.CV

2406.04249 2024-06-07 cs.CV 70%

Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

Zhicheng Cai

专题命中医疗多模态：MRI(abstract)；CT(abstract)；分类 cs.CV

2405.11338 2024-05-24 cs.CV cs.AI 70%

EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imaging

Danli Shi, Weiyi Zhang, Xiaolan Chen, Yexin Liu, Jiancheng Yang, Siyu Huang, Yih Chung Tham, Yingfeng Zheng, Mingguang He

专题命中医疗多模态：clinical AI(abstract)；diagnosis(abstract)；分类 cs.CV

Comments 21 pages, 2 figures, 4 tables

2404.16192 2024-04-26 cs.CL cs.CV 70%

Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering

Cuong Nhat Ha, Shima Asaadi, Sanjeev Kumar Karn, Oladimeji Farri, Tobias Heimann, Thomas Runkler

专题命中医疗多模态：radiology(abstract)；biomedical(abstract)；分类 cs.CV

Comments Clinical NLP @ NAACL 2024

2307.02783 2023-11-21 cs.CV cs.HC 70%

UIT-Saviors at MEDVQA-GI 2023: Improving Multimodal Learning with Image Enhancement for Gastrointestinal Visual Question Answering

Triet M. Thai, Anh T. Vo, Hao K. Tieu, Linh N. P. Bui, Thien T. B. Nguyen

专题命中医疗多模态：medical image(abstract)；diagnosis(abstract)；分类 cs.CV

Comments ImageCLEF2023 published version: https://ceur-ws.org/Vol-3497/paper-129.pdf

2305.07920 2023-10-24 cs.CV cs.MM 70%

Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training

Ke Zhang, Yan Yang, Jun Yu, Hanliang Jiang, Jianping Fan, Qingming Huang, Weidong Han

专题命中医疗多模态：medical image(abstract)；diagnosis(abstract)；分类 cs.CV

2303.01615 2023-09-19 cs.CV 70%

ConTEXTual Net: A Multimodal Vision-Language Model for Segmentation of Pneumothorax

Zachary Huemann, Xin Tie, Junjie Hu, Tyler J. Bradshaw

专题命中医疗多模态：medical image(abstract)；radiology(abstract)；分类 cs.CV

2306.00890 2023-06-02 cs.CV cs.CL 70%

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

专题命中医疗多模态：medical image(abstract)；biomedical(abstract)；分类 cs.CV

Comments 17 pages; Website: https://aka.ms/llava-med

2101.02469 2021-01-08 cs.CV cs.AI 70%

Multimodal Gait Recognition for Neurodegenerative Diseases

Aite Zhao, Jianbo Li, Junyu Dong, Lin Qi, Qianni Zhang, Ning Li, Xin Wang, Huiyu Zhou

专题命中医疗多模态：medical image(abstract)；diagnosis(abstract)；分类 cs.CV

2002.03733 2020-02-11 cs.CV 70%

Robust Multimodal Image Registration Using Deep Recurrent Reinforcement Learning

Shanhui Sun, Jing Hu, Mingqing Yao, Jinrong Hu, Xiaodong Yang, Qi Song, Xi Wu

专题命中医疗多模态：medical image(abstract)；CT(abstract)；分类 cs.CV

Journal ref Asian Conference on Computer Vision (ACCV). 2018. 511-526

2509.12234 2025-09-17 cs.LG cs.AI cs.CV eess.IV 69%

Flexible Multimodal Neuroimaging Fusion for Alzheimer's Disease Progression Prediction

Benjamin Burns, Yuan Xue, Douglas W. Scharre, Xia Ning

机构 * Department of Computer Science and Engineering（计算机科学与工程系）； Department of Biomedical Informatics（生物医学信息学系）； Department of Neurology（神经病学系）； Translational Data Analytics Institute（转化数据分析研究所）

专题命中医疗多模态：MRI(abstract)；分类 cs.CV、cs.LG、eess.IV；medical AI(comments)

Comments Accepted at Applications of Medical AI 2025

2207.13150 2025-06-23 physics.med-ph 69%

Multimodal Imaging-based Material Mass Density Estimation for Proton Therapy Using Physics-Constrained Deep Learning

Chih-Wei Chang, Raanan Marants, Yuan Gao, Matthew Goette, Jessica E. Scholey, Jeffrey D. Bradley, Tian Liu, Jun Zhou, Atchar Sudhyadhom, Xiaofeng Yang

专题命中医疗多模态：MRI(abstract)；CT(abstract)；radiology(journal_ref)

Journal ref British Journal of Radiology, Volume 96, Issue 1152, 1 December 2023

2108.00860 2021-08-03 cs.CV cs.LG eess.IV 69%

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

Matthias Keicher, Hendrik Burwinkel, David Bani-Harouni, Magdalini Paschali, Tobias Czempiel, Egon Burian, Marcus R. Makowski, Rickmer Braren, Nassir Navab, Thomas Wendler

专题命中医疗多模态：CT(abstract)；分类 cs.CV、cs.LG、eess.IV；medical image(comments)

Comments 18 pages, 5 figures, submitted to Medical Image Analysis

2107.09892 2021-07-22 eess.IV cs.CE cs.CV cs.LG 69%

Towards Lower-Dose PET using Physics-Based Uncertainty-Aware Multimodal Learning with Robustness to Out-of-Distribution Data

Viswanath P. Sudarshan, Uddeshya Upadhyay, Gary F. Egan, Zhaolin Chen, Suyash P. Awate

专题命中医疗多模态：MRI(abstract)；分类 cs.CV、cs.LG、eess.IV；medical image(comments)

Comments Accepted at Medical Image Analysis

2107.07314 2021-07-16 cs.CV cs.LG eess.IV 69%

Variational Topic Inference for Chest X-Ray Report Generation

Ivona Najdenkoska, Xiantong Zhen, Marcel Worring, Ling Shao

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG、eess.IV；medical image(comments)

Comments To be published in the International Conference on Medical Image Computing and Computer Assisted Intervention 2021

2407.21368 2025-03-18 cs.CV cs.AI cs.CL cs.LG 68%

Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering

Danfeng Guo, Demetri Terzopoulos

专题命中医疗多模态：pathology(abstract)；biomedical(comments,journal_ref)；分类 cs.CV、cs.LG

Comments Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2025:004

Journal ref Machine.Learning.for.Biomedical.Imaging. 3 (2025)

2402.00038 2024-03-18 eess.IV cs.CV cs.LG q-bio.QM 68%

Detecting Brain Tumors through Multimodal Neural Networks

Antonio Curci, Andrea Esposito

专题命中医疗多模态：MRI(abstract)；分类 cs.CV、cs.LG、q-bio

Comments Presented at NeroPRAI 2024 (co-located with ICPRAM 2024). This version did not undergo peer review: refer to the open access version of record (see DOI)

Journal ref Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2024) - NeroPRAI 2024

2606.15910 2026-08-21 cs.CL 版本更新 67%

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

校准的分诊，而非自主：医学视觉-语言模型的置信度估计

Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi

机构 * Michigan State University（密歇根州立大学）

专题命中医疗多模态：pathology(abstract)；radiology(abstract)

AI总结针对医学视觉-语言模型在回答时可能忽略图像而依赖语言先验的问题，提出使用置信度估计进行校准分诊，通过评估七种置信度估计器，发现高置信度区域是区分可用估计器的关键，最佳探针可将错误率从41-45%降至1-4%，但无估计器在所有领域和模型中一致最优。

2608.08071 2026-08-11 cond-mat.mtrl-sci 新提交 67%

Multimodal deep learning framework to predict strain localization of Mg/LPSO two-phase alloys

预测Mg/LPSO两相合金应变局域化的多模态深度学习框架

Daiki Kuriki, Fabien Briffod, Takayuki Shiraiwa, Manabu Enoki

专题命中医疗多模态：CT(abstract,abstract_cn)

AI总结本研究采用多模态深度学习框架，结合三种微观结构描述符，从三维图像预测Mg/LPSO两相合金的压缩局域应变分布，验证了方法的有效性。

Comments Published in Acta Materialia, Volume 281, 120398 (2024)

Journal ref Acta Materialia 281 (2024) 120398

2607.18695 2026-08-11 cs.CV cs.AI cs.LG eess.IV 版本更新 67%

Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models

属性应来自图像，而非类名：视觉语言模型的分布条件属性选择

Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin, Deepak Pathak, John Galeotti, Deva Ramanan

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG、eess.IV

AI总结研究视觉语言模型可解释零样本分类，指出基于类名的描述符视觉证据不足。提出从目标图像集选属性的方法，该方法提升了准确率，性能优于CoOp，用时短，所选属性还能描述数据分布，是一种有效的分布条件属性选择策略。

Comments Accepted at the PFATCV Workshop, ECCV 2026. Project page: https://ggare-cmu.github.io/AttributeSelect/

2607.15565 2026-07-20 cs.CV cs.AI cs.LG eess.IV 新提交 67%

Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models

问两次，看两次：提示回声解决视觉语言模型中的问题优先悖论

Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG、eess.IV

AI总结研究视觉语言模型中问题优先提示的悖论，发现问题前置虽能引导感知但后续答案生成阶段难关注到问题。提出问题回声方法，即在图像两侧重复问题，解决了悖论，在多基准测试中表现出色且无需训练等。

2508.03750 2026-07-07 cs.LG cs.CE cs.CV eess.IV 版本更新 67%

GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

GlaBoost：用于青光眼风险分层的多模态结构化框架

Cheng Huang, Zeyu Han, Weizheng Xie, Karanjit Kooner, Tsengdar Lee, Jui-Kai Wang, Jia Zhang

机构 * Department of Computer Science, Southern Methodist University（计算机科学系，南方 Methodist 大学）； Department of Ophthalmology, UT Southwestern Medical Center（眼科学系，UT 南方医学中心）； High Performance Computing Program, National Aeronautics and Space Administration（高性能计算计划，国家航空航天局）

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG、eess.IV

AI总结研究针对青光眼检测，提出多模态梯度提升框架GlaBoost，统一眼底图像嵌入、文本评估及生物标志物三信号预测风险，融合后用增强XGBoost分类，性能超基线且可解释。

Comments Accepted by IEEE 48th EMBC (2026)

2606.28556 2026-06-30 cs.AI 67%

IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations

IMCBench：面向多模态大语言模型在图像基础医疗对话中的基准测试

Maria Xenochristou, Ashutosh Joshi, Korosh Vatanparvar, Mohammad Abuzar Hashemi, Prasad Kasu, Deepak Bansal, Anchal Nema, Nivedita Wadhwa, Prashams S Jain, Rebecca Abraham, Will Kimbrough, Dilek Hakkani-Tur, Wilko Schulz-Mahlendorf

机构 * Amazon Health AI（亚马逊健康AI）

专题命中医疗多模态：medical AI(abstract)；diagnosis(abstract)

AI总结提出IMCBench基准，结合临床图像与合成患者档案模拟多轮医疗对话，从安全性、准确性和不确定性使用三个维度评估模型，发现准确描述不等于安全指导。

Comments Accepted at ECML PKDD 2026. 22 pages, 2 figures

1. 医疗多模态 972 篇

Visual question answering: from early developments to recent advances -- a survey

Toward Robust Early Detection of Alzheimer's Disease via an Integrated Multimodal Learning Approach

MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinement

Large Language Models for Multimodal Deformable Image Registration

A Refer-and-Ground Multimodal Large Language Model for Biomedicine

Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology

Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imaging

Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering

UIT-Saviors at MEDVQA-GI 2023: Improving Multimodal Learning with Image Enhancement for Gastrointestinal Visual Question Answering

Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training

ConTEXTual Net: A Multimodal Vision-Language Model for Segmentation of Pneumothorax

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Multimodal Gait Recognition for Neurodegenerative Diseases

Robust Multimodal Image Registration Using Deep Recurrent Reinforcement Learning

Flexible Multimodal Neuroimaging Fusion for Alzheimer's Disease Progression Prediction

Multimodal Imaging-based Material Mass Density Estimation for Proton Therapy Using Physics-Constrained Deep Learning

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

Towards Lower-Dose PET using Physics-Based Uncertainty-Aware Multimodal Learning with Robustness to Out-of-Distribution Data

Variational Topic Inference for Chest X-Ray Report Generation

Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering

Detecting Brain Tumors through Multimodal Neural Networks

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

Multimodal deep learning framework to predict strain localization of Mg/LPSO two-phase alloys

Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models

Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models

GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations