医学 AI

Robust Heartbeat Detection from Multimodal Data via CNN-based Generalizable Information Fusion

B S Chandra, C S Sastry, S Jana

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV、eess.SP

Label-driven weakly-supervised learning for multimodal deformable image registration

Yipeng Hu, Marc Modat, Eli Gibson, Nooshin Ghavami, Ester Bonmati, Caroline M. Moore, Mark Emberton, J. Alison Noble, Dean C. Barratt, Tom Vercauteren

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ISBI 2018

Developing a comprehensive framework for multimodal feature extraction

Quinten McNamara, Alejandro de la Vega, Tal Yarkoni

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.CV、cs.LG

Pathologist Attention-Aligned Report Generation for Prostate Histopathology

用于前列腺组织病理学的病理学家注意力对齐报告生成

Ruoyu Xue, Suryakant Singh, Souradeep Chakraborty, Pierre Marza, Oksana Yaskiv, Constantin Friedman, Natallia Sheuka, Paul Friedman, Bharat Ramlal, Beatrice Knudsen, Rajarsi Gupta, Joel Saltz, Prateek Prasanna, Gregory Zelinsky, Dimitris Samaras

机构 * Department of Computer Science, Stony Brook University（纽约州立大学石溪分校计算机科学系） ； Department of Biomedical Informatics, Stony Brook University（纽约州立大学石溪分校生物医学信息学系） ； Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit（巴黎萨克雷大学、中央理工高等电力学院、古斯塔夫·鲁西研究所、法国国家健康与医学研究院、PRISM综合大学医院、癌症数据科学单元） ； Université Paris-Saclay, CentraleSupélec, MICS Laboratory（巴黎萨克雷大学、中央理工高等电力学院、MICS实验室） ； Department of Pathology and Laboratory Medicine, Northwell Health Laboratories（诺斯韦尔健康实验室病理与检验医学部） ； Department of Pathology, University of Utah School of Medicine（犹他大学医学院病理系） ； Department of Psychology, Stony Brook University（纽约州立大学石溪分校心理学系） 纠错

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV；medical image(comments)

AI总结 研究将人类注意力引入前列腺病理报告生成模型训练，收集多模态数据集，通过注意力对齐损失微调模型，在两个模型上评估，在报告生成和视觉问答任务中取得较好效果，模型注意力图与病理学家注意力更对齐。

Comments 11 pages, 4 figures, accepted for publication at the 29th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2026)

Why Text Prevails: Vision May Undermine Multimodal Medical Decision Making

为何文本占上风：视觉可能损害多模态医疗决策制定

Siyuan Dai, Lunxiao Li, Kun Zhao, Eardi Lila, Paul K. Crane, Heng Huang, Dongkuan Xu, Haoteng Tang, Liang Zhan

机构 * University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德谷大学） ； University of Pittsburgh（匹兹堡大学） ； NC State University（北卡罗来纳州立大学） ； University of Washington（华盛顿大学） ； University of Maryland（马里兰大学） 纠错

专题命中 医疗多模态 ：biomedical(abstract,comments)；分类 cs.CV

AI总结 本研究发现文本推理在医疗多模态决策中优于多模态输入，提出三种策略以提升多模态医疗决策能力。

Comments Accepted by ICDM 2025 the Workshop on Synergy of AI and Multimodal Biomedical Data Mining

On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?

Raza Imam, Rufael Marew, Mohammad Yaqub

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·穆萨大学人工智能学院） 纠错

专题命中 医疗多模态 ：medical image(abstract,comments)；分类 cs.CV

Comments Dataset and Code is available at https://github.com/BioMedIA-MBZUAI/RobustMedCLIP Accepted at: Medical Image Understanding and Analysis (MIUA) 2025

A Large-scale Multimodal Study for Predicting Mortality Risk Using Minimal and Low Parameter Models and Separable Risk Assessment

Alvaro E. Ulloa Cerna, Marios Pattichis, David P. vanMaanen, Linyuan Jing, Aalpen A. Patel, Joshua V. Stough, Christopher M. Haggerty, Brandon K. Fornwalt

机构 * Department of Translational Data Science and Informatics, Geisinger（转化数据科学与信息学部门，Geisinger） ； Department of Electrical and Computer Engineering, University of New Mexico（电气与计算机工程系，新墨西哥大学） 纠错

专题命中 医疗多模态 ：biomedical(abstract,journal_ref)；分类 cs.LG

Journal ref IEEE Journal of Biomedical and Health Informatics, 2025

MedFILIP: Medical Fine-grained Language-Image Pre-training

Xinjie Liang, Xiangyu Li, Fanding Li, Jie Jiang, Qing Dong, Wei Wang, Kuanquan Wang, Suyu Dong, Gongning Luo, Shuo Li

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV；biomedical(comments)

Comments 10 pages, 5 figures, IEEE Journal of Biomedical and Health Informatics 2025

A Hybrid Artificial Intelligence System for Automated EEG Background Analysis and Report Generation

Chin-Sung Tung, Sheng-Fu Liang, Shu-Feng Chang, Chung-Ping Young

专题命中 医疗多模态 ：diagnosis(abstract)；分类 eess.SP；biomedical(journal_ref)

Comments Example code available at https://github.com/tcs211/AI_EEEG_REPORT

Journal ref IEEE Journal of Biomedical and Health Informatics (2024)

A reinforcement learning approach for VQA validation: an application to diabetic macular edema grading

Tatiana Fountoukidou, Raphael Sznitman

专题命中 医疗多模态 ：medical image(abstract,journal_ref)；分类 cs.CV

Comments 16 pages (+ 23 pages supplementary material)

Journal ref Medical image analysis 87 (2023): 102822

Integrated Construction of Multimodal Atlases with Structural Connectomes in the Space of Riemannian Metrics

Kristen M. Campbell, Haocheng Dai, Zhe Su, Martin Bauer, P. Thomas Fletcher, Sarang C. Joshi

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.CV；biomedical(comments)

Comments Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://www.melba-journal.org/papers/2022:016.html. arXiv admin note: substantial text overlap with arXiv:2103.05730

Speech Map: A Statistical Multimodal Atlas of 4D Tongue Motion During Speech from Tagged and Cine MR Images

Jonghye Woo, Fangxu Xing, Maureen Stone, Jordan Green, Timothy G. Reese, Thomas J. Brady, Van J. Wedeen, Jerry L. Prince, Georges El Fakhri

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.CV；biomedical(comments)

Comments Accepted at Journal of Computer Methods in Biomechanics and Biomedical Engineering

Multimodal Higher-Order Brain Networks: A Topological Signal Processing Perspective

多模态高阶脑网络：拓扑信号处理视角

Breno C. Bispo, Stefania Sardellitti, Juliano B. Lima, Fernando A. N. Santos

专题命中 医疗多模态 ：MRI(abstract)；分类 q-bio、eess.SP

AI总结 本文提出基于拓扑信号处理的多模态框架，通过高阶拓扑域建模脑部，利用扩散MRI和静息态fMRI学习个体化脑细胞复合体，揭示高阶相互作用的拓扑特征及其与行为的关联。

Comments This paper has been sumbmitted to IEEE Transactions on Medical Imaging (TMI), March 2026

BrainSymphony: A parameter-efficient multimodal foundation model for brain dynamics with limited data

BrainSymphony: 一种参数高效、多模态的基础模型，用于在有限数据下的脑动态

Moein Khajehnejad, Forough Habibollahi, Devon Stoliker, Adeel Razi

机构 * Turner Institute for Brain and Mental Health（大脑与心理健康Turner研究所） ； School of Psychological Sciences, Monash University（墨尔本大学心理学科学学院） ； Cortical Labs（皮层实验室） ； CIFAR Azrieli Global Scholars Program（CIFAR阿兹里埃利全球学者计划） 纠错

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.LG、q-bio

AI总结 BrainSymphony是一种参数高效、多模态的基础模型，通过整合fMRI和扩散MRI数据，实现有限数据下的脑动态分析，优于更大模型并提升神经科学应用。

Comments 32 pages, 14 figures

Analyzing heterogeneity in Alzheimer Disease using multimodal normative modeling on imaging-based ATN biomarkers

利用多模态规范建模分析阿尔茨海默病的异质性：基于影像学ATN生物标志物

Sayantan Kumar, Tom Earnest, Braden Yang, Deydeep Kothapalli, Andrew J. Aschenbrenner, Jason Hassenstab, Chengie Xiong, Beau Ances, John Morris, Tammie L. S. Benzinger, Brian A. Gordon, Philip Payne, Aristeidis Sotiras

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.LG、q-bio

AI总结 本研究利用多模态规范建模分析阿尔茨海默病影像学ATN生物标志物的异质性，揭示了疾病严重程度与认知功能的关系。

Comments Under review in Alzheimer's & Dementia

Journal ref Alzheimer's Dement. 2025; 21:e70143

An interpretable generative multimodal neuroimaging-genomics framework for decoding Alzheimer's disease

可解释的生成多模态神经影像-基因组框架用于解码阿尔茨海默病

Giorgio Dolci, Federica Cruciani, Md Abdur Rahaman, Anees Abrol, Jiayu Chen, Zening Fu, Ilaria Boscolo Galazzo, Gloria Menegaz, Vince D. Calhoun

机构 * Department of Computer Science, University of Verona（威尼斯大学计算机科学系） ； Department of Engineering for Innovation Medicine, University of Verona（威尼斯大学创新医学工程系） ； Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University（神经影像与数据科学转化研究三机构中心（TReNDS），佐治亚州立大学，佐治亚理工学院，埃默里大学） 纠错

专题命中 医疗多模态 ：MRI(abstract)；分类 q-bio、eess.IV

AI总结 本文提出了一种可解释的生成多模态神经影像-基因组框架，用于解码阿尔茨海默病，通过多模态数据和单核苷酸多态性实现AD检测和MCI预测，并揭示了与疾病相关的生物学机制。

Comments 33 pages, 8 figures (main text + supplementary materials), submitted to a journal

Journal ref J. Neural Eng. 22 056021 (2025)

Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model

Jalil Jalili, Yashraj Gavhane, Evan Walker, Anna Heinke, Christopher Bowd, Akram Belghith, Massimo A. Fazio, Christopher A. Girkin, C. Gustavo De Moraes, Jeffrey M. Liebmann, Sally L. Baxter, Robert N. Weinreb, Linda M. Zangwill, Mark Christopher

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV、q-bio

Platform for Representation and Integration of multimodal Molecular Embeddings

Erika Yilin Zheng, Yu Yan, Baradwaj Simha Sankar, Ethan Ji, Steven Swee, Irsyad Adam, Ding Wang, Alexander Russell Pelletier, Alex Bui, Wei Wang, Peipei Ping

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.LG、q-bio

Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learning

Melda Yeghaian, Zuhir Bodalal, Daan van den Broek, John B A G Haanen, Regina G H Beets-Tan, Stefano Trebeschi, Marcel A J van Gerven

专题命中 医疗多模态 ：CT(abstract)；分类 cs.LG、q-bio

Journal ref Journal of the American Medical Informatics Association, 2025;, ocaf074

GenDMR: A dynamic multimodal role-swapping network for identifying risk gene phenotypes

Lina Qin, Cheng Zhu, Chuqi Zhou, Yukun Huang, Jiayi Zhu, Ping Liang, Jinju Wang, Yixing Huang, Cheng Luo, Dezhong Yao, Ying Tan

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG、q-bio

Comments 31 pages, 9 figures

Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Research

Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, W. Jim Zheng, Hongyu Zhao

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.LG、q-bio

Comments 8 pages

Journal ref EMNLP 2024

Deep multimodal saliency parcellation of cerebellar pathways: linking microstructure and individual function through explainable multitask learning

Ari Tchetchenian, Leo Zekelman, Yuqian Chen, Jarrett Rushmore, Fan Zhang, Edward H. Yeterian, Nikos Makris, Yogesh Rathi, Erik Meijering, Yang Song, Lauren J. O'Donnell

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.LG、q-bio

MolFM: A Multimodal Molecular Foundation Model

Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, Zaiqing Nie

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.LG、q-bio

Comments 31 pages, 15 figures, and 15 tables

Multimodal Learning for Multi-Omics: A Survey

Sina Tabakhi, Mohammod Naimul Islam Suvon, Pegah Ahadian, Haiping Lu

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG、q-bio

Comments 52 pages, 3 figures; Revised matrix factorization fusion section

A multimodal approach for Parkinson disease analysis

Marcos Faundez-Zanuy, Antonio Satue-Villar, Jiri Mekyska, Viridiana Arreola, Pilar Sanz, Carles Paul, Luis Guirao, Mateu Serra, Laia Rofes, Pere Clavé, Enric Sesa-Nogueras, Josep Roure

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG、q-bio

Comments 10 pages

Journal ref In: Bassis S., Esposito A., Morabito F. (eds) Advances in Neural Networks: Computational and Theoretical Issues. Smart Innovation, Systems and Technologies, vol 37. Springer, Cham. 2015

Next Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Approaches

Isaac Shiri, Hassan Maleki, Ghasem Hajianfar, Hamid Abdollahi, Saeed Ashrafinia, Mathieu Hatt, Mehrdad Oveisi, Arman Rahmim

专题命中 医疗多模态 ：CT(abstract)；分类 cs.LG、q-bio

Comments 42 pages,3 Figures,4 Tables, 13 Supplemental Figures, 11 Supplemental Tables

Multimodal Brain Visualization

Saad Nadeem, Arie Kaufman

专题命中 医疗多模态 ：MRI(abstract)；分类 q-bio；biomedical(comments)

Comments SPIE Medical Imaging 2016, Proc. SPIE Medical Imaging: Biomedical Applications in Molecular, Structural, and Functional Imaging, 2016

Journal ref SPIE Medical Imaging, pp. 97881Y-97881Y. 2016

Kinematic Knowledge Maps for Pattern Alignment: Structured Latent Representational Learning in Multimodal Gait Analysis

用于模式对齐的运动学知识图谱：多模态步态分析中的结构化潜在表示学习

Chen Dong, He Zonglin, Cheung Kenneth M.C

专题命中 医疗多模态 ：clinical AI(abstract)；分类 cs.CV

AI总结 本研究提出ScoliDetect框架，通过运动学知识图谱（KKM）实现多模态步态分析，其介导的融合结合三模态对比预训练提升了脊柱侧凸筛查的泛化性与可解释性，外部ROC-AUC达0.972。

How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQA

视觉语言模型如何失败？组合式视觉问答中的视觉-操作不对齐

Navya Gupta, Bingjie Xu, Avinash Anand, Timothy Liu, Zhengchen Zhang

机构 * Singapore Institute of Technology（新加坡科技学院） ； NVIDIA（英伟达） 纠错

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

AI总结 研究组合式视觉问答中视觉语言模型失败的机制，引入以操作为中心的框架分解失败模式，揭示四种失败模式及传播路径，表明不同失败类型需不同纠正策略，为提升模型可靠性提供基础。

Comments Accepted at ACM Multimedia 2026

Cytoarchitecture in Words: Weakly Supervised Vision-Language Modeling for Human Brain Microscopy

词中结构：用于人类大脑显微镜的弱监督视觉-语言建模

Matthew Sutton, Katrin Amunts, Timo Dickscheid, Christian Schiffer

机构 * Institute of Neuroscience and Medicine (INM-1), Research Centre Jülich（神经科学与医学研究所（INM-1），焦耳研究中心） ； Helmholtz AI, Research Centre Jülich（海德堡人工智能研究所，焦耳研究中心） ； Institute for Brain Research, University Hospital Düsseldorf（脑研究所在杜塞尔多夫大学医院） ； Computer Vision, Institute for Computational Visualistics, University of Koblenz（计算机视觉，计算视觉研究所，科布伦茨大学） 纠错

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.CV

AI总结 本研究提出了一种弱监督视觉-语言建模方法，通过标签介导生成人类大脑显微镜的区域描述，实现了在缺乏配对标注情况下的自然语言生成。

Comments 13 pages, 5 figures, accepted for inclusion at GCPR 2026