We study multimodal survival analysis integrating clinical text, tabular covariates, and genomic profiles using locally deployable large language models (LLMs). As many institutions face tight computational and privacy constraints, this setting motivates the use of lightweight, on-premises models. Our approach jointly estimates calibrated survival probabilities and generates concise, evidence-grounded prognosis text via teacher-student distillation and principled multimodal fusion. On a TCGA cohort, it outperforms standard baselines, avoids reliance on cloud services and associated privacy concerns, and reduces the risk of hallucinated or miscalibrated estimates that can be observed in base LLMs.
RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistants
Singapore University of Technology and Design（新加坡科技设计大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）；Chongqing University（重庆大学）
The deep supervised and reinforcement learning paradigms (among others) have the potential to endow interactive multimodal social robots with the ability of acquiring skills autonomously. But it is still not very clear yet how they can be best deployed in real world applications. As a step in this direction, we propose a deep learning-based approach for efficiently training a humanoid robot to play multimodal games---and use the game of `Noughts & Crosses' with two variants as a case study. Its minimum requirements for learning to perceive and interact are based on a few hundred example images…
EsurvFusion: An evidential multimodal survival fusion model based on Gaussian random fuzzy numbers
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学陈笃生公共卫生学院）；Université de Rouen Normandie（鲁昂诺曼底大学）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究院）
EAGLE: Efficient Alignment of Generalized Latent Embeddings for Multimodal Survival Prediction with Interpretable Attribution Analysis
Dept. of Machine Learning Moffitt Cancer Center（机器学习系莫菲特癌症中心）；Dept. of Cancer Epidemiology Moffitt Cancer Center（癌症流行病学系莫菲特癌症中心）；Dept. of Electrical Engineering University of South Florida（电气工程系佛罗里达州立大学）
SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与通信工程学院）；The Second Xiangya Hospital, Central South University（中南大学湘雅二医院）
XAI-CLIP: ROI-Guided Perturbation Framework for Explainable Medical Image Segmentation in Multimodal Vision-Language Models
King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学）；Massachusetts Institute of Technology（麻省理工学院）；Imperial College London（伦敦帝国学院）；KFUPM-SDAIA Joint Research Centre for Artificial Intelligence（KFUPM-SDAIA联合人工智能研究中心）
Machine learning-based multimodal prognostic models integrating pathology images and high-throughput omic data for overall survival prediction in cancer: a systematic review
Leeds Teaching Hospitals NHS Trust（利兹教学医院国民医疗服务信托）；University of Leeds（利兹大学）；Linköping University（林雪平大学）
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
When oncologists estimate cancer patient survival, they rely on multimodal data. Even though some multimodal deep learning methods have been proposed in the literature, the majority rely on having two or more independent networks that share knowledge at a later stage in the overall model. On the other hand, oncologists do not do this in their analysis but rather fuse the information in their brain from multiple sources such as medical images and patient history. This work proposes a deep learning method that mimics oncologists' analytical behavior when quantifying cancer and estimating patient…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；IBM Research（IBM研究院）；VinUniversity（文大）
Effective understanding of a disease such as cancer requires fusing multiple sources of information captured across physical scales by multimodal data. In this work, we propose a novel feature embedding module that derives from canonical correlation analyses to account for intra-modality and inter-modality correlations. Experiments on simulated and real data demonstrate how our proposed module can learn well-correlated multi-dimensional embeddings. These embeddings perform competitively on one-year survival classification of TCGA-BRCA breast cancer patients, yielding average F1 scores up to 58…
Genentech（基因泰克）；University of Central Florida（中佛罗里达大学）
The integration of multi-modal data, such as pathological images and genomic data, is essential for understanding cancer heterogeneity and complexity for personalized treatments, as well as for enhancing survival predictions. Despite the progress made in integrating pathology and genomic data, most existing methods cannot mine the complex inter-modality relations thoroughly. Additionally, identifying explainable features from these models that govern preclinical discovery and clinical prediction is crucial for cancer diagnosis, prognosis, and therapeutic response studies. We propose PONET- a n…
ConSurv: Multimodal Continual Learning for Survival Analysis
The Chinese University of Hong Kong（香港中文大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology（香港科技大学）
Multimodal Deep Survival Analysis for Sinkhole Susceptibility
Huron High School（休伦高中）；University of Florida（佛罗里达大学）；Industrial and Systems Engineering（工业与系统工程系）；Forest, Fisheries, and Geomatics Sciences（森林、渔业与地理信息科学系）
Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learning
Radboud University（拉德堡德大学）；The Netherlands Cancer Institute（荷兰癌症研究所）；Maastricht University（马斯特里赫特大学）；Oncode Institute（Oncode研究所）；Leiden University Medical Center（莱顿大学医学中心）；Centre Hospitalier Universitaire Vaudois（沃州大学医院）；University of Southern Denmark（南丹麦大学）
Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastoma
University Hospital Erlangen（埃尔朗根大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN（埃尔朗根-欧洲大都会区综合癌症中心）；The Bavarian Cancer Research Center（巴伐利亚癌症研究中心）；FAU Profile Center Immunomedicine（FAU免疫医学特色中心）
Multimodal Deep Learning for Prediction of Progression-Free Survival in Patients with Neuroendocrine Tumors Undergoing 177Lu-based Peptide Receptor Radionuclide Therapy
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能部门，弗劳恩霍夫海因里希·赫兹研究所）；Department of Nuclear Medicine, Charité—Universitätsmedizin Berlin（核医学部门，柏林查理医院）；Department of Hepatology and Gastroenterology, Charité—Universitätsmedizin Berlin（肝病与胃肠病学部门，柏林查理医院）；Division of Interventional Radiology, Department of Radiology, Memorial Sloan Kettering Cancer Center（介入放射学部门，纪念斯隆-凯特琳癌症中心）；Department of Endocrinology and Metabolism, Charité—Universitätsmedizin Berlin（内分泌与代谢学部门，柏林查理医院）；Clinic for Gastroenterology, Hepatology and Infectious Diseases, University Hospital Düsseldorf, Medical Faculty of Heinrich Heine University Düsseldorf（胃肠病、肝病和传染病诊所，杜塞尔多夫大学医院，海因里希·海涅大学医学部）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学部门，柏林技术大学）；Berlin Institute of Health at Charité – Universitätsmedizin Berlin（柏林查理医院健康研究所）
When Does Multimodal Learning Help in Healthcare? A Benchmark on EHR and Chest X-Ray Fusion
Department of Computer Science（计算机科学系）；Hong Kong Baptist University（香港 Baptist 大学）；Division of Artificial Intelligence（人工智能 division）；Lingnan University（Lingnan 大学）；School of Nursing（护理学院）；The Hong Kong Polytechnic University（香港理工大学）
Multimodal Cancer Modeling in the Age of Foundation Model Embeddings
Center for Translational Data Science（转化数据科学中心）；Department of Computer Science（计算机科学系）；University of Chicago（芝加哥大学）；Medical Scientist Training Program（医学科学家培训计划）；Brown University（布朗大学）；Section of Biomedical Data Science（生物医学数据科学部门）；Department of Medicine（医学系）
