BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma
Department of Radiology \& Biomedical Imaging, Department of Biomedical Engineering, Department of Electrical Engineering, Department of Statistics \& Data Science Yale University, New Haven, CT, 06510, USA
Junlin Yang, Tian Yu, Nicha C. Dvornek, Yuexi Du, Peiyu Duan, Annabella Shewarega, Lawrence H. Staib, James S. Duncan, Julius Chapiro
Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic factors; thus, similar survival outcomes may reflect fundamentally different underlying biological processes. Prognostic modeling in HCC is informed by rich multimodal information from multiparametric MRI and radiology reports from routine clinical practice. Existing prognostic vision-language models (VLMs) learn a single entangled latent representation that blends hepatic and tumor-related factors, limiting both accuracy and biological interpretability. We present BioFact-MoE, a biologically factorized Mixture of Experts (MoE) framework that explicitly decomposes liver and tumor factors via biologically supervised experts within a residual MoE survival architecture. On a HCC cohort of N=588 patients (pretrained on 4,582 3D MRI image-report pairs), BioFact-MoE consistently improves survival prediction over all baselines across time horizons, achieving 12-, 18-, and 24-month AUCs of 75.33%, 75.85%, and 73.96%. Beyond scalar risk prediction, gated expert weights enable phenotype-aware risk stratification. Pathway-informed gating uncovers clinically meaningful treatment-associated survival heterogeneity. In held-out validation, hepatic and tumor embeddings show selective associations with liver function and tumor burden markers, respectively (p<0.05), without supervision. The code is available at https://github.com/jy-639/BioFact-MoE.
A Retrospective Systematic Study on Hierarchical Sparse Query Transformer-assisted Ultrasound Screening for Early Hepatocellular Carcinoma
Northwestern Polytechnical University（西北工业大学）；The First Affiliated Hospital of Sun Yat-Sen University（中山大学附属第一医院）；The Seventh Affiliated Hospital of Sun Yat-Sen University（中山大学附属第七医院）；The First Affiliated Hospital of Guangxi Medical University（广西医科大学附属第一医院）；Xi’an Jiaotong University（西安交通大学）
A Hierarchical Transformer Encoder to Improve Entire Neoplasm Segmentation on Whole Slide Image of Hepatocellular Carcinoma
Université Paris Cité（巴黎西岱大学）；University of Applied Sciences of Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））；French University Institute (IUF)（法国大学研究院）
针对肝细胞癌 WSI 全肿瘤分割难题，提出分层 Transformer 架构 HiTrans，在 4096×4096 图像块上学习全局依赖并提升分割性能。
Mask-guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma
Universidad Industrial de Santander（桑坦德工业大学）；University of Toulouse（图卢兹大学）；Université Paul Sabatier Toulouse 3（图卢兹第三大学（保罗·萨巴蒂埃大学））；Bicêtre University Hospital（比塞特尔大学医院）；Assistance publique des hôpitaux de Paris（巴黎公共援助医院）；Université Claude Bernard Lyon 1（里昂第一大学（克洛德·贝尔纳里昂第一大学））
Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）；PAII Inc.（PAII公司）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
Hepatocellular carcinoma (HCC) can be potentially discovered from abdominal computed tomography (CT) studies under varied clinical scenarios, e.g., fully dynamic contrast enhanced (DCE) studies, non-contrast (NC) plus venous phase (VP) abdominal studies, or NC-only studies. We develop a flexible three-dimensional deep algorithm, called hetero-phase volumetric detection (HPVD), that can accept any combination of contrast-phase inputs and with adjustable sensitivity depending on the clinical purpose. We trained HPVD on 771 DCE CT scans to detect HCCs and tested on external 164 positives and 206 …
Hybrid deep learning-based strategy for the hepatocellular carcinoma cancer grade classification of H&E stained liver histopathology images
Visvesvaraya National Institute of Technology（维斯瓦拉亚国家理工学院）；AIIMS Nagpur（那格浦尔全印度医学科学研究所）；Indian Institute of Technology Mandi（曼迪印度理工学院）；Universitat de Barcelona（巴塞罗那大学）
A Polarization and Radiomics Feature Fusion Network for the Classification of Hepatocellular Carcinoma and Intrahepatic Cholangiocarcinoma
University College London（伦敦大学学院）；Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Fujian Medical University Cancer Hospital（福建医科大学肿瘤医院）；Fujian Cancer Hospital（福建省肿瘤医院）
Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma by Integrating Multimodal Ultrasound and Clinical Data: A Multicenter Study
Shenzhen University（深圳大学）；Peking University Shenzhen Hospital（北京大学深圳医院）；Chinese PLA General Hospital（中国人民解放军总医院）
Technische Universität Berlin（柏林工业大学）；Aignostics GmbH（Aignostics 公司）；Charité Universitätsmedizin Berlin（柏林夏里特医学院）；Berlin Institute of Health at Charité – Universitätsmedizin Berlin（柏林健康研究所（夏里特医学院））；University Hospital Düsseldorf（杜塞尔多夫大学医院）；Ludwig-Maximilians-Universität München（慕尼黑大学）；German Cancer Research Center（德国癌症研究中心）
In this paper, we present a deep learning segmentation approach to classify and quantify the two most prevalent primary liver cancers - hepatocellular carcinoma and intrahepatic cholangiocarcinoma - from hematoxylin and eosin (H&E) stained whole slide images. While semantic segmentation of medical images typically requires costly pixel-level annotations by domain experts, there often exists additional information which is routinely obtained in clinical diagnostics but rarely utilized for model training. We propose to leverage such weak information from patient diagnoses by deriving complementa…
Hepatocellular carcinoma (HCC) is a leading cause of cancer-related death worldwide. Whole-slide imaging which is a method of scanning glass slides have been employed for diagnosis of HCC. Using high resolution Whole-slide images is infeasible for Convolutional Neural Network applications. Hence tiling the Whole-slide images is a common methodology for assigning Convolutional Neural Networks for classification and segmentation. Determination of the tile size affects the performance of the algorithms since small field of view can not capture the information on a larger scale and large field of …
Yale University（耶鲁大学）；University of Washington（华盛顿大学）
This work proposes a pipeline to predict treatment response to intra-arterial therapy of patients with Hepatocellular Carcinoma (HCC) for improved therapeutic decision-making. Our graph neural network model seamlessly combines heterogeneous inputs of baseline MR scans, pre-treatment clinical information, and planned treatment characteristics and has been validated on patients with HCC treated by transarterial chemoembolization (TACE). It achieves Accuracy of $0.713 \pm 0.075$, F1 of $0.702 \pm 0.082$ and AUC of $0.710 \pm 0.108$. In addition, the pipeline incorporates uncertainty estimation to…
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The First Affiliated Hospital of Jinan University（暨南大学附属第一医院）
Automatic segmentation of hepatocellular carcinoma (HCC) in Digital Subtraction Angiography (DSA) videos can assist radiologists in efficient diagnosis of HCC and accurate evaluation of tumors in clinical practice. Few studies have investigated HCC segmentation from DSA videos. It shows great challenging due to motion artifacts in filming, ambiguous boundaries of tumor regions and high similarity in imaging to other anatomical tissues. In this paper, we raise the problem of HCC segmentation in DSA videos, and build our own DSA dataset. We also propose a novel segmentation network called DSA-LT…
Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局资讯通信研究院）；National University of Singapore（新加坡国立大学）；Singapore General Hospital（新加坡中央医院）；National Cancer Centre（新加坡国立癌症中心）
Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer and the fourth most common cause of cancer-related death worldwide. Understanding the underlying gene mutations in HCC provides great prognostic value for treatment planning and targeted therapy. Radiogenomics has revealed an association between non-invasive imaging features and molecular genomics. However, imaging feature identification is laborious and error-prone. In this paper, we propose an end-to-end deep learning framework for mutation prediction in APOB, COL11A1 and ATRX genes using multiphasic CT scans. Con…
A baseline for machine-learning-based hepatocellular carcinoma diagnosis using multi-modal clinical data
Heidelberg University（海德堡大学）；Mannheim Institute for Intelligent Systems in Medicine（曼海姆医学智能系统研究所）；UMM Mannheim（曼海姆大学医学中心）；Mannheim Medical Faculty（曼海姆医学院）；Clinic for Radiology and Nuclear Medicine（放射科与核医学科）；Interdisciplinary Center for Scientific Computing（跨学科科学计算中心）；Central Institute for Computer Engineering（中央计算机工程研究所）；Heidelberg Center for Model-Based AI, Data Analysis and Modeling in Medicine（海德堡基于模型的人工智能、医学数据分析与建模中心）；School of Information Engineering, Nanjing University of Finance and Economics（南京财经大学信息工程学院）
Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine（浙江大学医学院附属杭州市第一人民医院）；Syracuse University（雪城大学）；Zhejiang University（浙江大学）；The First Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第一医院）
This study aimed to develop a classifier of prognosis after resection or liver transplantation (LT) for HCC by directly analysing the ubiquitously available histological images using deep learning based neural networks. Nucleus map set was used to train U-net to capture the nuclear architectural information. Train set included the patients with HCC treated by resection and has a distinct outcome. LT set contained patients with HCC treated by LT. Train set and its nuclear architectural information extracted by U-net were used to train MobileNet V2 based classifier (MobileNetV2_HCC_Class), purpo…
Shandong Normal University（山东师范大学）；Western University（韦仕敦大学）
Simultaneous segmentation and detection of liver tumors (hemangioma and hepatocellular carcinoma (HCC)) by using multi-modality non-contrast magnetic resonance imaging (NCMRI) are crucial for the clinical diagnosis. However, it is still a challenging task due to: (1) the HCC information on NCMRI is invisible or insufficient makes extraction of liver tumors feature difficult; (2) diverse imaging characteristics in multi-modality NCMRI causes feature fusion and selection difficult; (3) no specific information between hemangioma and HCC on NCMRI cause liver tumors detection difficult. In this stu…
Sun Yat-sen University（中山大学）；The First Affiliated Hospital, Sun Yat-sen University（中山大学附属第一医院）；Infervision Inc.（推想医疗科技股份有限公司）
Hepatocellular carcinoma (HCC) is the second most frequent cause of malignancy-related death and is one of the diseases with the highest incidence in the world. Because the liver is the only organ in the human body that is supplied by two major vessels: the hepatic artery and the portal vein, various types of malignant tumors can spread from other organs to the liver. And due to the liver masses' heterogeneous and diffusive shape, the tumor lesions are very difficult to be recognized, thus automatic lesion detection is necessary for the doctors with huge workloads. To assist doctors, this work…
Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Images
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（人工智能学院，北京邮电大学）；Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（pathology department, 国家癌症中心/国家癌症临床研究中心/癌症医院, 中国医学科学院和北京协和医学院）
GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC)（生物医学工程学院，生命科学与医学系，中国科学技术大学）；Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE)（医学影像、机器人、分析计算与学习中心）；Suzhou Institute for Advanced Research, USTC（苏州先进研究院，中国科学技术大学）；Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC（放射科，中国科学技术大学第一附属医院，生命科学与医学系，中国科学技术大学）；T Magnetic Resonance Translational Medicine Research Center, Department of Radiology, The First Affiliated Hospital (Southwest Hospital) of Army Medical University（7T磁共振转化医学研究中心，放射科，中国医学大学第一附属医院（西南医院））
Pathobiological Dictionary Defining Pathomics and Texture Features: Addressing Understandable AI Issues in Personalized Liver Cancer; Dictionary Version LCP1.0
University of British Columbia（不列颠哥伦比亚大学）；BC Cancer Research Institute（BC癌症研究所）；TECVICO Corp.（TECVICO公司）；Tehran University of Medical Sciences（德黑兰医科大学）；Motamed Cancer Institute（莫塔梅德癌症研究所）；University of Iowa Carver College of Medicine（爱荷华大学卡弗医学院）；Iran University of Medical Sciences（伊朗医科大学）；Université Laval（拉瓦尔大学）
Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院放射科）；Department of Pathology, Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属邵氏医院病理科）；Department of Pathology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属武汉中心医院病理科）；Department of Pathology, The First Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第一医院病理科）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学化学与生物工程系）；Division of Life Science, The Hong Kong University of Science and Technology（香港科学与技术大学生命科学系）
Johns Hopkins University（约翰斯·霍普金斯大学）；PAII Inc.（PAII公司）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）；Ping An Technology（平安科技）；Shanghai Jiao Tong University（上海交通大学）
The boundary of tumors (hepatocellular carcinoma, or HCC) contains rich semantics: capsular invasion, visibility, smoothness, folding and protuberance, etc. Capsular invasion on tumor boundary has proven to be clinically correlated with the prognostic indicator, microvascular invasion (MVI). Investigating tumor boundary semantics has tremendous clinical values. In this paper, we propose the first and novel computational framework that disentangles the task into two components: spatial vertex localization and sequential semantic classification. (1) A HCC tumor segmentor is built for tumor mask …
Technische Universit\"at M\"unchen, Image-Based Biomedical Modeling Group, Arccisstrasse 21, 80333 Munich Technische Universit\"at M\"unchen, Institute for diagnostic；interventional Radiology, Ismaninger Str. 22, 81675 Munich + Ludwig-Maximilians-Universit\"at, German Center for Vertigo
Automatic non-invasive assessment of hepatocellular carcinoma (HCC) malignancy has the potential to substantially enhance tumor treatment strategies for HCC patients. In this work we present a novel framework to automatically characterize the malignancy of HCC lesions from DWI images. We predict HCC malignancy in two steps: As a first step we automatically segment HCC tumor lesions using cascaded fully convolutional neural networks (CFCN). A 3D neural network (SurvivalNet) then predicts the HCC lesions' malignancy from the HCC tumor segmentation. We formulate this task as a classification prob…
