Large Language Models (LLMs), known for their versatility in textual data, are increasingly being explored for their potential to enhance medical image segmentation, a crucial task for accurate diagnostic imaging. This study explores enhancing Vision Transformers (ViTs) for medical image segmentation by integrating pre-trained LLM transformer blocks. Our approach, which incorporates a frozen LLM transformer block into the encoder of a ViT-based model, leads to substantial improvements in segmentation performance across various medical imaging modalities. We propose a Hybrid Attention Mechanism that combines global and local feature learning with a Multi-Scale Fusion Block for aggregating features across different scales. The enhanced model shows significant performance gains, including an average Dice score increase from 0.74 to 0.79 and improvements in accuracy, precision, and the Jaccard Index. These results demonstrate the effectiveness of LLM-based transformers in refining medical image segmentation, highlighting their potential to significantly boost model accuracy and robustness. The source code and our implementation are available at: https://github.com/AS-Lab/Marthi-et-al-2025-MedVisionLlama-Pre-Trained-LLM-Layers-to-Enhance-Medical-Image-Segmentation
Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Models
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Shenzhen University of Advanced Technology(深圳先进技术大学);Shenzhen Polytechnic University(深圳职业技术大学);University of Macau(澳门大学);Southern University of Science and Technology(南方科技大学)
Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation
Northwestern Polytechnical University(西北工业大学);Ningbo Institute of Northwestern Polytechnical University(西北工业大学宁波研究院);Research and Development Institute of Northwestern Polytechnical University in Shenzhen(西北工业大学深圳研究院)
Leveraging Task-Specific Knowledge from LLM for Semi-Supervised 3D Medical Image Segmentation
Indian Institute of Technology Roorkee(鲁尔基印度理工学院);Vellore Institute of Technology(韦洛尔理工大学);Alipurduar Government Engineering and Management College(阿里布尔杜阿尔政府工程与管理学院);Indian Institute of Technology Mandi(曼迪印度理工学院)
本文提出 LLM-SegNet,通过 LLM 将任务特定知识注入协同训练框架,并设计统一分割损失,提升半监督 3D 医学图像分割性能。
Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data
Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital(高级神经影像支持中心(SCAN)、伯尔尼大学诊断与介入神经放射学研究所、伯尔尼大学、因塞尔斯皮塔尔医院、伯尔尼大学医院);NVIDIA(英伟达)
Leveraging Labelled Data Knowledge: A Cooperative Rectification Learning Network for Semi-supervised 3D Medical Image Segmentation
School of Mechanical Engineering and Automation, Northeastern University(东北大学机械工程与自动化学院);Australian Institute for Machine Learning, University of Adelaide(阿德莱德大学澳大利亚机器学习研究所);Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey(萨里大学视觉、语音与信号处理中心)
Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study
University of Colorado Anschutz Medical Campus(科罗拉多大学安舒茨医学园区);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of Aberdeen(阿伯丁大学);Boston Children's Hospital, Harvard Medical School(哈佛医学院波士顿儿童医院);Loyola University Chicago(芝加哥洛约拉大学)
Semi-supervised Medical Image Segmentation Method Based on Cross-pseudo Labeling Leveraging Strong and Weak Data Augmentation Strategies
Hangzhou Dianzi University(杭州电子科技大学);Zhejiang University of Finance and Economics(浙江财经大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Shenzhen Children's Hospital(深圳市儿童医院);Shenzhen Research Institute of Big Data(深圳市大数据研究院)
Leveraging Computerized Adaptive Testing for Cost-effective Evaluation of Large Language Models in Medical Benchmarking
Institute of Medical Education, Health Science Center, Peking University(北京大学医学教育研究院、健康科学中心);School of Public Health, Peking University(北京大学公共卫生学院);Peking University Health Science Center-Chaoxing Joint Laboratory for Digital and Smart Medical(北京大学健康科学中心-超星数字与智能医疗联合实验室);Graduate School of Education, Peking University(北京大学教育学院)
Leveraging Large Language Models to Extract and Translate Medical Information in Doctors' Notes for Health Records and Diagnostic Billing Codes
Department of Computer Science & Information Technologies, Frostburg State University(弗罗斯堡州立大学计算机科学与信息科技系);Department of Emergency Medicine, FAU Charles E Schmidt College of Medicine(FAU查尔斯·E·施密特医学院急诊医学系)
Semi-Supervised Multi-Modal Medical Image Segmentation for Complex Situations
School of Physics, Peking University, Beijing, China(北京大学物理系);School of Computer Science, Peking University, Beijing, China(北京大学计算机系);Department of Radiotherapy, Peking University Cancer Hospital, Beijing, China(北京大学肿瘤医院放疗科)
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation
Pontificia Universidad Católica de Chile(智利天主教大学);University of Pennsylvania(宾夕法尼亚大学);KU Leuven(鲁汶大学);Millennium Institute for Intelligent Healthcare Engineering (iHEALTH)(千禧智能医疗工程研究院(iHEALTH));National Center for Artificial Intelligence (CENIA)(国家人工智能中心(CENIA))
Leveraging Large Language Model as Simulated Patients for Clinical Education
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Computer Science, Wuhan University(武汉大学计算机学院);CureFun Co.(CureFun公司)
A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing
Department of Computing;Games, Teesside University, Middlesbrough, United Kingdom Centre for Digital Innovation, Teesside University, Middlesbrough, United Kingdom Faculty of Business \& Technology, University of Sunderland, Sunderland, United Kingdom
Dual-Task Mutual Learning for Semi-Supervised Medical Image Segmentation
Beihang University(北京航空航天大学);Hefei Innovation Research Institute, Beihang University(北京航空航天大学合肥创新研究院);Beijing Advanced Innovation Centre for Biomedical Engineering(北京生物医学工程高精尖创新中心);Beijing Advanced Innovation Centre for Big Data-Based Precision Medicine(北京大数据精准医疗高精尖创新中心)
Towards General Purpose Medical AI: Continual Learning Medical Foundation Model
Beijing University of Posts and Telecommunications(北京邮电大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);West China Hospital, Sichuan University(四川大学华西医院);Shanghai Jiao Tong University(上海交通大学);Sichuan University Pittsburgh Institute(四川大学匹兹堡学院)
A Retrieval-Based Approach to Medical Procedure Matching in Romanian
National University of Science and Technology POLITEHNICA Bucharest(波兰技术大学科学与技术国家大学);Dalle Molle Institute for Artificial Intelligence Research (IDSIA)(达勒莫利人工智能研究所)
Leveraging weak complementary labels to improve semantic segmentation of hepatocellular carcinoma and cholangiocarcinoma in H&E-stained slides
Technische Universität Berlin(柏林工业大学);Aignostics GmbH(Aignostics 公司);Charité Universitätsmedizin Berlin(柏林夏里特医学院);Berlin Institute of Health at Charité – Universitätsmedizin Berlin(柏林健康研究所(夏里特医学院));University Hospital Düsseldorf(杜塞尔多夫大学医院);Ludwig-Maximilians-Universität München(慕尼黑大学);German Cancer Research Center(德国癌症研究中心)
Shanghai East Hospital(上海东方医院);Tongji University School of Medicine(同济大学医学院);Institute of Ethnology and Anthropology(民族学与人类学研究所);Chinese Academy of Social Sciences(中国社会科学院)
WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction
University of Toronto(多伦多大学);Vector Institute(矢量研究所);McGill University(麦吉尔大学);Peter Munk Cardiac Centre, University Health Network(大学健康网络彼得·芒克心脏中心);AI Hub, University Health Network(大学健康网络人工智能中心)
Pairwise Similarity Regularization for Semi-supervised Graph Medical Image Segmentation
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机学院);Academy of Military Sciences(军事科学院);Peking University(北京大学);Hebei University of Technology(河北工业大学)