Fine-grained Prompt Tuning: A Parameter and Memory Efficient Transfer Learning Method for High-resolution Medical Image Classification
School of Biomedical Engineering, The University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电机电子工程学系）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）；Department of Electronic and Electrical Engineering, Southern University of Science and Technology（南方科技大学电子与电气工程系）
Yijin Huang, Pujin Cheng, Roger Tam, Xiaoying Tang
Parameter-efficient transfer learning (PETL) is proposed as a cost-effective way to transfer pre-trained models to downstream tasks, avoiding the high cost of updating entire large-scale pre-trained models (LPMs). In this work, we present Fine-grained Prompt Tuning (FPT), a novel PETL method for medical image classification. FPT significantly reduces memory consumption compared to other PETL methods, especially in high-resolution input contexts. To achieve this, we first freeze the weights of the LPM and construct a learnable lightweight side network. The frozen LPM takes high-resolution images as input to extract fine-grained features, while the side network is fed low-resolution images to reduce memory usage. To allow the side network to access pre-trained knowledge, we introduce fine-grained prompts that summarize information from the LPM through a fusion module. Important tokens selection and preloading techniques are employed to further reduce training cost and memory requirements. We evaluate FPT on four medical datasets with varying sizes, modalities, and complexities. Experimental results demonstrate that FPT achieves comparable performance to fine-tuning the entire LPM while using only 1.8% of the learnable parameters and 13% of the memory costs of an encoder ViT-B model with a 512 x 512 input resolution.
EviPrompt: A Training-Free Evidential Prompt Generation Method for Segment Anything Model in Medical Images
Beijing University of Posts and Telecommunications（北京邮电大学）；National Institute of Health Data Science, Peking University（北京大学健康数据科学研究所）；The Chinese University of Hong Kong（香港中文大学）
DiffuSAM: Diffusion-Based Prompt-Free SAM2 for Few-Shot and Source-Free Medical Image Segmentation
School of Electrical Engineering, Tel Aviv University, Israel（特拉维夫大学电气工程学院）；School of Biomedical Engineering, Tel Aviv University, Israel（特拉维夫大学生物医学工程学院）
Medical Image Synthesis via Fine-Grained Image-Text Alignment and Anatomy-Pathology Prompting
City University of Hong Kong（香港城市大学）；The Chinese University of Hong Kong（香港中文大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Lehigh University（里海大学）
针对医学图像数据稀缺及现有方法难以准确呈现解剖与病理细节的问题，提出结合解剖-病理提示和视觉码本细粒度图文对齐的合成模型，并在胸部 X 射线数据上验证其语义准确性。
Sam-Guided Enhanced Fine-Grained Encoding with Mixed Semantic Learning for Medical Image Captioning
Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院，浙江大学）；Sir Run Run Shaw Hospital, Zhejiang University School of Medicine（浙江大学医学院附属邵逸夫医院）
Prompt Triage: Structured Optimization Enhances Vision-Language Model Performance on Medical Imaging Benchmarks
Stanford University, Department of Computer Science, Stanford, CA, USA（斯坦福大学计算机科学系）；Stanford University, Department of Biomedical Data Science, Stanford, CA, USA（斯坦福大学生物医学数据科学系）；Stanford University, Stanford Institute for Human-Centered Artificial Intelligence, Stanford, CA, USA（斯坦福大学人类中心人工智能研究所）；Stanford University, School of Medicine, Department of Dermatology, Stanford, CA, USA（斯坦福大学医学院皮肤科系）；Stanford University, Department of Radiology, Stanford, CA, USA（斯坦福大学放射科）
MedRCube: A Multidimensional Framework for Fine-Grained and In-Depth Evaluation of MLLMs in Medical Imaging
School of Data Science, Fudan University（复旦大学数据科学学院）；Shanghai Innovation Institute（上海创新研究院）；School of Integrated Circuits, Zhejiang University（浙江大学集成电路学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）
PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification
Dalian University of Technology（大连理工大学）；School of Computer Science and Technology（计算机科学与技术学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Dalian Minzu University（大连民族大学）；School of Information and Communication Engineering（信息与通信工程学院）
Prompt-driven Latent Domain Generalization for Medical Image Classification
Monash University（莫纳什大学）；Airdoc-Monash Research, Monash University（Airdoc-莫纳什研究，莫纳什大学）；Inception Institute of AI（Inception人工智能研究院）；University of Queensland Diamantina Institute, Dermatology Research Centre, The University of Queensland（昆士兰大学Diamantina研究所，皮肤病学研究中心，昆士兰大学）；Victorian Melanoma Service, Alfred Health（维多利亚黑色素瘤服务中心，阿尔弗雷德健康中心）
MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training
The Hong Kong University of Science and Technology（香港科技大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen People’s Hospital（深圳市人民医院）；The University of Hong Kong（香港大学）；Chinese PLA General Hospital（中国人民解放军总医院）
EAFP-Med: An Efficient Adaptive Feature Processing Module Based on Prompts for Medical Image Detection
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机学院）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；Xi'an Jiaotong-Liverpool University（西交利物浦大学）；University of Leicester（莱斯特大学）；Southeast University（东南大学）；University of Liverpool（利物浦大学）
Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering
Graduate School of Information Science and Technology, Hokkaido University（北海道大学信息科学与技术研究院）；Faculty of Information Science and Technology, Hokkaido University（北海道大学信息科学与技术学院）
TV-SAM: Increasing Zero-Shot Segmentation Performance on Multimodal Medical Images Using GPT-4 Generated Descriptive Prompts Without Human Annotation
Sichuan University（四川大学）；Case Western Reserve University（凯斯西储大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Urgench State University（乌尔根奇国立大学）；Gansu Agricultural University（甘肃农业大学）
Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation
City University of Hong Kong（香港城市大学）；Shenzhen University（深圳大学）；University of Rochester（罗切斯特大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Chinese University of Hong Kong（香港中文大学）
Prompt Mechanisms in Medical Imaging: A Comprehensive Survey
Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院，澳门理工学院）；Netherlands Cancer Institute, Department of Radiology（荷兰癌症研究所，放射科）；Radboud University Medical Centre, Department of Radiology and Nuclear Medicine（拉德堡德大学医学中心，放射科和核医学科）；College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）；Medical Department of Breast Cancer, Hunan Cancer Hospital（乳腺癌医学部，湖南癌症医院）；the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University（湘雅医学院附属肿瘤医院，中南大学）；Faculty of Biomedical Engineering, Eindhoven University of Technology（生物医学工程学院，埃因霍温理工大学）；Laboratory of Advanced Theranostic Materials and Technology, University of Chinese Academy of Sciences（先进诊疗材料与技术实验室，中国科学院大学）
CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentation
Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Zhongguancun Academy（中关村学院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）；School of Computer Science, the University of Sydney（悉尼大学计算机学院）；China United Network Communications Group Co., Ltd.（中国联合网络通信集团有限公司）
Hierarchical Self-Prompting SAM: A Prompt-Free Medical Image Segmentation Framework
State Key Laboratory of Multimodal Artificial Intelligence System, Institute of Automation, Chinese Academy of Science, China（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院，中国）；School of Artificial Intelligence, University of Chinese Academy of Science, China（人工智能学院，中国科学院大学，中国）；Department of Radiology, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing（放射科，北医三院，北京医科大学，中国医学科学院，北京）
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）
From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation
Technical University of Munich（慕尼黑技术大学）；Korea University（韩国大学）；Kyung Hee University（庆熙大学）；Seoul National University of Science and Technology（首尔科学技术大学）
Prompt-based Personalized Federated Learning for Medical Visual Question Answering
Graduate School of Information Science and Technology, Hokkaido University（北海道大学信息科学与技术研究生院）；Faculty of Information Science and Technology, Hokkaido University（北海道大学信息科学与技术学院）
