罕见事件由于其发生频率低,缺乏充足的数据,因此深度学习技术无法有效估计此类数据的分布。Open-vocabulary模型代表了一种创新的图像分类方法。与传统模型不同,这些模型在推理过程中将图像分类为由自然语言提示指定的任意类别集合。这些提示通常包含手动制作的模板(例如,'a photo of a {}'),并填入每个类别的名称。本文介绍了一种简单而有效的方法,用于生成包含判别性特征的高准确度且具有上下文描述性的提示。罕见事件检测,尤其是在医学领域,由于类间差异低和类内差异大而更具挑战性。为了解决这些问题,我们提出了一种新颖的方法,利用关于罕见事件的领域特定专家知识来生成定制化且与上下文相关的提示,这些提示随后被大型语言模型用于图像分类。我们的零样本、隐私保护方法无需额外训练即可增强罕见事件分类,优于现有最先进的技术。
英文摘要:
Rare events, due to their infrequent occurrences, do not have much data, and hence deep learning techniques fail in estimating the distribution for such data. Open-vocabulary models represent an innovative approach to image classification. Unlike traditional models, these models classify images into any set of categories specified with natural language prompts during inference. These prompts usually comprise manually crafted templates (e.g., 'a photo of a {}') that are filled in with the names of each category. This paper introduces a simple yet effective method for generating highly accurate and contextually descriptive prompts containing discriminative characteristics. Rare event detection, especially in medicine, is more challenging due to low inter-class and high intra-class variability. To address these, we propose a novel approach that uses domain-specific expert knowledge on rare events to generate customized and contextually relevant prompts, which are then used by large language models for image classification. Our zero-shot, privacy-preserving method enhances rare event classification without additional training, outperforming state-of-the-art techniques.
Topicwise Separable Sentence Retrieval for Medical Report Generation
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所)
Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition
Hong Kong University of Science and Technology(香港科技大学);Sun Yat-sen University(中山大学);Southern University of Science and Technology(南方科技大学);Peng Cheng Laboratory(鹏城实验室)
Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis
School of Information and Communication Engineering, Hainan University(海南大学信息与通信工程学院);College of Mathematical Sciences, BoHai University(渤海大学数学科学学院);Australian AI Institute, University of Technology Sydney(悉尼科技大学澳大利亚人工智能研究所);School of Electronic Science and Technology, Hainan University(海南大学电子科学与技术学院)
Unsupervised Task Design to Meta-Train Medical Image Classifiers
Australian Institute for Machine Learning, The University of Adelaide(阿德莱德大学澳大利亚机器学习研究所);Institute for Systems and Robotics, Instituto Superior Tecnico(高等技术学院系统与机器人研究所)
Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification
Southern University of Science and Technology(南方科技大学);The University of Hong Kong(香港大学);University of British Columbia(不列颠哥伦比亚大学);Jiaxing Research Institute, Southern University of Science and Technology(南方科技大学嘉兴研究院)
MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation
German Cancer Research Center (DKFZ)(德国癌症研究中心);Heidelberg University(海德堡大学);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心);University Medical Center Heidelberg(海德堡大学医院);Helmholtz Imaging(亥姆霍兹成像机构)
Investigation of Customized Medical Decision Algorithms Utilizing Graph Neural Networks
Stevens Institute of Technology(史蒂文斯理工学院);Northeastern University(东北大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校);Brandeis University(布兰迪斯大学);Carnegie Mellon University(卡内基梅隆大学)
Zero-Training Task-Specific Model Synthesis for Few-Shot Medical Image Classification
AI Innovation Department, Beijing 1st BioTech Group Co., Ltd.(人工智能创新部门,北京第一生物科技集团有限公司);Diplomatic Negotiation Simulation and Data Laboratory, China Foreign Affairs University(外交谈判模拟与数据实验室,中国外交学院)
Flow Matching with Optimized Subclass Priors for Medical Image Augmentation
Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91052 Erlangen, Germany(生物医学工程人工智能系,弗里德里希-亚历山大-厄林根-纽伦堡大学);Department of Computing, Imperial College London, London SW7 2AZ, UK(计算系,帝国理工学院伦敦分校)
Radiuma: A Unified Zero-Code Executable Graphical Workflow Generator for Reproducible and Shareable Medical Image Analysis and Machine Learning
Department of Basic and Translational Research, BC Cancer Research Institute(基础与转化研究部,BC癌症研究中心);Department of Radiology, University of British Columbia(放射科,不列颠哥伦比亚大学);Technological Virtual Collaboration (TECVICO Corp.)(技术虚拟协作(TECVICO公司));Department of Computer Science, University of British Columbia(计算机科学系,不列颠哥伦比亚大学);Department of Cardiology, Inselspital, Bern University Hospital, University of Bern(心脏病学部,Inselspital,伯恩大学医院,伯恩大学);Department of Digital Medicine, University of Bern(数字医学系,伯恩大学);Departments of Physics & Biomedical Engineering, University of British Columbia(物理学与生物医学工程系,不列颠哥伦比亚大学)
Double Banking on Knowledge: Customized Modulation and Prototypes for Multi-Modality Semi-supervised Medical Image Segmentation
College of Computer Science, Sichuan University(四川大学计算机学院);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);College of Electrical Engineering, Sichuan University(四川大学电气工程学院);School of Cyber Science and Engineering, Sichuan University(四川大学网络空间安全学院)
Mask-guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma
Universidad Industrial de Santander(桑坦德工业大学);University of Toulouse(图卢兹大学);Université Paul Sabatier Toulouse 3(图卢兹第三大学(保罗·萨巴蒂埃大学));Bicêtre University Hospital(比塞特尔大学医院);Assistance publique des hôpitaux de Paris(巴黎公共援助医院);Université Claude Bernard Lyon 1(里昂第一大学(克洛德·贝尔纳里昂第一大学))
How precise are performance estimates for typical medical image segmentation tasks?
Sorbonne Université(索邦大学);Institut du Cerveau - Paris Brain Institute - ICM(巴黎大脑研究所(ICM));CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);Inserm(法国国家健康与医学研究院);AP-HP(巴黎公共援助医院集团);Hôpital de la Pitié Salpêtrière(皮蒂-萨尔佩特里埃医院)
Exploring Generalizable Distillation for Efficient Medical Image Segmentation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing University of Posts and Telecommunications(北京邮电大学);Shandong University of Science and Technology(山东科技大学)
RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation
Pioneer Centre for Artificial Intelligence, Department of Computer Science, University of Copenhagen, Copenhagen, Denmark(人工智能先锋中心、计算机科学系、哥本哈根大学、哥本哈根、丹麦)
FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition
The Hong Kong University of Science and Technology(香港科技大学);University of Girona(赫罗纳大学);HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute(港科大深港协同创新研究院)
6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems(德国癌症研究中心(DKFZ)海德堡,智能医学系统部门);Medical Faculty, Heidelberg University(海德堡大学医学院);Faculty of Mathematics and Computer Science, Heidelberg University(海德堡大学数学与计算机科学学院);HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg(HIDSS4Health - 哈勃-马克斯信息与数据科学健康学院,卡尔斯鲁厄/海德堡);Helmholtz Imaging, German Cancer Research Center (DKFZ)(哈勃-马克斯成像,德国癌症研究中心(DKFZ));Engineering Faculty, Heidelberg University(海德堡大学工程学院);School of Computation, Information and Technology, TUM(技术大学(TUM)计算、信息与技术学院);Weldon School of Biomedical Engineering, Purdue University(普渡大学韦尔登生物医学工程学院);Department of Visceral, Thoracic and Vascular Surgery, University Hospital and Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology(visceral、胸腔和血管外科部门,技术大学(TUD)德累斯顿大学医院和医学院);National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Hospital Heidelberg(肿瘤疾病国家中心(NCT),海德堡NCT,DKFZ与海德堡大学医院之间的合作);Heidelberg University Hospital, Surgical Clinic, Surgical AI Research Group(海德堡大学医院,外科诊所,外科人工智能研究组);Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE(Mohamed Bin Zayed人工智能大学(MBZUAI),阿布扎赫,阿拉伯联合酋长国)