Although active learning (AL) in segmentation tasks enables experts to annotate selected regions of interest (ROIs) instead of entire images, it remains highly challenging, labor-intensive, and cognitively demanding due to the blurry and ambiguous boundaries commonly observed in medical images. Also, in conventional AL, annotation effort is a function of the ROI- larger regions make the task cognitively easier but incur higher annotation costs, whereas smaller regions demand finer precision and more attention from the expert. In this context, language guidance provides an effective alternative, requiring minimal expert effort while bypassing the cognitively demanding task of precise boundary delineation in segmentation. Towards this goal, we introduce LINGUAL: a framework that receives natural language instructions from an expert, translates them into executable programs through in-context learning, and automatically performs the corresponding sequence of sub-tasks without any human intervention. We demonstrate the effectiveness of LINGUAL in active domain adaptation (ADA) achieving comparable or superior performance to AL baselines while reducing estimated annotation time by approximately 80%.
Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Zhongguancun Academy & Zhongguancun Institute of Artificial Intelligence, Beijing, China（北京中关村学院及中关村人工智能研究院）；Department of Molecular Imaging, Royal Prince Alfred Hospital（皇家珀斯阿尔弗雷德医院分子成像部）
CDPDNet: Integrating Text Guidance with Hybrid Vision Encoders for Medical Image Segmentation
J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida（J. Crayton Pruitt家庭生物医学工程系，佛罗里达大学）；Department of Medicine, University of Florida（医学系，佛罗里达大学）
Frequency-domain Multi-modal Fusion for Language-guided Medical Image Segmentation
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）
Peking University（北京大学）；Deepwise AI Lab（深睿医疗人工智能实验室）；Microsoft Research Asia（微软亚洲研究院）；The University of Hong Kong（香港大学）
As an important upstream task for many medical applications, supervised landmark localization still requires non-negligible annotation costs to achieve desirable performance. Besides, due to cumbersome collection procedures, the limited size of medical landmark datasets impacts the effectiveness of large-scale self-supervised pre-training methods. To address these challenges, we propose a two-stage framework for one-shot medical landmark localization, which first infers landmarks by unsupervised registration from the labeled exemplar to unlabeled targets, and then utilizes these noisy pseudo l…
Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation
Air Force Engineering University（空军工程大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Sydney（悉尼大学）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias
The Ohio State University（俄亥俄州立大学）；Imperial College London（伦敦帝国学院）；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Peking University（北京大学）
Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation
Nanjing University（南京大学）；Southeast University（东南大学）；Shandong Women’s University（山东女子学院）；State Key Laboratory for Novel Software Technology（现代软件技术国家重点实验室）；National Institute of Healthcare Data Science（国家医疗数据科学研究院）
The Chinese University of Hong Kong（香港中文大学）；University of Cambridge（剑桥大学）；Inception Institute of Artificial Intelligence（启迪人工智能研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Southern University of Science and Technology（南方科技大学）；Chinese Academy of Sciences（中国科学院）
Automatic breast lesion segmentation in ultrasound helps to diagnose breast cancer, which is one of the dreadful diseases that affect women globally. Segmenting breast regions accurately from ultrasound image is a challenging task due to the inherent speckle artifacts, blurry breast lesion boundaries, and inhomogeneous intensity distributions inside the breast lesion regions. Recently, convolutional neural networks (CNNs) have demonstrated remarkable results in medical image segmentation tasks. However, the convolutional operations in a CNN often focus on local regions, which suffer from limit…
Tianjin University（天津大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Segmentation is a fundamental task in medical image analysis. However, most existing methods focus on primary region extraction and ignore edge information, which is useful for obtaining accurate segmentation. In this paper, we propose a generic medical segmentation method, called Edge-aTtention guidance Network (ET-Net), which embeds edge-attention representations to guide the segmentation network. Specifically, an edge guidance module is utilized to learn the edge-attention representations in the early encoding layers, which are then transferred to the multi-scale decoding layers, fused usin…
Nankai University（南开大学）；Civil Aviation General Hospital（民航总医院）；Chinese PLA General Hospital（中国人民解放军总医院）
Breast lesions segmentation is an important step of computer-aided diagnosis system, and it has attracted much attention. However, accurate segmentation of malignant breast lesions is a challenging task due to the effects of heterogeneous structure and similar intensity distributions. In this paper, a novel bidirectional aware guidance network (BAGNet) is proposed to segment the malignant lesion from breast ultrasound images. Specifically, the bidirectional aware guidance network is used to capture the context between global (low-level) and local (high-level) features from the input coarse sal…
M$^3$-Med: A Benchmark for Multi-lingual, Multi-modal, and Multi-hop Reasoning in Medical Instructional Video Understanding
Beijing Institute of Technology（北京理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Chinese Academy of Sciences（中国科学院）；Air Force Hospital of Southern Theater Command of PLA（中国人民解放军南部战区空军医院）；Shenzhen Traditional Chinese Medicine Hospital（深圳中医药医院）
General Vision Encoder Features as Guidance in Medical Image Registration
Klinikum Rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院）；School of Computation, Information & Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑中心生物医学影像机器学习研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Cancer Research UK Cambridge Institute, University of Cambridge（剑桥大学英国癌症研究剑桥研究所）；Department of Oncology, University of Cambridge（剑桥大学肿瘤学系）；German Cancer Consortium (DKTK), Partner Site Munich（德国癌症联盟慕尼黑合作站点）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）
Medical imaging is a cornerstone of therapy and diagnosis in modern medicine. However, the choice of imaging modality for a particular theranostic task typically involves trade-offs between the feasibility of using a particular modality (e.g., short wait times, low cost, fast acquisition, reduced radiation/invasiveness) and the expected performance on a clinical task (e.g., diagnostic accuracy, efficacy of treatment planning and guidance). In this work, we aim to apply the knowledge learned from the less feasible but better-performing (superior) modality to guide the utilization of the more-fe…
DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classification
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；University of Cambridge（剑桥大学）
MedDiff-FT: Data-Efficient Diffusion Model Fine-tuning with Structural Guidance for Controllable Medical Image Synthesis
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Peking University Shenzhen Graduate School South China University of Technology, China
On-the-Fly Guidance Training for Medical Image Registration
University of Leeds（利兹大学）；University of California, Irvine（加州大学欧文分校）；Tongji University（同济大学）；Huazhong University of Science and Technology（华中科技大学）；Fudan Univeristy（复旦大学）；University of California, San Diego（加州大学圣迭戈分校）
LLM-Bootstrapped Targeted Finding Guidance for Factual MLLM-based Medical Report Generation
Zhejiang University（浙江大学）；The Second Affiliated Hospital Zhejiang University School of Medicine（浙江大学医学院附属第二医院）；Hangzhou Pu Jian Medical Technology Co., Ltd.（杭州普健医疗科技有限公司）
Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models
The Key Laboratory of Smart Human-Computer Interaction and Wearable Technology of Shaanxi Province, Xidian University, China（陕西省智能人机交互与可穿戴技术重点实验室，西安电子科技大学）；Warren Alpert Medical School, Brown University, USA（布朗大学沃伦·阿尔珀特医学院）；National University of Singapore, Singapore（新加坡国立大学）
FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video Generation
School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China（生物科学与医学工程学院、软件开发环境国家重点实验室、教育部生物力学与机械生物学重点实验室、北京生物医学工程先进创新中心、北京航空航天大学）；Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China（生物医学工程系、清华大学）；Department of Gynecology Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China（妇科肿瘤科、国家癌症中心/国家癌症临床研究中心/癌症医院、中国医学科学院和北京协和医学院）；ByteDance Inc., Beijing 100098, China（字节跳动公司）
Ultrasound Report Generation with Cross-Modality Feature Alignment via Unsupervised Guidance
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Shenzhen Institute of Advanced Technology, the Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序与增强现实教席）；The Third Medical Centre of Chinese PLA General Hospital（中国人民解放军总医院第三医学中心）；The Second School of Clinical Medicine, Southern Medical University（南方医科大学第二临床医学院）
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Delft University of Technology（代尔夫特理工大学）；Leiden University Medical Center（莱顿大学医学中心）
Purpose: Deformable Image Registration (DIR) can benefit from additional guidance using corresponding landmarks in the images. However, the benefits thereof are largely understudied, especially due to the lack of automatic landmark detection methods for three-dimensional (3D) medical images. Approach: We present a Deep Convolutional Neural Network (DCNN), called DCNN-Match, that learns to predict landmark correspondences in 3D images in a self-supervised manner. We trained DCNN-Match on pairs of Computed Tomography (CT) scans containing simulated deformations. We explored five variants of DCNN…
VinAI Research（VinAI研究院）；FPT University（FPT大学）；University of Arkansas（阿肯色大学）
Image synthesis from corrupted contrasts increases the diversity of diagnostic information available for many neurological diseases. Recently the image-to-image translation has experienced significant levels of interest within medical research, beginning with the successful use of the Generative Adversarial Network (GAN) to the introduction of cyclic constraint extended to multiple domains. However, in current approaches, there is no guarantee that the mapping between the two image domains would be unique or one-to-one. In this paper, we introduce a novel approach to unpaired image-to-image tr…
Guiding the Guidance: A Comparative Analysis of User Guidance Signals for Interactive Segmentation of Volumetric Images
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；HIDSS4Health - Helmholtz Information and Data Science School for Health（HIDSS4Health——亥姆霍兹健康信息与数据科学学院）；Institute for AI in Medicine, University Hospital Essen（埃森大学医院医学人工智能研究所）；Cancer Research Center Cologne Essen (CCCE), University Medicine Essen（埃森大学医学院科隆埃森癌症研究中心（CCCE））
University of Wollongong（伍伦贡大学）；Shenzhen International Graduate School of Tsinghua University（清华大学深圳国际研究生院）；University of Sydney（悉尼大学）
Automated radiographic report generation is a challenging cross-domain task that aims to automatically generate accurate and semantic-coherence reports to describe medical images. Despite the recent progress in this field, there are still many challenges at least in the following aspects. First, radiographic images are very similar to each other, and thus it is difficult to capture the fine-grained visual differences using CNN as the visual feature extractor like many existing methods. Further, semantic information has been widely applied to boost the performance of generation tasks (e.g. imag…
HQG-Net: Unpaired Medical Image Enhancement with High-Quality Guidance
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；NEC Laboratories America Inc.（NEC美国实验室）；Norwegian University of Science and Technology（挪威科技大学）；ETH Zürich（苏黎世联邦理工学院）；Peng Cheng Laboratory（鹏城实验室）
University of Copenhagen（哥本哈根大学）；McGill Centre for Integrative Neuroscience（麦吉尔整合神经科学中心）；Erasmus Medical Center（伊拉斯谟医学中心）；Technische Universität Ilmenau（伊尔默瑙工业大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Delft University of Technology（代尔夫特理工大学）；University of Nottingham（诺丁汉大学）；University of Oxford（牛津大学）；Zooniverse, University of Oxford（牛津大学Zooniverse）；Eindhoven University of Technology（埃因霍温理工大学）
Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-s…
Deep learning-based image processing is capable of creating highly appealing results. However, it is still widely considered as a "blackbox" transformation. In medical imaging, this lack of comprehensibility of the results is a sensitive issue. The integration of known operators into the deep learning environment has proven to be advantageous for the comprehensibility and reliability of the computations. Consequently, we propose the use of the locally linear guided filter in combination with a learned guidance map for general purpose medical image processing. The output images are only process…
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；College of AI, Tsinghua University（清华大学人工智能学院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）
Point of care ultrasound (POCUS) consists in the use of ultrasound imaging in critical or emergency situations to support clinical decisions by healthcare professionals and first responders. In this setting it is essential to be able to provide means to obtain diagnostic data to potentially inexperienced users who did not receive an extensive medical training. Interpretation and acquisition of ultrasound images is not trivial. First, the user needs to find a suitable sound window which can be used to get a clear image, and then he needs to correctly interpret it to perform a diagnosis. Althoug…
