Executing machine learning (ML) pipelines in real-time on radiology images is hard due to the limited computing resources in clinical environments and the lack of efficient data transfer capabilities to run them on research clusters. We propose Niffler, an integrated framework that enables the execution of ML pipelines at research clusters by efficiently querying and retrieving radiology images from the Picture Archiving and Communication Systems (PACS) of the hospitals. Niffler uses the Digital Imaging and Communications in Medicine (DICOM) protocol to fetch and store imaging data and provides metadata extraction capabilities and Application programming interfaces (APIs) to apply filters on the images. Niffler further enables the sharing of the outcomes from the ML pipelines in a de-identified manner. Niffler has been running stable for more than 19 months and has supported several research projects at the department. In this paper, we present its architecture and three of its use cases: an inferior vena cava (IVC) filter detection from the images in real-time, identification of scanner utilization, and scanner clock calibration. Evaluations on the Niffler prototype highlight its feasibility and efficiency in facilitating the ML pipelines on the images and metadata in real-time and retrospectively.
Martinos Center for Biomedical Imaging（马蒂诺斯生物医学成像中心）；Massachusetts General Hospital（麻省总医院）；MGH & BWH Center for Clinical Data Science（麻省总医院与布莱根妇女医院临床数据科学中心）；Mass General Brigham（麻省医疗布列根）；Isomics, Inc.（Isomics公司）；Center for Integrated Diagnostics（综合诊断中心）；Harvard Medical School（哈佛医学院）；PixelMed Publishing, LLC.（PixelMed出版有限责任公司）；Surgical Planning Laboratory（外科规划实验室）；Brigham and Women’s Hospital（布莱根妇女医院）
Machine learning is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but their lack of interoperability has been a major barrier for clinical integration and evaluation. The DICOM a standard specifies Information Object Definitions and Services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software l…
Sharing standardized image-derived data in computational pathology using DICOM
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；PixelMed Publishing（PixelMed出版公司）；Radical Imaging LLC（Radical成像有限责任公司）；Massachusetts General Hospital（马萨诸塞州总医院）；Stanford University（斯坦福大学）；University Hospital Erlangen（埃尔朗根大学医院）；Stony Brook University（石溪大学）；National Institute of Allergy and Infectious Diseases（美国国家过敏和传染病研究所）；Google Research（谷歌研究院）；Brigham and Women's Hospital（布莱根和妇女医院）
Building RadiologyNET: Unsupervised annotation of a large-scale multimodal medical database
University of Rijeka – Faculty of Engineering（里耶卡大学工程学院）；Center for Artificial Intelligence and Cybersecurity（人工智能与网络安全中心）；Medical University of Graz（格拉茨医科大学）
Institute of Cancer Research（癌症研究所）；Universidad de Valladolid（巴利亚多利德大学）；AGH University of Science and Technology（AGH科技大学）；Cardiff University（卡迪夫大学）；Aberystwyth University（阿伯里斯特威斯大学）
Gliomas are brain tumours with a high mortality rate. There are various grades and sub-types of this tumour, and the treatment procedure varies accordingly. Clinicians and oncologists diagnose and categorise these tumours based on visual inspection of radiology and histology data. However, this process can be time-consuming and subjective. The computer-assisted methods can help clinicians to make better and faster decisions. In this paper, we propose a pipeline for automatic classification of gliomas into three sub-types: oligodendroglioma, astrocytoma, and glioblastoma, using both radiology a…
A Utility-preserving De-identification Pipeline for Cross-hospital Radiology Data Sharing
Shandong University（山东大学）；Australian National University（澳大利亚国立大学）；Hong Kong University of Science and Technology（香港科技大学）；Curtin University（科廷大学）；Edith Cowan University（埃迪斯科文大学）；University of Western Australia（西澳大利亚大学）
We propose a data collecting and annotation pipeline that extracts information from Vietnamese radiology reports to provide accurate labels for chest X-ray (CXR) images. This can benefit Vietnamese radiologists and clinicians by annotating data that closely match their endemic diagnosis categories which may vary from country to country. To assess the efficacy of the proposed labeling technique, we built a CXR dataset containing 9,752 studies and evaluated our pipeline using a subset of this dataset. With an F1-score of at least 0.9923, the evaluation demonstrates that our labeling tool perform…
Faculty of Science, Technology and Communication（科学、技术与传播学院）
Within this thesis we propose a platform for combining Augmented Reality (AR) hardware with machine learning in a user-oriented pipeline, offering to the medical staff an intuitive 3D visualization of volumetric Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) medical image segmentations inside the AR headset, that does not need human intervention for loading, processing and segmentation of medical images. The AR visualization, based on Microsoft HoloLens, employs a modular and thus scalable frontend-backend architecture for real-time visualizations on multiple AR headsets. As Con…
To conduct a radiomics or deep learning research experiment, the radiologists or physicians need to grasp the needed programming skills, which, however, could be frustrating and costly when they have limited coding experience. In this paper, we present DARWIN, a flexible research platform with a graphical user interface for medical imaging research. Our platform is consists of a radiomics module and a deep learning module. The radiomics module can extract more than 1000 dimension features(first-, second-, and higher-order) and provided many draggable supervised and unsupervised machine learnin…
Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting
Computer Aided Medical Procedures, Technische Universität München, Germany（计算机辅助医疗程序，慕尼黑技术大学，德国）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心，德国）
NeuroSymb-MRG: Differentiable Abductive Reasoning with Active Uncertainty Minimization for Radiology Report Generation
University of Macau（澳门大学）；Tsinghua University（清华大学）；Fudan University（复旦大学）；Peking University（北京大学）；The First People’s Hospital of Foshan（佛山市第一人民医院）；Capital Medical University（首都医科大学）；Rutgers Cancer Institute, NJ, USA（罗格斯癌症研究所（美国新泽西州）；纽约大学朗格尼健康中心（美国纽约州））；and NYU Langone Health, NY, USA（Techno国际新城（印度加尔各答））；Techno International New Town, Kolkata, India
AI generated annotations for Breast, Brain, Liver, Lungs and Prostate cancer collections in National Cancer Institute Imaging Data Commons
BAMF Health（BAMF健康公司）；National Institute of Health（美国国立卫生研究院）；Frederick National Laboratory for Cancer Research（弗雷德里克国家癌症研究实验室）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Visual Prompt Engineering for Vision Language Models in Radiology
Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany（德国癌症研究中心医学图像计算部）；Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany（海德堡大学数学与计算机科学学院）；Medical Faculty Heidelberg, University of Heidelberg, Heidelberg, Germany（海德堡大学医学学院）
A Synthetic Data-Driven Radiology Foundation Model for Pan-tumor Clinical Diagnosis
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Lab（上海人工智能实验室）；The First Hospital of China Medical University（中国医科大学附属第一医院）；Harvard Medical School（哈佛医学院）
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；College of AI, Tsinghua University（清华大学人工智能学院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Beijing Electronic Digital & Intelligence（北京电子数字与智能）
University of Southampton（南安普顿大学）；University Hospital Southampton NHS Foundation Trust（南安普顿大学医院NHS基金会信托）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
A desire to achieve large medical imaging datasets keeps increasing as machine learning algorithms, parallel computing, and hardware technology evolve. Accordingly, there is a growing demand in pooling data from multiple clinical and academic institutes to enable large-scale clinical or translational research studies. Magnetic resonance imaging (MRI) is a frequently used, non-invasive imaging modality. However, constructing a big MRI data repository has multiple challenges related to privacy, data size, DICOM format, logistics, and non-standardized images. Not only building the data repository…
University of Rijeka（里耶卡大学）；University of Ljubljana（卢布尔雅那大学）；Jozef Stefan Institute（约泽夫·斯蒂芬研究所）
Creating large datasets of medical radiology images from several sources can be challenging because of the differences in the acquisition and storage standards. One possible way of controlling and/or assessing the image selection process is through medical image clustering. This, however, requires an efficient method for learning latent image representations. In this paper, we tackle the problem of fully-unsupervised clustering of medical images using pixel data only. We test the performance of several contemporary approaches, built on top of a convolutional autoencoder (CAE) - convolutional d…
S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Framework
University of Sydney（悉尼大学）；University of Wollongong（沃林戈大学）；University of Adelaide（阿德莱德大学）；First Clinical Medical College, Guangzhou University of Chinese Medicine（广州中医药大学第一临床学院）
CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback
Stanford University（斯坦福大学）；University of Oxford（牛津大学）；University Hospital Zurich, University of Zurich（苏黎世大学医院、苏黎世大学）；Yongin Severance Hospital, Yonsei University（永临松医院、延世大学）
ReportMedSAM: Guiding Segmentation Through Radiology Reports
School of Engineering, College of Engineering and Physical Sciences, University of Birmingham（工程学院，工程与物理科学学院，伯明翰大学）；Department of Cancer and Genomic Sciences, College of Medicine and Health, University of Birmingham（癌症与基因组科学系，医学与健康学院，伯明翰大学）；Department of Computer Science, Faculty of Science and Engineering, University of Manchester（计算机科学系，科学与工程学院，曼彻斯特大学）；The Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine, and Health, University of Manchester（信息学、成像与数据科学部，生物、医学与健康学院，曼彻斯特大学）
LLaVA-RadZ: Can Multimodal Large Language Models Effectively Tackle Zero-shot Radiology Recognition?
East China Normal University（华东师范大学）；The Chinese University of Hong Kong（香港中文大学）；Northwest A&F University（西北农林科技大学）；Tencent Youtu Lab（腾讯优图实验室）；Xiamen University（厦门大学）
University of Rochester（罗切斯特大学）；Ludwig Maximilian University（路德维希马克西米利安大学）
Chest x-rays are the most common radiology studies for diagnosing lung and heart disease. Hence, a system for automated pre-reporting of pathologic findings on chest x-rays would greatly enhance radiologists' productivity. To this end, we investigate a deep-learning framework with novel training schemes for classification of different thoracic pathology labels from chest x-rays. We use the currently largest publicly available annotated dataset ChestX-ray14 of 112,120 chest radiographs of 30,805 patients. Each image was annotated with either a 'NoFinding' class, or one or more of 14 thoracic pa…
Medical Image De-Identification Benchmark Challenge
Frederick National Laboratory for Cancer Research（弗雷德里克国家癌症研究中心）；National Cancer Institute（国家癌症研究所）；University of Arkansas for Medical Sciences（亚利桑那医学科学大学）；PixelMed Publishing（PixelMed出版公司）；Essential Software Inc.（Essential软件公司）；Amazon Web Services (AWS)（亚马逊网络服务（AWS））；University of California（加州大学）；Cedars-Sinai Medical Center（Cedars-Sinai医疗中心）；Stanford University（斯坦福大学）；Biomedical Engineering & Imaging Institute（生物医学工程与成像研究所）；Icahn School of Medicine at Mount Sinai（辛格纳医学中心学院）；University Medical Center Freiburg（弗赖堡大学医学中心）；German Cancer Research Center（德国癌症研究中心）；Heidelberg University Hospital（海德堡大学医院）；GE Healthcare（通用电气医疗）；ShanghaiTech University（上海科技大学）；Vanderbilt University Medical Center（范德比尔大学医学中心）
Leveraging Clinical Text and Class Conditioning for 3D Prostate MRI Generation
University of Toronto（多伦多大学）；Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital（西奈山医院卢嫩菲尔德-塔南鲍姆研究所）；KITE Research Institute, Toronto Rehabilitation Institute, University Health Network（大学健康网络多伦多康复研究所KITE研究所）；Vector Institute（Vector研究所）；Princess Margaret Hospital（玛格丽特公主医院）；Sinai Health System（西奈健康系统）
In the United States, heart disease is the leading cause of death for both men and women, accounting for 610,000 deaths each year [1]. Physicians use Magnetic Resonance Imaging (MRI) scans to take images of the heart in order to non-invasively estimate its structural and functional parameters for cardiovascular diagnosis and disease management. The end-systolic volume (ESV) and end-diastolic volume (EDV) of the left ventricle (LV), and the ejection fraction (EF) are indicators of heart disease. These measures can be derived from the segmented contours of the LV; thus, consistent and accurate s…
With the mushrooming use of computed tomography (CT) images in clinical decision making, management of CT data becomes increasingly difficult. From the patient identification perspective, using the standard DICOM tag to track patient information is challenged by issues such as misspelling, lost file, site variation, etc. In this paper, we explore the feasibility of leveraging the faces in 3D CT images as biometric features. Specifically, we propose an automatic processing pipeline that first detects facial landmarks in 3D for ROI extraction and then generates aligned 2D depth images, which are…
De-Identification of Medical Imaging Data: A Comprehensive Tool for Ensuring Patient Privacy
Institute for AI in Medicine (IKIM)（人工智能医学研究所（IKIM））；University Hospital Essen（埃森大学医院）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心（CCCE））；University Medicine Essen（埃森大学医学部）；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；Technical University Dortmund（多特蒙德工业大学）
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）；Royal Free London NHS Foundation Trust（伦敦皇家自由NHS基金会信托）
During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. This variation is seen in the CXR projections used, image annotations added and in the inspiratory effort and degree of rotation of clinical images. The image analysis community has attempted to ease the burden on overstretched radiology departments during the pandemic by developing automated COVID-19 diagnostic algorithms, the input for which has been CXR imaging. Large publicly available CXR datasets have been leve…
University of California, Los Angeles（加州大学洛杉矶分校）；VA Phoenix Healthcare System（凤凰城退伍军人医疗系统）
Treatment of acute ischemic strokes (AIS) is largely contingent upon the time since stroke onset (TSS). However, TSS may not be readily available in up to 25% of patients with unwitnessed AIS. Current clinical guidelines for patients with unknown TSS recommend the use of MRI to determine eligibility for thrombolysis, but radiology assessments have high inter-reader variability. In this work, we present deep learning models that leverage MRI diffusion series to classify TSS based on clinically validated thresholds. We propose an intra-domain task-adaptive transfer learning method, which involve…
Johns Hopkins University（约翰斯·霍普金斯大学）；University of Maryland School of Medicine（马里兰大学医学院）
Modern machine learning pipelines, in particular those based on deep learning (DL) models, require large amounts of labeled data. For classification problems, the most common learning paradigm consists of presenting labeled examples during training, thus providing strong supervision on what constitutes positive and negative samples. This constitutes a major obstacle for the development of DL models in radiology--in particular for cross-sectional imaging (e.g., computed tomography [CT] scans)--where labels must come from manual annotations by expert radiologists at the image or slice-level. The…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, …
Foundation Models in Medical Imaging: A Review and Outlook
Netherlands Cancer Institute（荷兰癌症研究所）；Radboud University Medical Center（拉德堡德大学医学中心）；Delft University of Technology（代尔夫特理工大学）；University of Amsterdam（阿姆斯特丹大学）；Oncode Institute（Oncode研究所）
SUNY Buffalo（纽约州立大学布法罗分校）；Medical College of Wisconsin（威斯康星医学院）；Washington University School of Medicine（华盛顿大学医学院）；University of California, Davis Medical Center（加州大学戴维斯医学中心）
Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We …
DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysis
