NiftyNet: a deep-learning platform for medical imaging
University College London（伦敦大学学院）；National Hospital for Neurology and Neurosurgery（国家神经病学和神经外科医院）
Eli Gibson, Wenqi Li, Carole Sudre, Lucas Fidon, Dzhoshkun I. Shakir, Guotai Wang, Zach Eaton-Rosen, Robert Gray, Tom Doel, Yipeng Hu, Tom Whyntie, Parashkev Na… 展开作者
Eli Gibson, Wenqi Li, Carole Sudre, Lucas Fidon, Dzhoshkun I. Shakir, Guotai Wang, Zach Eaton-Rosen, Robert Gray, Tom Doel, Yipeng Hu, Tom Whyntie, Parashkev Nachev, Marc Modat, Dean C. Barratt, Sébastien Ourselin, M. Jorge Cardoso, Tom Vercauteren
备注：
Wenqi Li and Eli Gibson contributed equally to this work. M. Jorge Cardoso and Tom Vercauteren contributed equally to this work. 26 pages, 6 figures; Update includes additional applications, updated author list and formatting for journal submission
Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this application requires substantial implementation effort. Thus, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon.
NiftyNet provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D and 3D images and computational graphs by default.
We present 3 illustrative medical image analysis applications built using NiftyNet: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses.
NiftyNet enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.
Peng Cheng Laboratory（鹏城实验室）；Harbin Institute of Technology(Shenzhen)（哈尔滨工业大学（深圳））；National University of Defense Technology（国防科技大学）；Peking University（北京大学）
In this paper, we present OpenMedIA, an open-source toolbox library containing a rich set of deep learning methods for medical image analysis under heterogeneous Artificial Intelligence (AI) computing platforms. Various medical image analysis methods, including 2D/3D medical image classification, segmentation, localisation, and detection, have been included in the toolbox with PyTorch and/or MindSpore implementations under heterogeneous NVIDIA and Huawei Ascend computing systems. To our best knowledge, OpenMedIA is the first open-source algorithm library providing compared PyTorch and MindSpor…
Xidian University（西安电子科技大学）；Shanghai BNC（上海BNC）；Murdoch University（默多克大学）；The University of Western Australia（西澳大利亚大学）；The University of Tokyo（东京大学）；Imperial College London（帝国理工学院）；University of Augsburg（奥格斯堡大学）
In the past decade, deep learning (DL) has achieved unprecedented success in numerous fields including computer vision, natural language processing, and healthcare. In particular, DL is experiencing an increasing development in applications for advanced medical image analysis in terms of analysis, segmentation, classification, and furthermore. On the one hand, tremendous needs that leverage the power of DL for medical image analysis are arising from the research community of a medical, clinical, and informatics background to jointly share their expertise, knowledge, skills, and experience. On …
All-in-one platform for AI R&D in medical imaging, encompassing data collection, selection, annotation, and pre-processing
Callisto Inc（卡利斯托公司）；The University of Tokyo（东京大学）；St. Marianna University School of Medicine（圣玛丽安娜医科大学医学院）；Harada Academy（原田学园）；St. Marianna University Yokohama Seibu Hospital（圣玛丽安娜大学横滨西部医院）
King Fahd University of Petroleum & Minerals（法赫德国王石油与矿业大学）；Tilburg University（蒂尔堡大学）；Banglalink Digital Communications Limited（Banglalink数字通信有限公司）
In this paper, we implemented both sequential and parallel version of fractal image compression algorithms using CUDA (Compute Unified Device Architecture) programming model for parallelizing the program in Graphics Processing Unit for medical images, as they are highly similar within the image itself. There are several improvement in the implementation of the algorithm as well. Fractal image compression is based on the self similarity of an image, meaning an image having similarity in majority of the regions. We take this opportunity to implement the compression algorithm and monitor the effe…
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；University Hospital of Marburg（马尔堡大学医院）；Isomics, Inc.（Isomics公司）；GE Research（通用电气研究）
Volumetric change in glioblastoma multiforme (GBM) over time is a critical factor in treatment decisions. Typically, the tumor volume is computed on a slice-by-slice basis using MRI scans obtained at regular intervals. (3D)Slicer - a free platform for biomedical research - provides an alternative to this manual slice-by-slice segmentation process, which is significantly faster and requires less user interaction. In this study, 4 physicians segmented GBMs in 10 patients, once using the competitive region-growing based GrowCut segmentation module of Slicer, and once purely by drawing boundaries …
MAIA: A Collaborative Medical AI Platform for Integrated Healthcare Innovation
Department of Biomedical Engineering and Health Systems, KTH, Royal Institute of Technology（生物医学工程与健康系统系，皇家理工学院）；Department of Clinical Sciences, Intervention and Engineering, Karolinska Institutet（临床科学、干预与工程系，卡罗林斯卡研究所）；Division of Medical Radiation Physics, Department of Physics, Stockholm University（医学放射物理 division，物理系，斯德哥尔摩大学）；Department of Oncology-Pathology, Karolinska Institutet（肿瘤学-病理学系，卡罗林斯卡研究所）；Department of Radiology, Karolinska University Hospital（放射学系，卡罗林斯卡大学医院）；Department of Molecular Medicine and Surgery, Karolinska Institutet（分子医学与外科系，卡罗林斯卡研究所）；Department of Neurobiology, Care Sciences and Society, Karolinska Institutet（神经生物学、护理科学与社会系，卡罗林斯卡研究所）
本文提出开源协作平台 MAIA，基于 Kubernetes 集成数据、建模、标注、部署与临床反馈工具，以推动医学影像 AI 从研究到临床的可复现转化。
MHub.ai: A Simple, Standardized, and Reproducible Platform for AI Models in Medical Imaging
Mass General Brigham, Harvard Medical School（麻省总布莱根医院、哈佛医学院）；Maastricht University（马斯特里赫特大学）；Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School（丹娜-法伯癌症研究所、布莱根妇女医院、哈佛医学院）；KnowledgeVis, LLC（KnowledgeVis有限责任公司）；Isomics, Inc.（Isomics公司）；Brigham and Women’s Hospital, Harvard Medical School（布莱根妇女医院、哈佛医学院）；Radboud University Medical Center（拉德堡德大学医学中心）；BAMF Health LLC（BAMF Health有限责任公司）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）；Technical University of Munich（慕尼黑工业大学）；German Heart Center（德国心脏中心）
Kaapana: A Comprehensive Open-Source Platform for Integrating AI in Medical Imaging Research Environments
Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany（德国癌症研究中心医学影像计算部，海德堡，德国）；Medical Faculty, University of Heidelberg, Heidelberg, Germany（海德堡大学医学系，海德堡，德国）；Computer Science Faculty, University of Heidelberg, Heidelberg, Germany（海德堡大学计算机科学系，海德堡，德国）；Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany（放射肿瘤学部门模式分析与学习组，海德堡大学医院，海德堡，德国）；German Cancer Consortium (DKTK), Partner Site Heidelberg, Heidelberg, Germany（德国癌症联盟（DKTK），海德堡合作伙伴站点，海德堡，德国）；Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany（海德堡放射肿瘤学研究所（HIRO），肿瘤放射研究国家中心（NCRO），海德堡，德国）；National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and the University Medical Center Heidelberg, 69120 Heidelberg, Germany（肿瘤疾病国家中心（NCT），海德堡NCT，德克萨斯儿童医学研究中心（DKFZ）与海德堡大学医学中心之间的合作，海德堡，德国）
The University of Tokyo（东京大学）；The University of Tokyo Hospital（东京大学医院）
A computer-aided detection (CAD) system based on machine learning is expected to assist radiologists in making a diagnosis. It is desirable to build CAD systems for the various types of diseases accumulating daily in a hospital. An obstacle in developing a CAD system for a disease is that the number of medical images is typically too small to improve the performance of the machine learning model. In this paper, we aim to explore ways to address this problem through a sim2real transfer approach in medical image fields. To build a platform to evaluate the performance of sim2real transfer methods…
Graz University of Technology（格拉茨工业大学）；Medical University of Graz（格拉茨医科大学）
We introduce a fully automatic system for cranial implant design, a common task in cranioplasty operations. The system is currently integrated in Studierfenster (http://studierfenster.tugraz.at/), an online, cloud-based medical image processing platform for medical imaging applications. Enhanced by deep learning algorithms, the system automatically restores the missing part of a skull (i.e., skull shape completion) and generates the desired implant by subtracting the defective skull from the completed skull. The generated implant can be downloaded in the STereoLithography (.stl) format directl…
Faculty of Science, Technology and Communication（科学、技术与传播学院）
Within this thesis we propose a platform for combining Augmented Reality (AR) hardware with machine learning in a user-oriented pipeline, offering to the medical staff an intuitive 3D visualization of volumetric Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) medical image segmentations inside the AR headset, that does not need human intervention for loading, processing and segmentation of medical images. The AR visualization, based on Microsoft HoloLens, employs a modular and thus scalable frontend-backend architecture for real-time visualizations on multiple AR headsets. As Con…
Imperial College London（伦敦帝国学院）；Aichi Cancer Center（爱知县癌症中心）；Nagoya University（名古屋大学）
To efficiently establish training databases for machine learning methods, collaborative and crowdsourcing platforms have been investigated to collectively tackle the annotation effort. However, when this concept is ported to the medical imaging domain, reading expertise will have a direct impact on the annotation accuracy. In this study, we examine the impact of expertise and the amount of available annotations on the accuracy outcome of a liver segmentation problem in an abdominal computed tomography (CT) image database. In controlled experiments, we study this impact for different types of w…
Radiuma: A Unified Zero-Code Executable Graphical Workflow Generator for Reproducible and Shareable Medical Image Analysis and Machine Learning
Department of Basic and Translational Research, BC Cancer Research Institute（基础与转化研究部，BC癌症研究中心）；Department of Radiology, University of British Columbia（放射科，不列颠哥伦比亚大学）；Technological Virtual Collaboration (TECVICO Corp.)（技术虚拟协作（TECVICO公司））；Department of Computer Science, University of British Columbia（计算机科学系，不列颠哥伦比亚大学）；Department of Cardiology, Inselspital, Bern University Hospital, University of Bern（心脏病学部，Inselspital，伯恩大学医院，伯恩大学）；Department of Digital Medicine, University of Bern（数字医学系，伯恩大学）；Departments of Physics & Biomedical Engineering, University of British Columbia（物理学与生物医学工程系，不列颠哥伦比亚大学）
We examine the Xeon Phi, which is based on Intel's Many Integrated Cores architecture, for its suitability to run the FDK algorithm--the most commonly used algorithm to perform the 3D image reconstruction in cone-beam computed tomography. We study the challenges of efficiently parallelizing the application and means to enable sensible data sharing between threads despite the lack of a shared last level cache. Apart from parallelization, SIMD vectorization is critical for good performance on the Xeon Phi; we perform various micro-benchmarks to investigate the platform's new set of vector instru…
To conduct a radiomics or deep learning research experiment, the radiologists or physicians need to grasp the needed programming skills, which, however, could be frustrating and costly when they have limited coding experience. In this paper, we present DARWIN, a flexible research platform with a graphical user interface for medical imaging research. Our platform is consists of a radiomics module and a deep learning module. The radiomics module can extract more than 1000 dimension features(first-, second-, and higher-order) and provided many draggable supervised and unsupervised machine learnin…
MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
University of Science and Technology of China（中国科学技术大学）；Eastern Institute of Technology（东方理工大学）；Shanghai Jiao Tong University（上海交通大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）
OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；University of Electronic Science and Technology of China（电子科技大学）；Xi’an Jiaotong University（西安交通大学）；Zhejiang Laboratory（浙江实验室）；Fudan University（复旦大学）；Chinese University of Hong Kong（香港中文大学）；University of British Columbia（不列颠哥伦比亚大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；East China University Of Science And Technology（华东理工大学）；University College London（伦敦大学学院）；Guangzhou Laboratory（广州实验室）；The First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院）；West China Hospital, Sichuan University（四川大学华西医院）；Xinhua Hospital, Shanghai Jiaotong University School of Medicine（上海交通大学医学院附属新华医院）；Ruijin Hospital, Shanghai Jiaotong University School of Medicine（上海交通大学医学院附属瑞金医院）
Heidelberg University（海德堡大学）；German Cancer Research Center（德国癌症研究中心）
One of the greatest challenges in the medical imaging domain is to successfully transfer deep learning models into clinical practice. Since models are often trained on a specific body region, a robust transfer into the clinic necessitates the selection of images with body regions that fit the algorithm to avoid false-positive predictions in unknown regions. Due to the insufficient and inaccurate nature of manually-defined imaging meta-data, automated body part recognition is a key ingredient towards the broad and reliable adoption of medical deep learning models. While some approaches to this …
UniCAD: Efficient and Extendable Architecture for Multi-Task Computer-Aided Diagnosis System
School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University（生物医学工程学院及先进医疗材料与设备国家重点实验室，上海科技大学）；School of Biomedical Engineering, Shanghai Jiao Tong University（生物医学工程学院，上海交通大学）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）；Shanghai United Imaging Intelligence Co. Ltd.（上海联合影像智能科技有限公司）
Transition from conventional to digital pathology requires a new category of biomedical informatic infrastructure which could facilitate delicate pathological routine. Pathological diagnoses are sensitive to many external factors and is known to be subjective. Only systems that can meet strict requirements in pathology would be able to run along pathological routines and eventually digitized the study area, and the developed platform should comply with existing pathological routines and international standards. Currently, there are a number of available software tools which can perform histopa…
Johns Hopkins University（约翰斯·霍普金斯大学）；King’s College London（伦敦国王学院）；Tsinghua University（清华大学）
The Segment Anything Model (SAM) has drawn significant attention from researchers who work on medical image segmentation because of its generalizability. However, researchers have found that SAM may have limited performance on medical images compared to state-of-the-art non-foundation models. Regardless, the community sees potential in extending, fine-tuning, modifying, and evaluating SAM for analysis of medical imaging. An increasing number of works have been published focusing on the mentioned four directions, where variants of SAM are proposed. To this end, a unified platform helps push the…
MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentation
Yale University（耶鲁大学）；University of Florida（佛罗里达大学）；Wichita State University（威奇托州立大学）；Purdue University（普渡大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Stevens Institute of Technology（史蒂文斯理工学院）；Zhejiang University（浙江大学）
The increased availability and usage of modern medical imaging induced a strong need for automatic medical image segmentation. Still, current image segmentation platforms do not provide the required functionalities for plain setup of medical image segmentation pipelines. Already implemented pipelines are commonly standalone software, optimized on a specific public data set. Therefore, this paper introduces the open-source Python library MIScnn. The aim of MIScnn is to provide an intuitive API allowing fast building of medical image segmentation pipelines including data I/O, preprocessing, data…
Nanjing University of Science and Technology（南京理工大学）
Segmentation is one of the most important and popular tasks in medical image analysis, which plays a critical role in disease diagnosis, surgical planning, and prognosis evaluation. During the past five years, on the one hand, thousands of medical image segmentation methods have been proposed for various organs and lesions in different medical images, which become more and more challenging to fairly compare different methods. On the other hand, international segmentation challenges can provide a transparent platform to fairly evaluate and compare different methods. In this paper, we present a …
Thomas Jefferson High School for Science and Technology（托马斯·杰斐逊科学技术高中）
According to the WHO, Cerebrovascular Stroke, or CS, is the second largest cause of death worldwide. Current diagnosis of CS relies on labor and cost intensive neuroimaging techniques, unsuitable for areas with inadequate access to quality medical facilities. Thus, there is a great need for an efficient diagnosis alternative. StrokeSave is a platform for users to self-diagnose for prevalence to stroke. The mobile app is continuously updated with heart rate, blood pressure, and blood oxygen data from sensors on the patient wrist. Once these measurements reach a threshold for possible stroke, th…
Generative AI for Medical Imaging: extending the MONAI Framework
King’s College London（伦敦国王学院）；National Institute of Mental Health（美国国家精神卫生研究所）；The University of Edinburgh（爱丁堡大学）；University of Basel（巴塞尔大学）；Korea Advanced Institute of Science & Technology（韩国科学技术院）；NVIDIA Corporation（英伟达公司）；Stanford University（斯坦福大学）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）
