Pseudo-healthy synthesis is the task of creating a subject-specific `healthy' image from a pathological one. Such images can be helpful in tasks such as anomaly detection and understanding changes induced by pathology and disease. In this paper, we present a model that is encouraged to disentangle the information of pathology from what seems to be healthy. We disentangle what appears to be healthy and where disease is as a segmentation map, which are then recombined by a network to reconstruct the input disease image. We train our models adversarially using either paired or unpaired settings, where we pair disease images and maps when available. We quantitatively and subjectively, with a human study, evaluate the quality of pseudo-healthy images using several criteria. We show in a series of experiments, performed on ISLES, BraTS and Cam-CAN datasets, that our method is better than several baselines and methods from the literature. We also show that due to better training processes we could recover deformations, on surrounding tissue, caused by disease. Our implementation is publicly available at https://github.com/xiat0616/pseudo-healthy-synthesis. This paper has been accepted by Medical Image Analysis: https://doi.org/10.1016/j.media.2020.101719.
University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Pseudo healthy synthesis, i.e. the creation of a subject-specific `healthy' image from a pathological one, could be helpful in tasks such as anomaly detection, understanding changes induced by pathology and disease or even as data augmentation. We treat this task as a factor decomposition problem: we aim to separate what appears to be healthy and where disease is (as a map). The two factors are then recombined (by a network) to reconstruct the input disease image. We train our models in an adversarial way using either paired or unpaired settings, where we pair disease images and maps (as segme…
McGill University（麦吉尔大学）；MILA (Québec Artificial Intelligence Institute)（MILA（魁北克人工智能研究所））；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain lesions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternat…
Enhancing Corpus Callosum Segmentation in Fetal MRI via Pathology-Informed Domain Randomization
Univeristat Politècnica de Catalunya, Barcelona, Spain（巴塞罗那理工大学）；Department of Radiology, Lausanne University Hospital and University of Lausanne (UNIL), Lausanne, Switzerland（拉索恩大学医院放射科）；University of Lausanne (UNIL), Lausanne, Switzerland（洛桑大学）；CIBM Center for Biomedical Imaging, Lausanne, Switzerland（瑞士拉索恩生物医学成像中心）；BCNatal Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), Universitat de Barcelona, Barcelona, Spain（巴塞罗那大学BCNatal胎儿医学研究中心）；Department Woman-Mother-Child, CHUV, Lausanne, Switzerland（瑞士拉索恩大学医院妇女-母亲-儿童部门）
Johns Hopkins University（约翰斯·霍普金斯大学）；Sigma Technologies（12西格玛科技）；Johns Hopkins School of Medicine（约翰斯·霍普金斯医学院）
Medical images are often used to detect and characterize pathology and disease; however, automatically identifying and segmenting pathology in medical images is challenging because the appearance of pathology across diseases varies widely. To address this challenge, we propose a Bayesian deep learning method that learns to translate healthy computed tomography images to magnetic resonance images and simultaneously calculates voxel-wise uncertainty. Since high uncertainty occurs in pathological regions of the image, this uncertainty can be used for unsupervised anomaly segmentation. We show enc…
Automatic segmentation of brain abnormalities is challenging, as they vary considerably from one pathology to another. Current methods are supervised and require numerous annotated images for each pathology, a strenuous task. To tackle anatomical variability, Unsupervised Anomaly Detection (UAD) methods are proposed, detecting anomalies as outliers of a healthy model learned using a Variational Autoencoder (VAE). Previous work on UAD adopted a 2D approach, meaning that MRIs are processed as a collection of independent slices. Yet, it does not fully exploit the spatial information contained in …
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Maryland School of Dentistry（马里兰大学牙医学院）
Cycle reconstruction regularized adversarial training -- e.g., CycleGAN, DiscoGAN, and DualGAN -- has been widely used for image style transfer with unpaired training data. Several recent works, however, have shown that local distortions are frequent, and structural consistency cannot be guaranteed. Targeting this issue, prior works usually relied on additional segmentation or consistent feature extraction steps that are task-specific. To counter this, this work aims to learn a general add-on structural feature extractor, by explicitly enforcing the structural alignment between an input and it…
The Pennsylvania State University（宾夕法尼亚州立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Houston Methodist Hospital（休斯顿卫理公会医院）
Accurate infarct segmentation in non-contrast CT (NCCT) images is a crucial step toward computer-aided acute ischemic stroke (AIS) assessment. In clinical practice, bilateral symmetric comparison of brain hemispheres is usually used to locate pathological abnormalities. Recent research has explored asymmetries to assist with AIS segmentation. However, most previous symmetry-based work mixed different types of asymmetries when evaluating their contribution to AIS. In this paper, we propose a novel Asymmetry Disentanglement Network (ADN) to automatically separate pathological asymmetries and int…
Harnessing Intra-group Variations Via a Population-Level Context for Pathology Detection
Tsinghua University（清华大学）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Mechanical, Electrical & Information Engineering, Shandong University（山东大学机电与信息工程学院）；School of Automation, Hangzhou Dianzi University（杭州电子科技大学自动化学院）
Helmholtz AI（亥姆霍兹AI）；Helmholtz Center Munich（慕尼黑亥姆霍兹中心）；Klinikum Rechts der Isar（右岸医院）；Technical University of Munich (TUM)（慕尼黑工业大学）
In recent years, data-driven machine learning (ML) methods have revolutionized the computer vision community by providing novel efficient solutions to many unsolved (medical) image analysis problems. However, due to the increasing privacy concerns and data fragmentation on many different sites, existing medical data are not fully utilized, thus limiting the potential of ML. Federated learning (FL) enables multiple parties to collaboratively train a ML model without exchanging local data. However, data heterogeneity (non-IID) among the distributed clients is yet a challenge. To this end, we pro…
Bayer AG（拜耳股份公司）；Median Technologies（Median科技公司）；Bayer Crop Science（拜耳作物科学公司）；Pfizer Worldwide Research Development and Medical（辉瑞全球研发与医疗部门）
In pre-clinical pathology, there is a paradox between the abundance of raw data (whole slide images from many organs of many individual animals) and the lack of pixel-level slide annotations done by pathologists. Due to time constraints and requirements from regulatory authorities, diagnoses are instead stored as slide labels. Weakly supervised training is designed to take advantage of those data, and the trained models can be used by pathologists to rank slides by their probability of containing a given lesion of interest. In this work, we propose a novel contextualized eXplainable AI (XAI) f…
Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大大学埃尔朗根-纽伦堡分校）；Massachusetts Institute of Technology（麻省理工学院）；New England Eye Center（新英格兰眼科中心）；Tufts Medical Center（塔夫茨医疗中心）
We propose a novel method for non-rigid 3-D motion correction of orthogonally raster-scanned optical coherence tomography angiography volumes. This is the first approach that aligns predominantly axial structural features like retinal layers and transverse angiographic vascular features in a joint optimization. Combined with the use of orthogonal scans and favorization of kinematically more plausible displacements, the approach allows subpixel alignment and micrometer-scale distortion correction in all 3 dimensions. As no specific structures or layers are segmented, the approach is by design r…
Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in the reasoning behind medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attributes. Attribute-based methods enforcing attribute disentanglement have been proposed in the literature for classical computer vision tasks in benchmark data. In this paper, we propose a VAE approach, th…
Towards Segmenting the Invisible: An End-to-End Registration and Segmentation Framework for Weakly Supervised Tumour Analysis
Institute of Technical；Business Information Systems, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany Human Technopole, Milan, Italy Institute of Medical Engineering, Faculty of Electrical Engineering；Information Technology, Otto von Guericke University Magdeburg, Magdeburg, Germany Centre for Behavioural Brain Sciences, Magdeburg, Germany
New York University（纽约大学）；IIIT Hyderabad（印度海得拉巴国际信息技术研究所）
Systemic lupus erythematosus (SLE) is an autoimmune disease in which the immune system of the patient starts attacking healthy tissues of the body. Lupus Nephritis (LN) refers to the inflammation of kidney tissues resulting in renal failure due to these attacks. The International Society of Nephrology/Renal Pathology Society (ISN/RPS) has released a classification system based on various patterns observed during renal injury in SLE. Traditional methods require meticulous pathological assessment of the renal biopsy and are time-consuming. Recently, computational techniques have helped to allevi…
Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, USA（Athinoula A. Martinos 生物医学成像中心、麻省总医院和哈佛医学院，波士顿，美国）；Danish Research Centre for Magnetic Resonance, Department of Radiology and Nuclear Medicine（丹麦磁共振研究中心、放射学与核医学部门）；Copenhagen University Hospital– Amager and Hvidovre（哥本哈根大学医院–阿马格和赫维多尔）
Whole-body CT attenuation and volume charts from routine clinical scans via evidence-grounded LLM report filtering
Institute of Radiology, School of Medicine and Health, Technical University of Munich（医学与健康学院放射学研究所，慕尼黑技术大学）；Institute of Neuroradiology, School of Medicine and Health, Technical University of Munich（医学与健康学院神经放射学研究所，慕尼黑技术大学）；Munich Data Science Institute (MDSI), Technical University of Munich（慕尼黑数据科学研究所（MDSI），慕尼黑技术大学）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML），德国慕尼黑）
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Friedrich–Alexander University Erlangen–Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Supervised learning of every possible pathology is unrealistic for many primary care applications like health screening. Image anomaly detection methods that learn normal appearance from only healthy data have shown promising results recently. We propose an alternative to image reconstruction-based and image embedding-based methods and propose a new self-supervised method to tackle pathological anomaly detection. Our approach originates in the foreign patch interpolation (FPI) strategy that has shown superior performance on brain MRI and abdominal CT data. We propose to use a better patch inte…
Recent works show that Generative Adversarial Networks (GANs) can be successfully applied to chest X-ray data augmentation for lung disease recognition. However, the implausible and distorted pathology features generated from the less than perfect generator may lead to wrong clinical decisions. Why not keep the original pathology region? We proposed a novel approach that allows our generative model to generate high quality plausible images that contain undistorted pathology areas. The main idea is to design a training scheme based on an image-to-image translation network to introduce variation…
We propose to learn a probabilistic motion model from a sequence of images. Besides spatio-temporal registration, our method offers to predict motion from a limited number of frames, useful for temporal super-resolution. The model is based on a probabilistic latent space and a novel temporal dropout training scheme. This enables simulation and interpolation of realistic motion patterns given only one or any subset of frames of a sequence. The encoded motion also allows to be transported from one subject to another without the need of inter-subject registration. An unsupervised generative defor…
Xiamen University（厦门大学）；Westlake University（西湖大学）；Chinese PLA General Hospital（中国人民解放军总医院）；Deepwise AI Laboratory（深睿人工智能实验室）；The University of Hong Kong（香港大学）
Synthesizing a subject-specific pathology-free image from a pathological image is valuable for algorithm development and clinical practice. In recent years, several approaches based on the Generative Adversarial Network (GAN) have achieved promising results in pseudo-healthy synthesis. However, the discriminator (i.e., a classifier) in the GAN cannot accurately identify lesions and further hampers from generating admirable pseudo-healthy images. To address this problem, we present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pse…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, …
Unsupervised Anomaly Detection in Brain MRI via Disentangled Anatomy Learning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China.（自动化与智能感知学院，上海交通大学，中国）；School of Computer Science, The University of Sydney, Sydney, Australia.（计算机科学学院，悉尼大学，澳大利亚）；Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.（放射科，仁济医院，医学院，上海交通大学，中国）；Biomedical Engineering Department, Tulane University, New Orleans, USA.（生物医学工程系， Tulane 大学，美国新奥尔良）
Imperial College London（帝国理工学院）；MRC London Clinical Sciences Centre（MRC伦敦临床科学中心）；National Heart & Lung Institute（国家心肺研究所）；Duke-NUS Graduate Medical School（杜克-新加坡国立大学研究生医学院）
Alterations in the geometry and function of the heart define well-established causes of cardiovascular disease. However, current approaches to the diagnosis of cardiovascular diseases often rely on subjective human assessment as well as manual analysis of medical images. Both factors limit the sensitivity in quantifying complex structural and functional phenotypes. Deep learning approaches have recently achieved success for tasks such as classification or segmentation of medical images, but lack interpretability in the feature extraction and decision processes, limiting their value in clinical…
Apple orchards in the U.S. are under constant threat from a large number of pathogens and insects. Appropriate and timely deployment of disease management depends on early disease detection. Incorrect and delayed diagnosis can result in either excessive or inadequate use of chemicals, with increased production costs, environmental, and health impacts. We have manually captured 3,651 high-quality, real-life symptom images of multiple apple foliar diseases, with variable illumination, angles, surfaces, and noise. A subset, expert-annotated to create a pilot dataset for apple scab, cedar apple ru…
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Mannheim University of Applied Science（曼海姆应用科学大学）
Cardiac magnetic resonance imaging improves on diagnosis of cardiovascular diseases by providing images at high spatiotemporal resolution. Manual evaluation of these time-series, however, is expensive and prone to biased and non-reproducible outcomes. In this paper, we present a method that addresses named limitations by integrating segmentation and disease classification into a fully automatic processing pipeline. We use an ensemble of UNet inspired architectures for segmentation of cardiac structures such as the left and right ventricular cavity (LVC, RVC) and the left ventricular myocardium…
Université Paris-Saclay（巴黎萨克雷大学）；Télécom Paris（巴黎电信学院）；Johannes-Gutenberg Univ. of Mainz（美因茨约翰内斯·古腾堡大学）；University of Milan（米兰大学）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CHU Grenoble Alpe（格勒诺布尔阿尔卑斯大学中心医院）；Centre for Neuroimaging & Cognitive Genomics (NICOG)（神经影像与认知基因组学中心）
Traditional supervised learning with deep neural networks requires a tremendous amount of labelled data to converge to a good solution. For 3D medical images, it is often impractical to build a large homogeneous annotated dataset for a specific pathology. Self-supervised methods offer a new way to learn a representation of the images in an unsupervised manner with a neural network. In particular, contrastive learning has shown great promises by (almost) matching the performance of fully-supervised CNN on vision tasks. Nonetheless, this method does not take advantage of available meta-data, suc…
We propose a novel framework for controllable pathological image synthesis for data augmentation. Inspired by CycleGAN, we perform cycle-consistent image-to-image translation between two domains: healthy and pathological. Guided by a semantic mask, an adversarially trained generator synthesizes pathology on a healthy image in the specified location. We demonstrate our approach on an institutional dataset of cerebral microbleeds in traumatic brain injury patients. We utilize synthetic images generated with our method for data augmentation in cerebral microbleeds detection. Enriching the trainin…
Principled Feature Disentanglement for High-Fidelity Unified Brain MRI Synthesis
School of Computing, Korea Advanced Institute of Science（韩国科学技术院计算机学院）；Samsung Research, Seoul 06765, Republic of Korea（三星研究院）；Gordon Center for Medical Imaging, Massachusetts General Hospital（麻省总医院影像中心）；Harvard Medical School, Boston, MA 02114, USA（哈佛医学院）
Introducing NCL-SM: A Fully Annotated Dataset of Images from Human Skeletal Muscle Biopsies
Wellcome Centre for Mitochondrial Research（惠康线粒体研究中心）；Translational and Clinical Research Institute（转化与临床研究所）；Faculty of Medical Sciences（医学院）；Newcastle University（纽卡斯尔大学）；School of Computing（计算机学院）
Facebook Artificial Intelligence（脸书人工智能）；University of Southern California（南加州大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Deformable registration of magnetic resonance images between patients with brain tumors and healthy subjects has been an important tool to specify tumor geometry through location alignment and facilitate pathological analysis. Since tumor region does not match with any ordinary brain tissue, it has been difficult to deformably register a patients brain to a normal one. Many patient images are associated with irregularly distributed lesions, resulting in further distortion of normal tissue structures and complicating registration's similarity measure. In this work, we follow a multi-step contex…
Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training
Department of Engineering Science, University of Oxford, Oxford, UK（工程科学系，牛津大学，牛津，英国）；Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK（牛津大学临床神经科学系，牛津，英国）
CADD: Context aware disease deviations via restoration of brain images using normative conditional diffusion models
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School（阿提诺拉A.马丁纳生物医学成像中心，麻省总医院和哈佛医学院）；Hawkes Institute, University College London（霍克斯研究所，伦敦大学学院）；Mecha Health（Mecha健康）；Computer Science & Artificial Intelligence Lab, Massachusetts Institute of Technology（计算机科学与人工智能实验室，麻省理工学院）；Department of Radiology, the University of Tokyo（东京大学放射科）；Medical Engineering, Chiba University（千叶大学医学工程）；Department of Computational Diagnostic Radiology and Preventive Medicine, the University of Tokyo Hospital（东京大学医院计算诊断放射学与预防医学科）
ADPv2: A Hierarchical Histological Tissue Type-Annotated Dataset for Potential Biomarker Discovery of Colorectal Disease
Department of Computer Science \& Software Engineering, Concordia University, 2155 Guy St, Montreal, QC H3H 2L9, Canada；Department of Electrical \& Computer Engineering, University of Toronto, 10 King's College Rd, Toronto, ON M5S 3G8, Canada；Department of Chemistry \& Biology, Toronto Metropolitan University, 350 Victoria St. Toronto, ON M5B 2K3, Canada；Department of Medicine, Université de Montréal, Pavillon Roger-Gaudry, 2900 Edouard Montpetit Blvd, Montreal, QC H3T 1J4, Canada；Axe Cancer, Centre de recherche du CHUM, 900 Saint-Denis St, Montréal, QC H2X 0A9, Canada；Anatomic Pathology, Sunnybrook Health Sciences Centre, 2075 Bayview Ave, Toronto, ON M4N 3M5, Canada；Department of Laboratory Medicine \& Pathobiology, University of Toronto, Simcoe Hall, 1 King's College Circle, Toronto, ON M5S 3K3, Canada；Department of Pathology \& Molecular Medicine, Queen’s University, 88 Stuart Street Queen's University Kingston, ON K7L 3N6 Canada；Department of Pathology \& Molecular Medicine, Université de Montréal, 2900 Édouard-Montpetit Blvd, Montréal, QC H3T 1J4, Canada
Tau Anomaly Detection in PET Imaging via Bilateral-Guided Deterministic Diffusion Model
Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California（斯蒂文斯神经影像与信息学研究所，凯克医学院，南加州大学）；Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California（明希部门电气与计算机工程系，维特比工程学院，南加州大学）；Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California（阿尔弗雷德·E·曼生物医学工程系，维特比工程学院，南加州大学）
Generative Adversarial Networks for Weakly Supervised Generation and Evaluation of Brain Tumor Segmentations on MR Images
Institute of Medical Science, University of Toronto（多伦多大学医学科学研究所）；The Hospital for Sick Children（病童医院）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Vector Institute（矢量研究所）；Department of Medical Imaging, University of Toronto（多伦多大学医学影像系）；The Arthur and Sonia Labatt Brain Tumor Research Centre, The Hospital for Sick Children（病童医院亚瑟与索尼娅·拉巴特脑肿瘤研究中心）；Division of Hematology/Oncology, The Hospital for Sick Children（病童医院血液学/肿瘤学科）；Developmental and Stem Cell Biology Program, The Hospital for Sick Children（病童医院发育与干细胞生物学项目）；Department of Medical Biophysics, University of Toronto（多伦多大学医学生物物理系）
