The 3D morphology and quantitative assessment of knee articular cartilages (i.e., femoral, tibial, and patellar cartilage) in magnetic resonance (MR) imaging is of great importance for knee radiographic osteoarthritis (OA) diagnostic decision making. However, effective and efficient delineation of all the knee articular cartilages in large-sized and high-resolution 3D MR knee data is still an open challenge. In this paper, we propose a novel framework to solve the MR knee cartilage segmentation task. The key contribution is the adversarial learning based collaborative multi-agent segmentation network. In the proposed network, we use three parallel segmentation agents to label cartilages in their respective region of interest (ROI), and then fuse the three cartilages by a novel ROI-fusion layer. The collaborative learning is driven by an adversarial sub-network. The ROI-fusion layer not only fuses the individual cartilages from multiple agents, but also backpropagates the training loss from the adversarial sub-network to each agent to enable joint learning of shape and spatial constraints. Extensive evaluations are conducted on a dataset including hundreds of MR knee volumes with diverse populations, and the proposed method shows superior performance.
Brigham and Women's Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
Background: MRI is the modality of choice for cartilage imaging; however, its diagnostic performance is variable and significantly lower than the gold standard diagnostic knee arthroscopy. In recent years, deep learning has been used to automatically interpret medical images to improve diagnostic accuracy and speed. Purpose: The primary purpose of this study was to evaluate whether deep learning applied to the interpretation of knee MRI images can be utilized to identify cartilage defects accurately. Methods: We analyzed data from patients who underwent knee MRI evaluation and consequently had…
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属同仁医院影像科）；School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）
Knee osteoarthritis (OA) is the most common osteoarthritis and a leading cause of disability. Cartilage defects are regarded as major manifestations of knee OA, which are visible by magnetic resonance imaging (MRI). Thus early detection and assessment for knee cartilage defects are important for protecting patients from knee OA. In this way, many attempts have been made on knee cartilage defect assessment by applying convolutional neural networks (CNNs) to knee MRI. However, the physiologic characteristics of the cartilage may hinder such efforts: the cartilage is a thin curved layer, implying…
Harbin Institute of Technology（哈尔滨工业大学）；University of Western Ontario（西安大略大学）
The morphological changes in knee cartilage (especially femoral and tibial cartilages) are closely related to the progression of knee osteoarthritis, which is expressed by magnetic resonance (MR) images and assessed on the cartilage segmentation results. Thus, it is necessary to propose an effective automatic cartilage segmentation model for longitudinal research on osteoarthritis. In this research, to relieve the problem of inaccurate discontinuous segmentation caused by the limited receptive field in convolutional neural networks, we proposed a novel position-prior clustering-based self-atte…
Analyzing knee cartilage thickness and strain under load can help to further the understanding of the effects of diseases like Osteoarthritis. A precise segmentation of the cartilage is a necessary prerequisite for this analysis. This segmentation task has mainly been addressed in Magnetic Resonance Imaging, and was rarely investigated on contrast-enhanced Computed Tomography, where contrast agent visualizes the border between femoral and tibial cartilage. To overcome the main drawback of manual segmentation, namely its high time investment, we propose to use a 3D Convolutional Neural Network …
New York University School of Medicine（纽约大学医学院）；New York University（纽约大学）
The integrity of articular cartilage is a crucial aspect in the early diagnosis of osteoarthritis (OA). Many novel MRI techniques have the potential to assess compositional changes of the cartilage extracellular matrix. Among these techniques, diffusion tensor imaging (DTI) of cartilage provides a simultaneous assessment of the two principal components of the solid matrix: collagen structure and proteoglycan concentration. DTI, as for any other compositional MRI technique, require a human expert to perform segmentation manually. The manual segmentation is error-prone and time-consuming ($\sim$…
The University of Queensland（昆士兰大学）；Australian e-Health Research Centre（澳大利亚电子健康研究中心）；Commonwealth Scientific and Industrial Research Organisation（联邦科学与工业研究组织）
In medical image analysis, automated segmentation of multi-component anatomical structures, which often have a spectrum of potential anomalies and pathologies, is a challenging task. In this work, we develop a multi-step approach using U-Net-based neural networks to initially detect anomalies (bone marrow lesions, bone cysts) in the distal femur, proximal tibia and patella from 3D magnetic resonance (MR) images of the knee in individuals with varying grades of osteoarthritis. Subsequently, the extracted data are used for downstream tasks involving semantic segmentation of individual bone and c…
Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学医学院生物医学工程学院医学影像技术研究所）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）；Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属同仁医院影像科）
Knee osteoarthritis (OA) is one of the most common musculoskeletal disorders and requires early-stage diagnosis. Nowadays, the deep convolutional neural networks have achieved greatly in the computer-aided diagnosis field. However, the construction of the deep learning models usually requires great amounts of annotated data, which is generally high-cost. In this paper, we propose a novel approach for knee cartilage defects assessment, including severity classification and lesion localization. This can be treated as a subtask of knee OA diagnosis. Particularly, we design a self-ensembling frame…
MonoUNet: A Robust Tiny Neural Network for Automated Knee Cartilage Segmentation on Point-of-Care Ultrasound Devices
School of Biomedical Engineering, The University of British Columbia（生物医学工程学院，不列颠哥伦比亚大学）；Department of Kinesiology, Michigan State University（运动科学系，密歇根州立大学）；Department of Rheumatology, The University of British Columbia（风湿病学系，不列颠哥伦比亚大学）；Department of Radiology, The University of British Columbia（放射学系，不列颠哥伦比亚大学）；Department of Medicine, The University of British Columbia（医学系，不列颠哥伦比亚大学）
Reliability assessment and multicenter clinical application of magnetic resonance methods for knee cartilage quantification
Academy for Clinical Innovation and Translation of Shanghai Co., Ltd. (ACITS)（上海临床创新与转化研究院有限公司）；Shanghai IxCell Biotechnology Co., LTD（上海爱科百发生物科技有限公司）；Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第九人民医院）；Nanyang Technological University（南洋理工大学）；Renji Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属仁济医院）；Shanghai Jiading District Central Hospital（上海市嘉定区中心医院）
Stanford University（斯坦福大学）；University of Central Florida（中佛罗里达大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Minnesota（明尼苏达大学）；University of Copenhagen（哥本哈根大学）；Cleveland Clinic（克利夫兰诊所）；New York University（纽约大学）；Langone Health（朗格尼健康中心）
Purpose: To organize a knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at two timepoints with ground-truth articular (femoral, tibial, patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a hold-out test set. Simil…
ITMO University（圣彼得堡国立信息技术机械与光学大学）；Federal Almazov North-West Medical Research Center（阿尔马佐夫西北联邦医学研究中心）；Aix-Marseille Universite（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）
Detection of cartilage loss is crucial for the diagnosis of osteo- and rheumatoid arthritis. A large number of automatic segmentation tools have been reported so far for cartilage assessment in magnetic resonance images of large joints. As compared to knee or hip, wrist cartilage has a more complex structure so that automatic tools developed for large joints are not expected to be operational for wrist cartilage segmentation. In that respect, a fully automatic wrist cartilage segmentation method would be of high clinical interest. We assessed the performance of four optimized variants of the U…
Schulich School of Engineering, University of Calgary（卡尔加里大学舒立克工程学院）；Department of Radiology & Diagnostic Imaging, University of Alberta（阿尔伯塔大学放射与诊断成像系）；Alberta Machine Intelligence Institute (AMII), University of Alberta（阿尔伯塔大学阿尔伯塔机器智能研究所（AMII））；Mechanical and Manufacturing Engineering, University of Calgary（卡尔加里大学机械与制造工程系）
Objective assessment of Magnetic Resonance Imaging (MRI) scans of osteoarthritis (OA) can address the limitation of the current OA assessment. Segmentation of bone, cartilage, and joint fluid is necessary for the OA objective assessment. Most of the proposed segmentation methods are not performing instance segmentation and suffer from class imbalance problems. This study deployed Mask R-CNN instance segmentation and improved it (improved-Mask R-CNN (iMaskRCNN)) to obtain a more accurate generalized segmentation for OA-associated tissues. Training and validation of the method were performed usi…
Utilizing 3D Fast Spin Echo Anatomical Imaging to Reduce the Number of Contrast Preparations in $T_{1ρ}$ Quantification of Knee Cartilage Using Learning-Based Methods
University of Copenhagen（哥本哈根大学）；Copenhagen University Hospital, Bispebjerg and Frederiksberg（哥本哈根大学医院比斯佩布杰格和弗雷德里克斯堡院区）
UNet has become the gold standard method for segmenting 2D medical images that any new method must be validated against. However, in recent years, several variations of the seminal UNet have been proposed with promising results. However, there is no clear consensus on the generalisability of these architectures, and UNet currently remains the methodological gold standard. The purpose of this study was to evaluate some of the most promising UNet-inspired architectures for 3D segmentation. For the segmentation of 3D scans, UNet-inspired methods are also dominant, but there is a larger variety ac…
Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), which are state of the art, have limitations owing to the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results. This paper presents a deep learning based automatic segmentation framework for knee tissue segmentation. A novel multipath CNN-based method is proposed, which consists of an encoder decoder-based segmentation network in combinatio…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；University of Helsinki（赫尔辛基大学）；Aalto University（阿尔托大学）
Three-dimensional (3D) semi-quantitative grading of pathological features in articular cartilage (AC) offers significant improvements in basic research of osteoarthritis (OA). We have earlier developed the 3D protocol for imaging of AC and its structures which includes staining of the sample with a contrast agent (phosphotungstic acid, PTA) and a consequent scanning with micro-computed tomography. Such a protocol was designed to provide X-ray attenuation contrast to visualize AC structure. However, at the same time, this protocol has one major disadvantage: the loss of contrast at the tidemark…
Magnetic Resonance Images (MRIs) are extremely used in the medical field to detect and better understand diseases. In order to fasten automatic processing of scans and enhance medical research, this project focuses on automatically segmenting targeted parts of MRIs and generating new MRI datasets from random noise. More specifically, a Deep Neural Network architecture called U-net is used to segment bones and cartilages of Knee MRIs, and several Generative Adversarial Networks (GANs) are compared and tuned to create new realistic and high quality brain MRIs that can be used as training set for…
University of California, San Francisco（加州大学旧金山分校）；University of California, Berkeley（加州大学伯克利分校）
Deep Learning (DL) has shown potential in accelerating Magnetic Resonance Image acquisition and reconstruction. Nevertheless, there is a dearth of tailored methods to guarantee that the reconstruction of small features is achieved with high fidelity. In this work, we employ adversarial attacks to generate small synthetic perturbations, which are difficult to reconstruct for a trained DL reconstruction network. Then, we use robust training to increase the network's sensitivity to these small features and encourage their reconstruction. Next, we investigate the generalization of said approach to…
