Background: MRI is the modality of choice for cartilage imaging; however, its diagnostic performance is variable and significantly lower than the gold standard diagnostic knee arthroscopy. In recent years, deep learning has been used to automatically interpret medical images to improve diagnostic accuracy and speed. Purpose: The primary purpose of this study was to evaluate whether deep learning applied to the interpretation of knee MRI images can be utilized to identify cartilage defects accurately. Methods: We analyzed data from patients who underwent knee MRI evaluation and consequently had arthroscopic knee surgery (207 with cartilage defect, 90 without cartilage defect). Patients' arthroscopic findings were compared to preoperative MRI images to verify the presence or absence of isolated tibiofemoral cartilage defects. We developed three convolutional neural networks (CNNs) to analyze the MRI images and implemented image-specific saliency maps to visualize the CNNs' decision-making process. To compare the CNNs' performance against human interpretation, the same test dataset images were provided to an experienced orthopaedic surgeon and an orthopaedic resident. Results: Saliency maps demonstrated that the CNNs learned to focus on the clinically relevant areas of the tibiofemoral articular cartilage on MRI images during the decision-making processes. One CNN achieved higher performance than the orthopaedic surgeon, with two more accurate diagnoses made by the CNN. All the CNNs outperformed the orthopaedic resident. Conclusion: CNN can be used to enhance the diagnostic performance of MRI in identifying isolated tibiofemoral cartilage defects and may replace diagnostic knee arthroscopy in certain cases in the future.
New York University School of Medicine（纽约大学医学院）；New York University（纽约大学）
The integrity of articular cartilage is a crucial aspect in the early diagnosis of osteoarthritis (OA). Many novel MRI techniques have the potential to assess compositional changes of the cartilage extracellular matrix. Among these techniques, diffusion tensor imaging (DTI) of cartilage provides a simultaneous assessment of the two principal components of the solid matrix: collagen structure and proteoglycan concentration. DTI, as for any other compositional MRI technique, require a human expert to perform segmentation manually. The manual segmentation is error-prone and time-consuming ($\sim$…
The University of Iowa（爱荷华大学）；University of Iowa Carver College of Medicine（爱荷华大学卡佛医学院）
A fully automated knee MRI segmentation method to study osteoarthritis (OA) was developed using a novel hierarchical set of random forests (RF) classifiers to learn the appearance of cartilage regions and their boundaries. A neighborhood approximation forest is used first to provide contextual feature to the second-level RF classifier that also considers local features and produces location-specific costs for the layered optimal graph image segmentation of multiple objects and surfaces (LOGISMOS) framework. Double echo steady state (DESS) MRIs used in this work originated from the Osteoarthrit…
Reducing Annotation Burden for Femoral Cartilage Segmentation in Knee MRI via Cross-Sequence Transfer Learning
School of Electronic Engineering, Dublin City University（都柏林城市大学电子工程学院）；Bioengineering and Computing Laboratory, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所生物工程与计算实验室）；Dipartimento di Scienze Mediche e Chirurgiche (DIMEC), Alma Mater Studiorum – Università di Bologna（博洛尼亚大学医学与外科科学系（DIMEC））；Diagnostic and Interventional Radiology, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所诊断与介入放射学）；Department of Biomedical and Neuromotor Sciences, University of Bologna（博洛尼亚大学生物医学与神经运动科学系）；nd Orthopedics and Trauma Unit, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所第二骨科与创伤单元）；Independent Researcher（独立研究员）
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）；University of Pennsylvania（宾夕法尼亚大学）
We present a fully automated learning-based approach for segmenting knee cartilage in the presence of osteoarthritis (OA). The algorithm employs a hierarchical set of two random forest classifiers. The first is a neighborhood approximation forest, the output probability map of which is utilized as a feature set for the second random forest (RF) classifier. The output probabilities of the hierarchical approach are used as cost functions in a Layered Optimal Graph Segmentation of Multiple Objects and Surfaces (LOGISMOS). In this work, we highlight a novel post-processing interaction called just-…
Stanford University（斯坦福大学）；University of Central Florida（中佛罗里达大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Minnesota（明尼苏达大学）；University of Copenhagen（哥本哈根大学）；Cleveland Clinic（克利夫兰诊所）；New York University（纽约大学）；Langone Health（朗格尼健康中心）
Purpose: To organize a knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at two timepoints with ground-truth articular (femoral, tibial, patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a hold-out test set. Simil…
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）
State-of-the-art automated segmentation algorithms are not 100\% accurate especially when segmenting difficult to interpret datasets like those with severe osteoarthritis (OA). We present a novel interactive method called just-enough interaction (JEI), which adds a fast correction step to the automated layered optimal graph segmentation of multiple objects and surfaces (LOGISMOS). After LOGISMOS segmentation in knee MRI, the JEI user interaction does not modify boundary surfaces of the bones and cartilages directly. Local costs of underlying graph nodes are modified instead and the graph is re…
University of Copenhagen（哥本哈根大学）；Copenhagen University Hospital, Bispebjerg and Frederiksberg（哥本哈根大学医院比斯佩布杰格和弗雷德里克斯堡院区）
UNet has become the gold standard method for segmenting 2D medical images that any new method must be validated against. However, in recent years, several variations of the seminal UNet have been proposed with promising results. However, there is no clear consensus on the generalisability of these architectures, and UNet currently remains the methodological gold standard. The purpose of this study was to evaluate some of the most promising UNet-inspired architectures for 3D segmentation. For the segmentation of 3D scans, UNet-inspired methods are also dominant, but there is a larger variety ac…
Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), which are state of the art, have limitations owing to the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results. This paper presents a deep learning based automatic segmentation framework for knee tissue segmentation. A novel multipath CNN-based method is proposed, which consists of an encoder decoder-based segmentation network in combinatio…
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属同仁医院影像科）；School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）
Knee osteoarthritis (OA) is the most common osteoarthritis and a leading cause of disability. Cartilage defects are regarded as major manifestations of knee OA, which are visible by magnetic resonance imaging (MRI). Thus early detection and assessment for knee cartilage defects are important for protecting patients from knee OA. In this way, many attempts have been made on knee cartilage defect assessment by applying convolutional neural networks (CNNs) to knee MRI. However, the physiologic characteristics of the cartilage may hinder such efforts: the cartilage is a thin curved layer, implying…
Clinical utility of foundation models in musculoskeletal MRI for biomarker fidelity and predictive outcomes
Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco（加利福尼亚大学旧金山分校放射与生物医学影像系智能成像中心）；Department of Bioengineering, University of California, Berkeley（加利福尼亚大学伯克利分校生物工程系）；Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco（加利福尼亚大学旧金山分校生物工程与治疗科学系）；Department of Medical Sciences and Technology, Indian Institute of Technology, Madras（印度理工学院马德拉斯分校医学科学与技术系）；Bay Area Institute of Computation, Altos Labs（Altos Labs湾区计算研究所）
Schulich School of Engineering, University of Calgary（卡尔加里大学舒立克工程学院）；Department of Radiology & Diagnostic Imaging, University of Alberta（阿尔伯塔大学放射与诊断成像系）；Alberta Machine Intelligence Institute (AMII), University of Alberta（阿尔伯塔大学阿尔伯塔机器智能研究所（AMII））；Mechanical and Manufacturing Engineering, University of Calgary（卡尔加里大学机械与制造工程系）
Objective assessment of Magnetic Resonance Imaging (MRI) scans of osteoarthritis (OA) can address the limitation of the current OA assessment. Segmentation of bone, cartilage, and joint fluid is necessary for the OA objective assessment. Most of the proposed segmentation methods are not performing instance segmentation and suffer from class imbalance problems. This study deployed Mask R-CNN instance segmentation and improved it (improved-Mask R-CNN (iMaskRCNN)) to obtain a more accurate generalized segmentation for OA-associated tissues. Training and validation of the method were performed usi…
Objective: We evaluate a fully-automated femoral cartilage segmentation model for measuring T2 relaxation values and longitudinal changes using multi-echo spin echo (MESE) MRI. We have open sourced this model and corresponding segmentations. Methods: We trained a neural network to segment femoral cartilage from MESE MRIs. Cartilage was divided into 12 subregions along medial-lateral, superficial-deep, and anterior-central-posterior boundaries. Subregional T2 values and four-year changes were calculated using a musculoskeletal radiologist's segmentations (Reader 1) and the model's segmentations…
ITMO University（圣彼得堡国立信息技术机械与光学大学）；Federal Almazov North-West Medical Research Center（阿尔马佐夫西北联邦医学研究中心）；Aix-Marseille Universite（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）
Detection of cartilage loss is crucial for the diagnosis of osteo- and rheumatoid arthritis. A large number of automatic segmentation tools have been reported so far for cartilage assessment in magnetic resonance images of large joints. As compared to knee or hip, wrist cartilage has a more complex structure so that automatic tools developed for large joints are not expected to be operational for wrist cartilage segmentation. In that respect, a fully automatic wrist cartilage segmentation method would be of high clinical interest. We assessed the performance of four optimized variants of the U…
Obtaining magnetic resonance images (MRI) with high resolution and generating quantitative image-based biomarkers for assessing tissue biochemistry is crucial in clinical and research applications. How- ever, acquiring quantitative biomarkers requires high signal-to-noise ratio (SNR), which is at odds with high-resolution in MRI, especially in a single rapid sequence. In this paper, we demonstrate how super-resolution can be utilized to maintain adequate SNR for accurate quantification of the T2 relaxation time biomarker, while simultaneously generating high- resolution images. We compare the …
High-fidelity semantic segmentation of magnetic resonance volumes is critical for estimating tissue morphometry and relaxation parameters in both clinical and research applications. While manual segmentation is accepted as the gold-standard, recent advances in deep learning and convolutional neural networks (CNNs) have shown promise for efficient automatic segmentation of soft tissues. However, due to the stochastic nature of deep learning and the multitude of hyperparameters in training networks, predicting network behavior is challenging. In this paper, we quantify the impact of three factor…
Biomediq A/S（Biomediq公司）；University of Copenhagen（哥本哈根大学）
Scanner drift is a well-known magnetic resonance imaging (MRI) artifact characterized by gradual signal degradation and scan intensity changes over time. In addition, hardware and software updates may imply abrupt changes in signal. The combined effects are particularly challenging for automatic image analysis methods used in longitudinal studies. The implication is increased measurement variation and a risk of bias in the estimations (e.g. in the volume change for a structure). We proposed two quite different approaches for scanner drift normalization and demonstrated the performance for segm…
LM-CartSeg: Automated Segmentation of Lateral and Medial Cartilage and Subchondral Bone for Radiomics Analysis
Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China（香港理工大学康复科学系）；Department of Physical Therapy and Rehabilitation Science, University of Maryland, Baltimore, Maryland, USA（马里兰大学巴尔的摩分校物理治疗与康复科学系）
University of California, San Francisco（加州大学旧金山分校）；University of California, Berkeley（加州大学伯克利分校）
Deep Learning (DL) has shown potential in accelerating Magnetic Resonance Image acquisition and reconstruction. Nevertheless, there is a dearth of tailored methods to guarantee that the reconstruction of small features is achieved with high fidelity. In this work, we employ adversarial attacks to generate small synthetic perturbations, which are difficult to reconstruct for a trained DL reconstruction network. Then, we use robust training to increase the network's sensitivity to these small features and encourage their reconstruction. Next, we investigate the generalization of said approach to…
Integration of Swin UNETR and statistical shape modeling for a semi-automated segmentation of the knee and biomechanical modeling of articular cartilage
University of Calgary（卡尔加里大学）；University of Alberta（阿尔伯塔大学）
Deep Learning Superresolution for 7T Knee MR Imaging: Impact on Image Quality and Diagnostic Performance
Southwest Hospital, Army Medical University (Third Military Medical University)（陆军军医大学（第三军医大学）西南医院）；Siemens Healthineers Ltd.（西门子医疗有限公司）；RadioDynamic Healthcare（RadioDynamic医疗）；Tsinghua Cross-Strait Research Institute, Tsinghua University（清华大学清华两岸研究院）
