Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Erasmus MC, University Medical Center Rotterdam（伊拉斯谟医学中心，鹿特丹大学医学中心）；University Medical Center Rotterdam（鹿特丹大学医学中心）；Tampere University of Technology（坦佩雷理工大学）
Aleksei Tiulpin, Stefan Klein, Sita M. A. Bierma-Zeinstra, Jérôme Thevenot, Esa Rahtu, Joyce van Meurs, Edwin H. G. Oei, Simo Saarakkala
Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilizes raw radiographic data, clinical examination results and previous medical history of the patient. We validated this approach on an independent test set of 3,918 knee images from 2,129 subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78-0.81) and Average Precision (AP) of 0.68 (0.66-0.70). In contrast, a reference approach, based on logistic regression, yielded AUC of 0.75 (0.74-0.77) and AP of 0.62 (0.60-0.64). The proposed method could significantly improve the subject selection process for OA drug-development trials and help the development of personalized therapeutic plans.
Insight Centre for Data Analytics, Dublin City University（都柏林城市大学数据洞察中心）；School of Health and Human Performance, Dublin City University（都柏林城市大学健康与人体表现学院）
This paper proposes a new approach to automatically quantify the severity of knee osteoarthritis (OA) from radiographs using deep convolutional neural networks (CNN). Clinically, knee OA severity is assessed using Kellgren \& Lawrence (KL) grades, a five point scale. Previous work on automatically predicting KL grades from radiograph images were based on training shallow classifiers using a variety of hand engineered features. We demonstrate that classification accuracy can be significantly improved using deep convolutional neural network models pre-trained on ImageNet and fine-tuned on knee O…
University of Oulu（奥卢大学）；Tampere University of Technology（坦佩雷理工大学）；Oulu University Hospital（奥卢大学医院）
Knee osteoarthritis (OA) is the most common musculoskeletal disorder. OA diagnosis is currently conducted by assessing symptoms and evaluating plain radiographs, but this process suffers from subjectivity. In this study, we present a new transparent computer-aided diagnosis method based on the Deep Siamese Convolutional Neural Network to automatically score knee OA severity according to the Kellgren-Lawrence grading scale. We trained our method using the data solely from the Multicenter Osteoarthritis Study and validated it on randomly selected 3,000 subjects (5,960 knees) from Osteoarthritis …
VRVis Center for Virtual Reality and Visualization（VRVis虚拟现实与可视化中心）；Braincon Handels-GmbH（Braincon贸易有限公司）；NOVA Medical School, Universidade Nova de Lisboa（新里斯本大学NOVA医学院）；Instituto de Medicina Molecular（分子医学研究所）；Centro Hospitalar Lisboa Ocidental（里斯本西部医疗中心）
The relationship between knee osteoarthritis progression and changes in tibial bone structure has long been recognized and various texture descriptors have been proposed to detect early osteoarthritis (OA) from radiographs. This work aims to investigate (1) femoral textures as an OA indicator and (2) the potential of entropy as a computationally efficient alternative to established texture descriptors. We design a robust semi-automatically placed layout for regions of interest (ROI), compute the Hurst coefficient and the entropy in each ROI, and employ statistical and machine learning methods …
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）
State-of-the-art automated segmentation algorithms are not 100\% accurate especially when segmenting difficult to interpret datasets like those with severe osteoarthritis (OA). We present a novel interactive method called just-enough interaction (JEI), which adds a fast correction step to the automated layered optimal graph segmentation of multiple objects and surfaces (LOGISMOS). After LOGISMOS segmentation in knee MRI, the JEI user interaction does not modify boundary surfaces of the bones and cartilages directly. Local costs of underlying graph nodes are modified instead and the graph is re…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Medical Research Center, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）
Knee osteoarthritis (OA) is very common progressive and degenerative musculoskeletal disease worldwide creates a heavy burden on patients with reduced quality of life and also on society due to financial impact. Therefore, any attempt to reduce the burden of the disease could help both patients and society. In this study, we propose a fully automated novel method, based on combination of joint shape and convolutional neural network (CNN) based bone texture features, to distinguish between the knee radiographs with and without radiographic osteoarthritis. Moreover, we report the first attempt a…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）
Osteoarthritis (OA) is a common musculoskeletal condition typically diagnosed from radiographic assessment after clinical examination. However, a visual evaluation made by a practitioner suffers from subjectivity and is highly dependent on the experience. Computer-aided diagnostics (CAD) could improve the objectivity of knee radiographic examination. The first essential step of knee OA CAD is to automatically localize the joint area. However, according to the literature this task itself remains challenging. The aim of this study was to develop novel and computationally efficient method to tack…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）
Knee osteoarthritis (OA) is the most common musculoskeletal disease in the world. In primary healthcare, knee OA is diagnosed using clinical examination and radiographic assessment. Osteoarthritis Research Society International (OARSI) atlas of OA radiographic features allows to perform independent assessment of knee osteophytes, joint space narrowing and other knee features. This provides a fine-grained OA severity assessment of the knee, compared to the gold standard and most commonly used Kellgren-Lawrence (KL) composite score. However, both OARSI and KL grading systems suffer from moderate…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Medical Research Center, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）
Objective: To assess the ability of imaging-based deep learning to predict radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. Design: Knee lateral view radiographs were extracted from The Multicenter Osteoarthritis Study (MOST) (n = 18,436 knees). Patellar region-of-interest (ROI) was first automatically detected, and subsequently, end-to-end deep convolutional neural networks (CNNs) were trained and validated to detect the status of patellofemoral OA. Patellar ROI was detected using deep-learning-based object detection method. Manual PFOA status assessment p…
H-SemiS: Hierarchical Fusion of Semi and Self-Supervised Learning for Knee Osteoarthritis Severity Grading
Department of Computer Science and Engineering, Indian Institute of Technology (IIT) Indore, Indore 453552, India（印度理工学院Indore计算机科学与工程系）；Department of Computer Science and Engineering, National Institute of Technology Tiruchirappalli, Tiruchirappalli 620015, India（印度理工学院Tiruchirappalli计算机科学与工程系）
Self-Supervised Learning for Knee Osteoarthritis: Diagnostic Limitations and Prognostic Value of Hospital Data
Department of Physics, J.K. Institute of Science（物理系，J.K. 科学研究所）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；Center for Data Science, New York University（数据科学中心，纽约大学）；Bernard and Irene Schwartz Center for Biomedical Imaging, New York University Langone Health（伯纳德和艾琳·施瓦茨生物医学成像中心，纽约大学朗格尼健康中心）；Department of Radiology, New York University Langone Health（放射学系，纽约大学朗格尼健康中心）；Perlmutter Cancer Center, New York University Langone Health（珀尔马特癌症中心，纽约大学朗格尼健康中心）
Insight Centre for Data Analytics（洞察数据分析中心）；Dublin City University（都柏林城市大学）
This chapter presents the investigations and the results of feature learning using convolutional neural networks to automatically assess knee osteoarthritis (OA) severity and the associated clinical and diagnostic features of knee OA from X-ray images. Also, this chapter demonstrates that feature learning in a supervised manner is more effective than using conventional handcrafted features for automatic detection of knee joints and fine-grained knee OA image classification. In the general machine learning approach to automatically assess knee OA severity, the first step is to localize the regi…
Three-dimensional medical imaging enables detailed understanding of osteoarthritis structural status. However, there remains a vast need for automatic, thus, reader-independent measures that provide reliable assessment of subject-specific clinical outcomes. To this end, we derive a consistent generalization of the recently proposed B-score to Riemannian shape spaces. We further present an algorithmic treatment yielding simple, yet efficient computations allowing for analysis of large shape populations with several thousand samples. Our intrinsic formulation exhibits improved discrimination abi…
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）；University of Pennsylvania（宾夕法尼亚大学）
We present a fully automated learning-based approach for segmenting knee cartilage in the presence of osteoarthritis (OA). The algorithm employs a hierarchical set of two random forest classifiers. The first is a neighborhood approximation forest, the output probability map of which is utilized as a feature set for the second random forest (RF) classifier. The output probabilities of the hierarchical approach are used as cost functions in a Layered Optimal Graph Segmentation of Multiple Objects and Surfaces (LOGISMOS). In this work, we highlight a novel post-processing interaction called just-…
The severity of knee osteoarthritis is graded using the 5-point Kellgren-Lawrence (KL) scale where healthy knees are assigned grade 0, and the subsequent grades 1-4 represent increasing severity of the affliction. Although several methods have been proposed in recent years to develop models that can automatically predict the KL grade from a given radiograph, most models have been developed and evaluated on datasets not sourced from India. These models fail to perform well on the radiographs of Indian patients. In this paper, we propose a novel method using convolutional neural networks to auto…
A Progressive Risk Formulation for Enhanced Deep Learning based Total Knee Replacement Prediction in Knee Osteoarthritis
Center for Data Science, New York University（纽约大学数据科学中心）；Department of Radiology, New York University Langone Health（纽约大学朗格尼健康放射科）；Bernard and Irene Schwartz Center for Biomedical Imaging, New York University Langone Health（纽约大学朗格尼健康伯纳德与艾琳·施瓦茨生物医学影像中心）
From Kellgren-Lawrence to Calcium Pyrophosphate Crystal Deposition: A Soft-Labelling Framework for Knee Osteoarthritis Assessmen
organization= Department of Political Science, Communication；International Relations, University of Macerata , city= Macerata , country= Italy；organization= Department of Economics；Law, University of Macerata , city= Macerata , country= Italy；organization= Department of Innovative Technologies in Medicine \& Dentistry, Università degli Studi "G. D'Annunzio" Chieti - Pescara , city= Chieti , country= Italy；organization= Department of Internal Medicine, Azienda Ospedaliero Universitaria delle Marche , city = Ancona , country= Italy；organization= Academic Rheumatology, University of Nottingham , city = Nottingham , country= UK；organization= Department of Rheumatology, Polytechnic University of Marche , city= Ancona , country= Italy
Mayo Clinic（梅奥诊所）；Case Western Reserve University（凯斯西储大学）；University of Southern Maine（南缅因大学）；Dubyak Center for Digital Science and Innovation, University of Southern Maine（南缅因大学Dubyak数字科学与创新中心）；Department of Artificial Intelligence and Informatics, Mayo Clinic（梅奥诊所人工智能与信息学系）；Department of Health Sciences Research, Mayo Clinic（梅奥诊所健康科学研究系）；Department of Computer and Data Sciences, Case Western Reserve University（凯斯西储大学计算机与数据科学系）；Department of Orthopedics, Mayo Clinic（梅奥诊所骨科系）；Department of Computer Science, University of Southern Maine（南缅因大学计算机科学系）
Knee pain is undoubtedly the most common musculoskeletal symptom that impairs quality of life, confines mobility and functionality across all ages. Knee pain is clinically evaluated by routine radiographs, where the widespread adoption of radiographic images and their availability at low cost, make them the principle component in the assessment of knee pain and knee pathologies, such as arthritis, trauma, and sport injuries. However, interpretation of the knee radiographs is still highly subjective, and overlapping structures within the radiographs and the large volume of images needing to be …
Knee Osteoarthritis Severity Prediction using an Attentive Multi-Scale Deep Convolutional Neural Network
Indian Institute of Technology Guwahati（印度古瓦哈蒂印度理工学院）；Cleveland Clinic（克利夫兰诊所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；National University of Singapore（新加坡国立大学）
The University of Iowa（爱荷华大学）；University of Iowa Carver College of Medicine（爱荷华大学卡佛医学院）
A fully automated knee MRI segmentation method to study osteoarthritis (OA) was developed using a novel hierarchical set of random forests (RF) classifiers to learn the appearance of cartilage regions and their boundaries. A neighborhood approximation forest is used first to provide contextual feature to the second-level RF classifier that also considers local features and produces location-specific costs for the layered optimal graph image segmentation of multiple objects and surfaces (LOGISMOS) framework. Double echo steady state (DESS) MRIs used in this work originated from the Osteoarthrit…
Opportunistic Bone-Loss Screening from Routine Knee Radiographs Using a Multi-Task Deep Learning Framework with Sensitivity-Constrained Threshold Optimization
ORBIT Lab, College of Medicine and Biological Information Engineering, Northeastern University（ORBIT实验室，医学院和生物信息工程学院，东北大学）；Department of Radiology, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University（放射科，辽宁省医学影像重点实验室，辽宁省影像技术与人工智能重点实验室，中国医科大学盛京医院）；Rehabilitation Center, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University（康复中心，辽宁省医学影像重点实验室，辽宁省影像技术与人工智能重点实验室，中国医科大学盛京医院）；The University of Sydney, Sydney Musculoskeletal Health and the Kolling Institute, Northern Clinical School, Faculty of Medicine and Health and the Northern Sydney Local Health District（悉尼大学，悉尼骨科健康与Kolling研究所，北方临床医学院，医学院与健康学院，北悉尼地方卫生区）
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；KU Leuven（鲁汶大学）；Ailean Technologies Oy
Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In thi…
Knee-xRAI: An Explainable AI Framework for Automatic Kellgren-Lawrence Grading of Knee Osteoarthritis
Orthopaedic Department, Faculty of Medicine UIN Syarif Hidayatullah Jakarta（乌姆尼大学医学学院骨科部）；Informatics Engineering, Institut Teknologi Sepuluh Nopember（十月份技术研究所信息工程系）；Information Technology, Universitas Muhammadiyah Yogyakarta（尤科阿卡塔大学信息技术系）；Industrial and Systems Engineering, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学工业与系统工程系）
Deep Learning-based Alignment Measurement in Knee Radiographs
Division of Informatics, Imaging and Data Sciences, The University of Manchester（信息学与数据科学系，曼彻斯特大学）；Northern Care Alliance NHS Foundation Trust（北护理联盟国家健康服务信托基金）；Department of Trauma and Orthopaedics, Stockport NHS Foundation Trust（创伤与骨科部门，史托波特国家健康服务信托基金）；School of Health and Society, University of Salford（健康与社会学院，萨尔福德大学）；School of Biological Sciences, The University of Manchester（生物科学学院，曼彻斯特大学）；Research Unit of Health Sciences and Technology, University of Oulu（健康科学与技术研究单位，奥卢大学）
Stanford University（斯坦福大学）；University of Central Florida（中佛罗里达大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Minnesota（明尼苏达大学）；University of Copenhagen（哥本哈根大学）；Cleveland Clinic（克利夫兰诊所）；New York University（纽约大学）；Langone Health（朗格尼健康中心）
Purpose: To organize a knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at two timepoints with ground-truth articular (femoral, tibial, patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a hold-out test set. Simil…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Erasmus MC, University Medical Center（伊拉斯谟医学中心大学医院）；Medical Research Center, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）
The purposes of this study were to investigate: 1) the effect of placement of region-of-interest (ROI) for texture analysis of subchondral bone in knee radiographs, and 2) the ability of several texture descriptors to distinguish between the knees with and without radiographic osteoarthritis (OA). Bilateral posterior-anterior knee radiographs were analyzed from the baseline of OAI and MOST datasets. A fully automatic method to locate the most informative region from subchondral bone using adaptive segmentation was developed. We used an oversegmentation strategy for partitioning knee images int…
Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data and Symptomatic Assessments
University of Oulu（奥卢大学）；Lund University（隆德大学）；Diakonhjemmet Hospital（迪亚科涅米特医院）；Juntendo University（顺天堂大学）；Oulu University Hospital（奥卢大学医院）
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Medical Research Center（医学研究中心）
Objective is to assess the ability of texture features for detecting radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. We used lateral view knee radiographs from MOST public use datasets (n = 5507 knees). Patellar region-of-interest (ROI) was automatically detected using landmark detection tool (BoneFinder). Hand-crafted features, based on LocalBinary Patterns (LBP), were then extracted to describe the patellar texture. First, a machine learning model (Gradient Boosting Machine) was trained to detect radiographic PFOA from the LBP features. Furthermore, we u…
Duke University（杜克大学）；Duke Clinical Research Institute（杜克临床研究所）；Duke-Margolis Center for Health Policy（杜克-马戈利斯卫生政策中心）；Duke Cancer Institute（杜克癌症研究所）
The assessment of knee osteoarthritis (KOA) severity on knee X-rays is a central criteria for the use of total knee arthroplasty. However, this assessment suffers from imprecise standards and a remarkably high inter-reader variability. An algorithmic, automated assessment of KOA severity could improve overall outcomes of knee replacement procedures by increasing the appropriateness of its use. We propose a novel deep learning-based five-step algorithm to automatically grade KOA from posterior-anterior (PA) views of radiographs: (1) image preprocessing (2) localization of knees joints in the im…
The postoperative range of motion is one of the crucial factors indicating the outcome of Total Knee Arthroplasty (TKA). Although the correlation between range of knee flexion and posterior condylar offset (PCO) is controversial in the literature, PCO maintains its importance on evaluation of TKA. Due to limitations on PCO measurement, two novel parameters, posterior condylar offset ratio (PCOR) and anterior condylar offset ratio (ACOR), were introduced. Nowadays, the calculation of PCOR and ACOR on plain lateral radiographs is done manually by orthopedic surgeons. In this regard, we developed…
Compal Electronics（仁宝电脑）；Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）
Purpose Automated detection of region of interest (ROI) is a critical step for many medical image applications such as heart ROIs detection in perfusion MRI images, lung boundary detection in chest X-rays, and femoral head detection in pelvic radiographs. Thus, we proposed a practical framework of ROIs detection in medical images, with a case study for hip detection in anteroposterior (AP) pelvic radiographs. Materials and Methods: We conducted a retrospective study which analyzed hip joints seen on 7,399 AP pelvic radiographs from three diverse sources, including 4,290 high resolution radiogr…
This paper introduces a new approach to automatically quantify the severity of knee OA using X-ray images. Automatically quantifying knee OA severity involves two steps: first, automatically localizing the knee joints; next, classifying the localized knee joint images. We introduce a new approach to automatically detect the knee joints using a fully convolutional neural network (FCN). We train convolutional neural networks (CNN) from scratch to automatically quantify the knee OA severity optimizing a weighted ratio of two loss functions: categorical cross-entropy and mean-squared loss. This jo…
Landmark-free Assessment of Lower-limb Alignment with Implicit Neural Shape Functions from Knee Radiographs
Division of Informatics, Imaging and Data Sciences, The University of Manchester（曼彻斯特大学信息学、影像与数据科学部）；Research Unit of Health Sciences and Technology, University of Oulu（奥卢大学健康科学与技术研究部）；Medical Research Center Oulu, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）；Department of Trauma and Orthopaedics, Stockport NHS Foundation Trust, Stepping Hill Hospital（斯泰平希尔医院斯托克波特NHS基金会创伤与骨科）；School of Health and Society, University of Salford（索尔福德大学健康与社会学院）；School of Biological Sciences, The University of Manchester（曼彻斯特大学生物科学学院）；Weill Cornell Medicine, Cornell University（康奈尔大学威尔康奈尔医学院）
Feasibility Study of a Diffusion-Based Model for Cross-Modal Generation of Knee MRI from X-ray: Integrating Radiographic Feature Information
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University Sorbonne Paris Nord（巴黎北大索邦大学）；German Cancer Research Center（德国癌症研究中心）；Jiangsu Institute of Hematology（江苏血液学研究所）；The First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Henan University of Chinese Medicine（河南中医药大学）；Athinoula A. Martinos Centre for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；IDP Institute（IDP研究所）；University of Orleans（奥尔良大学）
A generalizable large-scale foundation model for musculoskeletal radiographs
Seoul National University Hospital（首尔大学医院）；Seoul National University（首尔大学）；Seoul National University College of Medicine（首尔大学医学院）；Seoul National University Bundang Hospital（首尔大学盆唐医院）
Biomediq A/S（Biomediq公司）；University of Copenhagen（哥本哈根大学）
Scanner drift is a well-known magnetic resonance imaging (MRI) artifact characterized by gradual signal degradation and scan intensity changes over time. In addition, hardware and software updates may imply abrupt changes in signal. The combined effects are particularly challenging for automatic image analysis methods used in longitudinal studies. The implication is increased measurement variation and a risk of bias in the estimations (e.g. in the volume change for a structure). We proposed two quite different approaches for scanner drift normalization and demonstrated the performance for segm…
Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), which are state of the art, have limitations owing to the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results. This paper presents a deep learning based automatic segmentation framework for knee tissue segmentation. A novel multipath CNN-based method is proposed, which consists of an encoder decoder-based segmentation network in combinatio…
