Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. Early detection and diagnosis are crucial for successful clinical intervention and management to prevent severe complications, such as loss of mobility. In this paper, we propose an automated approach that employs the Swin Transformer to predict the severity of KOA. Our model uses publicly available radiographic datasets with Kellgren and Lawrence scores to enable early detection and severity assessment. To improve the accuracy of our model, we employ a multi-prediction head architecture that utilizes multi-layer perceptron classifiers. Additionally, we introduce a novel training approach that reduces the data drift between multiple datasets to ensure the generalization ability of the model. The results of our experiments demonstrate the effectiveness and feasibility of our approach in predicting KOA severity accurately.
This paper introduces a new approach to automatically quantify the severity of knee OA using X-ray images. Automatically quantifying knee OA severity involves two steps: first, automatically localizing the knee joints; next, classifying the localized knee joint images. We introduce a new approach to automatically detect the knee joints using a fully convolutional neural network (FCN). We train convolutional neural networks (CNN) from scratch to automatically quantify the knee OA severity optimizing a weighted ratio of two loss functions: categorical cross-entropy and mean-squared loss. This jo…
Insight Centre for Data Analytics, Dublin City University（都柏林城市大学数据洞察中心）；School of Health and Human Performance, Dublin City University（都柏林城市大学健康与人体表现学院）
This paper proposes a new approach to automatically quantify the severity of knee osteoarthritis (OA) from radiographs using deep convolutional neural networks (CNN). Clinically, knee OA severity is assessed using Kellgren \& Lawrence (KL) grades, a five point scale. Previous work on automatically predicting KL grades from radiograph images were based on training shallow classifiers using a variety of hand engineered features. We demonstrate that classification accuracy can be significantly improved using deep convolutional neural network models pre-trained on ImageNet and fine-tuned on knee O…
University of Oulu（奥卢大学）；Tampere University of Technology（坦佩雷理工大学）；Oulu University Hospital（奥卢大学医院）
Knee osteoarthritis (OA) is the most common musculoskeletal disorder. OA diagnosis is currently conducted by assessing symptoms and evaluating plain radiographs, but this process suffers from subjectivity. In this study, we present a new transparent computer-aided diagnosis method based on the Deep Siamese Convolutional Neural Network to automatically score knee OA severity according to the Kellgren-Lawrence grading scale. We trained our method using the data solely from the Multicenter Osteoarthritis Study and validated it on randomly selected 3,000 subjects (5,960 knees) from Osteoarthritis …
VRVis Center for Virtual Reality and Visualization（VRVis虚拟现实与可视化中心）；Braincon Handels-GmbH（Braincon贸易有限公司）；NOVA Medical School, Universidade Nova de Lisboa（新里斯本大学NOVA医学院）；Instituto de Medicina Molecular（分子医学研究所）；Centro Hospitalar Lisboa Ocidental（里斯本西部医疗中心）
The relationship between knee osteoarthritis progression and changes in tibial bone structure has long been recognized and various texture descriptors have been proposed to detect early osteoarthritis (OA) from radiographs. This work aims to investigate (1) femoral textures as an OA indicator and (2) the potential of entropy as a computationally efficient alternative to established texture descriptors. We design a robust semi-automatically placed layout for regions of interest (ROI), compute the Hurst coefficient and the entropy in each ROI, and employ statistical and machine learning methods …
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）
State-of-the-art automated segmentation algorithms are not 100\% accurate especially when segmenting difficult to interpret datasets like those with severe osteoarthritis (OA). We present a novel interactive method called just-enough interaction (JEI), which adds a fast correction step to the automated layered optimal graph segmentation of multiple objects and surfaces (LOGISMOS). After LOGISMOS segmentation in knee MRI, the JEI user interaction does not modify boundary surfaces of the bones and cartilages directly. Local costs of underlying graph nodes are modified instead and the graph is re…
Biomediq A/S（Biomediq公司）；University of Copenhagen（哥本哈根大学）
Scanner drift is a well-known magnetic resonance imaging (MRI) artifact characterized by gradual signal degradation and scan intensity changes over time. In addition, hardware and software updates may imply abrupt changes in signal. The combined effects are particularly challenging for automatic image analysis methods used in longitudinal studies. The implication is increased measurement variation and a risk of bias in the estimations (e.g. in the volume change for a structure). We proposed two quite different approaches for scanner drift normalization and demonstrated the performance for segm…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Erasmus MC, University Medical Center Rotterdam（伊拉斯谟医学中心，鹿特丹大学医学中心）；University Medical Center Rotterdam（鹿特丹大学医学中心）；Tampere University of Technology（坦佩雷理工大学）
Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilizes raw radiographic data, clinical examination results and previous medical history of the patient. We validate…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）
Knee osteoarthritis (OA) is the most common musculoskeletal disease in the world. In primary healthcare, knee OA is diagnosed using clinical examination and radiographic assessment. Osteoarthritis Research Society International (OARSI) atlas of OA radiographic features allows to perform independent assessment of knee osteophytes, joint space narrowing and other knee features. This provides a fine-grained OA severity assessment of the knee, compared to the gold standard and most commonly used Kellgren-Lawrence (KL) composite score. However, both OARSI and KL grading systems suffer from moderate…
H-SemiS: Hierarchical Fusion of Semi and Self-Supervised Learning for Knee Osteoarthritis Severity Grading
Department of Computer Science and Engineering, Indian Institute of Technology (IIT) Indore, Indore 453552, India（印度理工学院Indore计算机科学与工程系）；Department of Computer Science and Engineering, National Institute of Technology Tiruchirappalli, Tiruchirappalli 620015, India（印度理工学院Tiruchirappalli计算机科学与工程系）
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Medical Research Center, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）
Objective: To assess the ability of imaging-based deep learning to predict radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. Design: Knee lateral view radiographs were extracted from The Multicenter Osteoarthritis Study (MOST) (n = 18,436 knees). Patellar region-of-interest (ROI) was first automatically detected, and subsequently, end-to-end deep convolutional neural networks (CNNs) were trained and validated to detect the status of patellofemoral OA. Patellar ROI was detected using deep-learning-based object detection method. Manual PFOA status assessment p…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；Medical Research Center, University of Oulu and Oulu University Hospital（奥卢大学与奥卢大学医院医学研究中心）
Knee osteoarthritis (OA) is very common progressive and degenerative musculoskeletal disease worldwide creates a heavy burden on patients with reduced quality of life and also on society due to financial impact. Therefore, any attempt to reduce the burden of the disease could help both patients and society. In this study, we propose a fully automated novel method, based on combination of joint shape and convolutional neural network (CNN) based bone texture features, to distinguish between the knee radiographs with and without radiographic osteoarthritis. Moreover, we report the first attempt a…
Self-Supervised Learning for Knee Osteoarthritis: Diagnostic Limitations and Prognostic Value of Hospital Data
Department of Physics, J.K. Institute of Science（物理系，J.K. 科学研究所）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；Center for Data Science, New York University（数据科学中心，纽约大学）；Bernard and Irene Schwartz Center for Biomedical Imaging, New York University Langone Health（伯纳德和艾琳·施瓦茨生物医学成像中心，纽约大学朗格尼健康中心）；Department of Radiology, New York University Langone Health（放射学系，纽约大学朗格尼健康中心）；Perlmutter Cancer Center, New York University Langone Health（珀尔马特癌症中心，纽约大学朗格尼健康中心）
Insight Centre for Data Analytics（洞察数据分析中心）；Dublin City University（都柏林城市大学）
This chapter presents the investigations and the results of feature learning using convolutional neural networks to automatically assess knee osteoarthritis (OA) severity and the associated clinical and diagnostic features of knee OA from X-ray images. Also, this chapter demonstrates that feature learning in a supervised manner is more effective than using conventional handcrafted features for automatic detection of knee joints and fine-grained knee OA image classification. In the general machine learning approach to automatically assess knee OA severity, the first step is to localize the regi…
PAII Inc.（PAII 公司）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Knee osteoarthritis (OA) is a common degenerate joint disorder that affects a large population of elderly people worldwide. Accurate radiographic assessment of knee OA severity plays a critical role in chronic patient management. Current clinically-adopted knee OA grading systems are observer subjective and suffer from inter-rater disagreements. In this work, we propose a computer-aided diagnosis approach to provide more accurate and consistent assessments of both composite and fine-grained OA grades simultaneously. A novel semi-supervised learning method is presented to exploit the underlying…
Three-dimensional medical imaging enables detailed understanding of osteoarthritis structural status. However, there remains a vast need for automatic, thus, reader-independent measures that provide reliable assessment of subject-specific clinical outcomes. To this end, we derive a consistent generalization of the recently proposed B-score to Riemannian shape spaces. We further present an algorithmic treatment yielding simple, yet efficient computations allowing for analysis of large shape populations with several thousand samples. Our intrinsic formulation exhibits improved discrimination abi…
Iowa Institute for Biomedical Imaging（爱荷华生物医学影像研究所）；The University of Iowa（爱荷华大学）；University of Pennsylvania（宾夕法尼亚大学）
We present a fully automated learning-based approach for segmenting knee cartilage in the presence of osteoarthritis (OA). The algorithm employs a hierarchical set of two random forest classifiers. The first is a neighborhood approximation forest, the output probability map of which is utilized as a feature set for the second random forest (RF) classifier. The output probabilities of the hierarchical approach are used as cost functions in a Layered Optimal Graph Segmentation of Multiple Objects and Surfaces (LOGISMOS). In this work, we highlight a novel post-processing interaction called just-…
The severity of knee osteoarthritis is graded using the 5-point Kellgren-Lawrence (KL) scale where healthy knees are assigned grade 0, and the subsequent grades 1-4 represent increasing severity of the affliction. Although several methods have been proposed in recent years to develop models that can automatically predict the KL grade from a given radiograph, most models have been developed and evaluated on datasets not sourced from India. These models fail to perform well on the radiographs of Indian patients. In this paper, we propose a novel method using convolutional neural networks to auto…
A Progressive Risk Formulation for Enhanced Deep Learning based Total Knee Replacement Prediction in Knee Osteoarthritis
Center for Data Science, New York University（纽约大学数据科学中心）；Department of Radiology, New York University Langone Health（纽约大学朗格尼健康放射科）；Bernard and Irene Schwartz Center for Biomedical Imaging, New York University Langone Health（纽约大学朗格尼健康伯纳德与艾琳·施瓦茨生物医学影像中心）
Temporal Evolution of Knee Osteoarthritis: A Diffusion-based Morphing Model for X-ray Medical Image Synthesis
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University of Orleans（奥尔良大学）；IDP Institute（IDP研究所）；Jiangsu Institute of Hematology（江苏省血液研究所）；First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Cancer Hospital of Pernambuco（伯南布哥癌症医院）
Unsupervised Domain Adaptation for Automated Knee Osteoarthritis Phenotype Classification
The Chinese University of Hong Kong（香港中文大学）；Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第六人民医院）
From Kellgren-Lawrence to Calcium Pyrophosphate Crystal Deposition: A Soft-Labelling Framework for Knee Osteoarthritis Assessmen
organization= Department of Political Science, Communication；International Relations, University of Macerata , city= Macerata , country= Italy；organization= Department of Economics；Law, University of Macerata , city= Macerata , country= Italy；organization= Department of Innovative Technologies in Medicine \& Dentistry, Università degli Studi "G. D'Annunzio" Chieti - Pescara , city= Chieti , country= Italy；organization= Department of Internal Medicine, Azienda Ospedaliero Universitaria delle Marche , city = Ancona , country= Italy；organization= Academic Rheumatology, University of Nottingham , city = Nottingham , country= UK；organization= Department of Rheumatology, Polytechnic University of Marche , city= Ancona , country= Italy
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；KU Leuven（荷语鲁汶大学）；Aalto University（阿尔托大学）；Ailean Technology Oy（艾兰科技有限公司）
In medical applications, deep learning methods are built to automate diagnostic tasks. However, a clinically relevant question that practitioners usually face, is how to predict the future trajectory of a disease (prognosis). Current methods for such a problem often require domain knowledge, and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many forecasting problem from multimodal data. Inspired by a clinical decision-making process with two agents -- a radiologist and a general practitioner, we model a prognosis prediction problem with two …
Insight Centre for Data Analytics（洞察数据分析中心）；National University of Ireland Galway（爱尔兰国立高威大学）；Dublin City University（都柏林城市大学）；School of Health and Human Performance（健康与人类表现学院）；ZB MED – Information Centre for Life Sciences（ZB MED——生命科学信息中心）；University of Cologne（科隆大学）；School of Mathematics, Statistics and Applied Mathematics（数学、统计与应用数学学院）
Knee osteoarthritis (KOA) is a disease that impairs knee function and causes pain. A radiologist reviews knee X-ray images and grades the severity level of the impairments according to the Kellgren and Lawrence grading scheme; a five-point ordinal scale (0--4). In this study, we used Elastic Net (EN) and Random Forests (RF) to build predictive models using patient assessment data (i.e. signs and symptoms of both knees and medication use) and a convolution neural network (CNN) trained using X-ray images only. Linear mixed effect models (LMM) were used to model the within subject correlation bet…
CT-based Subchondral Bone Microstructural Analysis in Knee Osteoarthritis via MR-Guided Distillation Learning
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第九人民医院）
An AI System for Continuous Knee Osteoarthritis Severity Grading Using Self-Supervised Anomaly Detection with Limited Data
Science Foundation Ireland Centre for Research Training in Machine Learning（科学基金会爱尔兰机器学习研究培训中心）；School of Medicine（医学院）；School of Computer Science（计算机科学学院）；University College Dublin（都柏林大学）；Insight Centre for Data Analytics（数据分析研究所）
Knee Osteoarthritis Severity Prediction using an Attentive Multi-Scale Deep Convolutional Neural Network
Indian Institute of Technology Guwahati（印度古瓦哈蒂印度理工学院）；Cleveland Clinic（克利夫兰诊所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；National University of Singapore（新加坡国立大学）
The University of Iowa（爱荷华大学）；University of Iowa Carver College of Medicine（爱荷华大学卡佛医学院）
A fully automated knee MRI segmentation method to study osteoarthritis (OA) was developed using a novel hierarchical set of random forests (RF) classifiers to learn the appearance of cartilage regions and their boundaries. A neighborhood approximation forest is used first to provide contextual feature to the second-level RF classifier that also considers local features and produces location-specific costs for the layered optimal graph image segmentation of multiple objects and surfaces (LOGISMOS) framework. Double echo steady state (DESS) MRIs used in this work originated from the Osteoarthrit…
Diffusion-based Counterfactual Augmentation: Towards Robust and Interpretable Knee Osteoarthritis Grading
Department of Radiology, Harvard Medical School（哈佛医学院放射科）；Jiangsu Institute of Hematology, The First Affiliated Hospital of Soochow University（苏州大学第一附属医院江苏血液研究所）；L2TI Laboratory, University Sorbonne Paris Nord（巴黎-萨克雷大学L2TI实验室）；department of Medical School, Henan University of Chinese Medicine（河南中医药大学医学院）；Institute for Diagnostic and Interventional Radiology, Faculty of Medicine and University Hospital Cologne, University of Cologne（科隆大学医学院和科隆大学医院诊断与介入放射学研究所）；Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子与电气工程系）；Nuclear Medicine Division, Geneva University Hospital（日内瓦大学医院核医学科）；IDP Institute, UMR CNRS 7013, University of Orleans（奥尔良大学IDP研究所，CNRS UMR 7013）
Adaptive Variance Thresholding: A Novel Approach to Improve Existing Deep Transfer Vision Models and Advance Automatic Knee-Joint Osteoarthritis Classification
University of Jyväskylä（于韦斯屈莱大学）；University of Helsinki（赫尔辛基大学）
Stacked Ensemble of Fine-Tuned CNNs for Knee Osteoarthritis Severity Grading
Indian Institute of Technology Guwahati（印度理工学院瓜瓦提分校）；Indian Institute of Technology Kanpur（印度理工学院坎浦尔分校）；California State University, Northridge（加州州立大学北岭分校）
Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）；Insight Centre for Data Analytics（Insight数据分析中心）；Dublin City University (DCU)（都柏林城市大学）
This work proposes a novel end-to-end convolutional neural network (CNN) architecture to automatically quantify the severity of knee osteoarthritis (OA) using X-Ray images, which incorporates trainable attention modules acting as unsupervised fine-grained detectors of the region of interest (ROI). The proposed attention modules can be applied at different levels and scales across any CNN pipeline helping the network to learn relevant attention patterns over the most informative parts of the image at different resolutions. We test the proposed attention mechanism on existing state-of-the-art CN…
Transformer with Selective Shuffled Position Embedding and Key-Patch Exchange Strategy for Early Detection of Knee Osteoarthritis
IDP Institute（IDP研究所）；UMR CNRS 7013（法国国家科学研究中心UMR 7013）；University of Orleans（奥尔良大学）；PRISME Laboratory（PRISME实验室）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Geneva University Hospital（日内瓦大学医院）
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；KU Leuven（鲁汶大学）；Ailean Technologies Oy
Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In thi…
The University of Queensland（昆士兰大学）；Australian e-Health Research Centre（澳大利亚电子健康研究中心）；Commonwealth Scientific and Industrial Research Organisation（联邦科学与工业研究组织）
In medical image analysis, automated segmentation of multi-component anatomical structures, which often have a spectrum of potential anomalies and pathologies, is a challenging task. In this work, we develop a multi-step approach using U-Net-based neural networks to initially detect anomalies (bone marrow lesions, bone cysts) in the distal femur, proximal tibia and patella from 3D magnetic resonance (MR) images of the knee in individuals with varying grades of osteoarthritis. Subsequently, the extracted data are used for downstream tasks involving semantic segmentation of individual bone and c…
Knee-xRAI: An Explainable AI Framework for Automatic Kellgren-Lawrence Grading of Knee Osteoarthritis
Orthopaedic Department, Faculty of Medicine UIN Syarif Hidayatullah Jakarta（乌姆尼大学医学学院骨科部）；Informatics Engineering, Institut Teknologi Sepuluh Nopember（十月份技术研究所信息工程系）；Information Technology, Universitas Muhammadiyah Yogyakarta（尤科阿卡塔大学信息技术系）；Industrial and Systems Engineering, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学工业与系统工程系）
Confidence-Driven Deep Learning Framework for Early Detection of Knee Osteoarthritis
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University Sorbonne Paris Nord（巴黎北索邦大学）；Jiangsu Institute of Hematology（江苏血液学研究所）；The First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Henan University of Chinese Medicine（河南中医药大学）；German Cancer Research Center（德国癌症研究中心）；Athinoula A. Martinos Centre for Biomedical Imaging（阿西诺拉·A.马蒂诺斯生物医学影像中心）；Geneva University Hospital（日内瓦大学医院）；IDP Institute（IDP研究所）；University of Orleans（奥尔良大学）
Stanford University（斯坦福大学）；University of Central Florida（中佛罗里达大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Minnesota（明尼苏达大学）；University of Copenhagen（哥本哈根大学）；Cleveland Clinic（克利夫兰诊所）；New York University（纽约大学）；Langone Health（朗格尼健康中心）
Purpose: To organize a knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at two timepoints with ground-truth articular (femoral, tibial, patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a hold-out test set. Simil…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
We propose iSegFormer, a memory-efficient transformer that combines a Swin transformer with a lightweight multilayer perceptron (MLP) decoder. With the efficient Swin transformer blocks for hierarchical self-attention and the simple MLP decoder for aggregating both local and global attention, iSegFormer learns powerful representations while achieving high computational efficiencies. Specifically, we apply iSegFormer to interactive 3D medical image segmentation.
Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data and Symptomatic Assessments
University of Oulu（奥卢大学）；Lund University（隆德大学）；Diakonhjemmet Hospital（迪亚科涅米特医院）；Juntendo University（顺天堂大学）；Oulu University Hospital（奥卢大学医院）
