The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging(MRI), and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machinelearning, specifically deep learning methods, can be used for rapidand accurate image analysis of MRI scans, an unmet clinicalneed in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi-view imaging when classifying the abnormalities. Promising results indicated the potential use of multi-view deep learning based classification of MSK abnormalities in routine clinical assessment.
Virginia Polytechnic Institute and State University (Virginia Tech)（弗吉尼亚理工大学（Virginia Tech））；Department of Mechanical Engineering（机械工程系）；Autonomous Systems and Intelligent Machines (ASIM) Lab（自主系统与智能机器（ASIM）实验室）；Graduate Research Assistant（研究生研究助理）
This paper introduces MuRAD (Musculoskeletal Radiograph Abnormality Detection tool), a tool that can help radiologists automate the detection of abnormalities in musculoskeletal radiographs (bone X-rays). MuRAD utilizes a Convolutional Neural Network (CNN) that can accurately predict whether a bone X-ray is abnormal, and leverages Class Activation Map (CAM) to localize the abnormality in the image. MuRAD achieves an F1 score of 0.822 and a Cohen's kappa of 0.699, which is comparable to the performance of expert radiologists.
CSG: A Context-Semantic Guided Diffusion Approach in De Novo Musculoskeletal Ultrasound Image Generation
GE Healthcare（GE医疗）；Science & Technology Organization, GE Healthcare（GE医疗科技组织）；Point of Care Ultrasound & Handheld（床旁超声与 handheld（注：此处保留原文表述，结合上下文为GE医疗相关临床应用部门，按要求提取到机构级别））
Ultrasound imaging provides information from a large part of the muscle. It has recently been shown that ultrafast ultrasound imaging can be used to record and analyze the mechanical response of individual MUs using blind source separation. In this work, we present an alternative method - a deep learning pipeline - to identify active MUs in ultrasound image sequences, including segmentation of their territories and signal estimation of their mechanical responses (twitch train). We train and evaluate the model using simulated data mimicking the complex activation pattern of tens of activated MU…
Validation of musculoskeletal segmentation model with uncertainty estimation for bone and muscle assessment in hip-to-knee clinical CT images
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；Osaka University（大阪大学）；Keio University School of Medicine（庆应义塾大学医学部）；Hitachi Ltd.（日立制作所）；Ehime University（爱媛大学）
IMT Atlantique（IMT大西洋）；LaTIM UMR 1101, Inserm（LaTIM UMR 1101，法国国家健康与医学研究院）；Symbiosis Centre for Medical Image Analysis, Symbiosis International University（共生国际大学共生医学图像分析中心）；Fondation ILDYS（ILDYS基金会）
Automatic segmentation of the musculoskeletal system in pediatric magnetic resonance (MR) images is a challenging but crucial task for morphological evaluation in clinical practice. We propose a deep learning-based regularized segmentation method for multi-structure bone delineation in MR images, designed to overcome the inherent scarcity and heterogeneity of pediatric data. Based on a newly devised shape code discriminator, our adversarial regularization scheme enforces the deep network to follow a learnt shape representation of the anatomy. The novel shape priors based adversarial regulariza…
University of Jyväskylä（于韦斯屈莱大学）；University of Gloucestershire（格洛斯特郡大学）；Norwegian School of Sport Sciences（挪威体育科学学院）
B-mode ultrasound is commonly used to image musculoskeletal tissues, but one major bottleneck is data interpretation, and analyses of muscle thickness, pennation angle and fascicle length are often still performed manually. In this study we trained deep neural networks (based on U-net) to detect muscle fascicles and aponeuroses using a set of labelled musculoskeletal ultrasound images. We then compared neural network predictions on new, unseen images to those obtained via manual analysis and two existing semi/automated analysis approaches (SMA and Ultratrack). With a GPU, inference time for a …
IMT Atlantique（大西洋国立高等矿业电信学院）；LaTIM UMR 1101, Inserm（法国国家健康与医学研究院LaTIM UMR 1101实验室）；Symbiosis International University（共生国际大学）；Centre Hospitalier Régional et Universitaire (CHRU) de Brest（布雷斯特大学区域中心医院）
Morphological and diagnostic evaluation of pediatric musculoskeletal system is crucial in clinical practice. However, most segmentation models do not perform well on scarce pediatric imaging data. We propose a new pre-trained regularized convolutional encoder-decoder network for the challenging task of segmenting heterogeneous pediatric magnetic resonance (MR) images. To this end, we have conceived a novel optimization scheme for the segmentation network which comprises additional regularization terms to the loss function. In order to obtain globally consistent predictions, we incorporate a sh…
SCOPE-MRI: Bankart Lesion Detection as a Case Study in Data Curation and Deep Learning for Challenging Diagnoses
Pritzker School of Medicine, University of Chicago（芝加哥大学普ritzker医学学院）；University of Chicago（芝加哥大学）；Department of Computer Science, University of Chicago（芝加哥大学计算机科学系）；Department of Orthopaedic Surgery & Rehabilitation Medicine, UChicago Medicine（芝加哥医学中心骨科与康复医学系）
IMT Atlantique（大西洋国立高等矿业电信学校）；LaTIM UMR 1101, Inserm（法国国家健康与医学研究院LaTIM UMR 1101实验室）；Centre Hospitalier Régional et Universitaire (CHRU) de Brest（布雷斯特大学区域中心医院）；Fondation ILDYS（ILDYS基金会）；Symbiosis Center for Medical Image Analysis, Symbiosis International University（共生国际大学共生医学图像分析中心）
Clinical diagnosis of the pediatric musculoskeletal system relies on the analysis of medical imaging examinations. In the medical image processing pipeline, semantic segmentation using deep learning algorithms enables an automatic generation of patient-specific three-dimensional anatomical models which are crucial for morphological evaluation. However, the scarcity of pediatric imaging resources may result in reduced accuracy and generalization performance of individual deep segmentation models. In this study, we propose to design a novel multi-task, multi-domain learning framework in which a …
Monitoring awkward postures is a proactive prevention for Musculoskeletal Disorders (MSDs)in construction. Machine Learning (ML) models have shown promising results for posture recognition from Wearable Sensors. However, further investigations are needed concerning: i) Incremental Learning (IL), where trained models adapt to learn new postures and control the forgetting of learned postures; ii) MSDs assessment with recognized postures. This study proposed an incremental Convolutional Long Short-Term Memory (CLN) model, investigated effective IL strategies, and evaluated MSDs assessment using r…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）；KU Leuven（鲁汶大学）；Ailean Technologies Oy
Knee osteoarthritis (OA) is one of the highest disability factors in the world. This musculoskeletal disorder is assessed from clinical symptoms, and typically confirmed via radiographic assessment. This visual assessment done by a radiologist requires experience, and suffers from moderate to high inter-observer variability. The recent literature has shown that deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. However, these methods require large amounts of labeled data, which are costly to obtain. In thi…
University of Oulu（奥卢大学）；Oulu University Hospital（奥卢大学医院）
Knee osteoarthritis (OA) is the most common musculoskeletal disease in the world. In primary healthcare, knee OA is diagnosed using clinical examination and radiographic assessment. Osteoarthritis Research Society International (OARSI) atlas of OA radiographic features allows to perform independent assessment of knee osteophytes, joint space narrowing and other knee features. This provides a fine-grained OA severity assessment of the knee, compared to the gold standard and most commonly used Kellgren-Lawrence (KL) composite score. However, both OARSI and KL grading systems suffer from moderate…
Fractures occur in the shoulder area, which has a wider range of motion than other joints in the body, for various reasons. To diagnose these fractures, data gathered from Xradiation (X-ray), magnetic resonance imaging (MRI), or computed tomography (CT) are used. This study aims to help physicians by classifying shoulder images taken from X-ray devices as fracture / non-fracture with artificial intelligence. For this purpose, the performances of 26 deep learning-based pretrained models in the detection of shoulder fractures were evaluated on the musculoskeletal radiographs (MURA) dataset, and …
Texas A&M University（德州农工大学）；Georgia Institute of Technology（佐治亚理工学院）
Advancements in deep generative models such as generative adversarial networks and variational autoencoders have resulted in the ability to generate realistic images that are visually indistinguishable from real images, which raises concerns about their potential malicious usage. In this paper, we present an analysis of the high-frequency Fourier modes of real and deep network generated images and show that deep network generated images share an observable, systematic shortcoming in replicating the attributes of these high-frequency modes. Using this, we propose a detection method based on the…
University of California, San Francisco（加州大学旧金山分校）；University of California, Berkeley（加州大学伯克利分校）
Deep Learning (DL) has shown potential in accelerating Magnetic Resonance Image acquisition and reconstruction. Nevertheless, there is a dearth of tailored methods to guarantee that the reconstruction of small features is achieved with high fidelity. In this work, we employ adversarial attacks to generate small synthetic perturbations, which are difficult to reconstruct for a trained DL reconstruction network. Then, we use robust training to increase the network's sensitivity to these small features and encourage their reconstruction. Next, we investigate the generalization of said approach to…
Deep Learning Superresolution for 7T Knee MR Imaging: Impact on Image Quality and Diagnostic Performance
Southwest Hospital, Army Medical University (Third Military Medical University)（陆军军医大学（第三军医大学）西南医院）；Siemens Healthineers Ltd.（西门子医疗有限公司）；RadioDynamic Healthcare（RadioDynamic医疗）；Tsinghua Cross-Strait Research Institute, Tsinghua University（清华大学清华两岸研究院）
University of California, Berkeley（加州大学伯克利分校）；University of California, San Francisco（加州大学旧金山分校）
Knee Osteoarthritis (OA) is a common musculoskeletal disorder in the United States. When diagnosed at early stages, lifestyle interventions such as exercise and weight loss can slow OA progression, but at later stages, only an invasive option is available: total knee replacement (TKR). Though a generally successful procedure, only 2/3 of patients who undergo the procedure report their knees feeling ''normal'' post-operation, and complications can arise that require revision. This necessitates a model to identify a population at higher risk of TKR, particularly at less advanced stages of OA, su…
Dual Attention Driven Lumbar Magnetic Resonance Image Feature Enhancement and Automatic Diagnosis of Herniation
Southern University of Science and Technology（南方科技大学）；Hainan Hospital of Chinese People’s Liberation Army General Hospital（中国人民解放军总医院海南医院）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Recently, convolutional neural networks (CNN) have been successfully applied to many remote sensing problems. However, deep learning techniques for multi-image super-resolution from multitemporal unregistered imagery have received little attention so far. This work proposes a novel CNN-based technique that exploits both spatial and temporal correlations to combine multiple images. This novel framework integrates the spatial registration task directly inside the CNN, and allows to exploit the representation learning capabilities of the network to enhance registration accuracy. The entire super-…
Mayo Clinic（梅奥诊所）；Case Western Reserve University（凯斯西储大学）；University of Southern Maine（南缅因大学）；Dubyak Center for Digital Science and Innovation, University of Southern Maine（南缅因大学Dubyak数字科学与创新中心）；Department of Artificial Intelligence and Informatics, Mayo Clinic（梅奥诊所人工智能与信息学系）；Department of Health Sciences Research, Mayo Clinic（梅奥诊所健康科学研究系）；Department of Computer and Data Sciences, Case Western Reserve University（凯斯西储大学计算机与数据科学系）；Department of Orthopedics, Mayo Clinic（梅奥诊所骨科系）；Department of Computer Science, University of Southern Maine（南缅因大学计算机科学系）
Knee pain is undoubtedly the most common musculoskeletal symptom that impairs quality of life, confines mobility and functionality across all ages. Knee pain is clinically evaluated by routine radiographs, where the widespread adoption of radiographic images and their availability at low cost, make them the principle component in the assessment of knee pain and knee pathologies, such as arthritis, trauma, and sport injuries. However, interpretation of the knee radiographs is still highly subjective, and overlapping structures within the radiographs and the large volume of images needing to be …
Early identification of COVID-19 using a deep model trained on Chest X-Ray and CT images has gained considerable attention from researchers to speed up the process of identification of active COVID-19 cases. These deep models act as an aid to hospitals that suffer from the unavailability of specialists or radiologists, specifically in remote areas. Various deep models have been proposed to detect the COVID-19 cases, but few works have been performed to prevent the deep models against adversarial attacks capable of fooling the deep model by using a small perturbation in image pixels. This paper…
The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artefacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In…
With limited storage/bandwidth resources, input images to Computer Vision (CV) applications that use Deep Neural Networks (DNNs) are often encoded with JPEG that is tailored to Human Vision (HV). This paper presents Deep Selector-JPEG, an adaptive JPEG compression method that targets image classification while satisfying HV criteria. For each image, Deep Selector-JPEG selects adaptively a Quality Factor (QF) to compress the image so that a good trade-off between the Compression Ratio (CR) and DNN classifier Accuracy (Rate-Accuracy performance) can be achieved over a set of images for a variety…
University of Nebraska at Omaha（内布拉斯加大学奥马哈分校）；New Jersey Institute of Technology（新泽西理工学院）
Digital image watermarking is the process of embedding and extracting watermark covertly on a carrier image. Incorporating deep learning networks with image watermarking has attracted increasing attention during recent years. However, existing deep learning-based watermarking systems cannot achieve robustness, blindness, and automated embedding and extraction simultaneously. In this paper, a fully automated image watermarking system based on deep neural networks is proposed to generalize the image watermarking processes. An unsupervised deep learning structure and a novel loss computation are …
Yonsei University（延世大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated. However, such an issue has not been thoroughly studied for image-to-image tasks that take an input image and generate an output image (e.g., colorization, denoising, deblurring, etc.) This paper presents comprehensive investigations into the vulnerability of deep image-to-image models to adversarial attacks. For five popular image-to-image tasks, 16 deep models are analyzed from various standpoints such as output quality degradation due to attacks, transferability of adversarial exampl…
For testing patients infected with COVID-19, along with RT-PCR testing, chest radiology images are being used. For the detection of COVID-19 from radiology images, many organizations are proposing the use of Deep Learning. University of Waterloo and DarwinAI, have designed their own Deep Learning model COVIDNet-CT to detect COVID-19 from infected chest CT images. Additionally, they have introduced a CT image dataset COVIDx-CT, from CT images collected by the China National Center for Bioinformation. COVIDx-CT contains 104,009 CT image slices across 1,489 patient cases. After obtaining remarkab…
University of Liverpool（利物浦大学）；Liverpool University Hospitals（利物浦大学医院）
Variational segmentation algorithms require a prior imposed in the form of a regularisation term to enforce smoothness of the solution. Recently, it was shown in the Deep Image Prior work that the explicit regularisation in a model can be removed and replaced by the implicit regularisation captured by the architecture of a neural network. The Deep Image Prior approach is competitive, but is only tailored to one specific image and does not allow us to predict future images. We propose to incorporate the ideas from Deep Image Prior into a more traditional learning algorithm to allow us to use th…
Azerbaijan State Oil and Industry University（阿塞拜疆国立石油与工业大学）；Azerbaijan Medical University（阿塞拜疆医科大学）；Seraj Higher Education Institute（塞拉杰高等教育学院）
Deep learning; it is often used in dividing processes on images in the biomedical field. In recent years, it has been observed that there is an increase in the division procedures performed on prostate images using deep learning compared to other methods of image division. Looking at the literature; It is seen that the process of dividing prostate images, which are carried out with deep learning, is an important step for the diagnosis and treatment of prostate cancer. For this reason, in this study; to be a source for future splitting operations; deep learning splitting procedures on prostate …
Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学医学院生物医学工程学院医学影像技术研究所）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）；Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属同仁医院影像科）
Knee osteoarthritis (OA) is one of the most common musculoskeletal disorders and requires early-stage diagnosis. Nowadays, the deep convolutional neural networks have achieved greatly in the computer-aided diagnosis field. However, the construction of the deep learning models usually requires great amounts of annotated data, which is generally high-cost. In this paper, we propose a novel approach for knee cartilage defects assessment, including severity classification and lesion localization. This can be treated as a subtask of knee OA diagnosis. Particularly, we design a self-ensembling frame…
Attenuation and scatter correction (AC) is crucial for quantitative Positron Emission Tomography (PET) imaging. Recently, direct application of AC in the image domain using deep learning approaches has been proposed for the hybrid PET/MR and dedicated PET systems that lack accompanying transmission or anatomical imaging. This study set out to investigate deep learning-based AC in the image domain using different input settings.
The University of Tokyo（东京大学）；Hitachi, Ltd.（株式会社日立制作所）
Image quality assessment (IQA) is a fundamental metric for image processing tasks (e.g., compression). With full-reference IQAs, traditional IQAs, such as PSNR and SSIM, have been used. Recently, IQAs based on deep neural networks (deep IQAs), such as LPIPS and DISTS, have also been used. It is known that image scaling is inconsistent among deep IQAs, as some perform down-scaling as pre-processing, whereas others instead use the original image size. In this paper, we show that the image scale is an influential factor that affects deep IQA performance. We comprehensively evaluate four deep IQAs…
University College London（伦敦大学学院）；InstaDeep；King’s College London（伦敦国王学院）
DeepReg (https://github.com/DeepRegNet/DeepReg) is a community-supported open-source toolkit for research and education in medical image registration using deep learning.
Since the first success of Dong et al., the deep-learning-based approach has become dominant in the field of single-image super-resolution. This replaces all the handcrafted image processing steps of traditional sparse-coding-based methods with a deep neural network. In contrast to sparse-coding-based methods, which explicitly create high/low-resolution dictionaries, the dictionaries in deep-learning-based methods are implicitly acquired as a nonlinear combination of multiple convolutions. One disadvantage of deep-learning-based methods is that their performance is degraded for images created …
Center for Translational Research in Neuroimaging and Data Science (TReNDS)（神经影像与数据科学转化研究中心（TReNDS））
Deep learning has become a prominent computational modeling tool in the areas of computer vision and image processing in recent years. This research comprehensively analyzes the different deep-learning methods used for image-to-image translation and reconstruction in the natural and medical imaging domains. We examine the famous deep learning frameworks, such as convolutional neural networks and generative adversarial networks, and their variants, delving into the fundamental principles and difficulties of each. In the field of natural computer vision, we investigate the development and extens…
Washington University in St. Louis（圣路易斯华盛顿大学）；Mallinckrodt Institute of Radiology（马林克罗特放射学研究所）
One of the key limitations in conventional deep learning based image reconstruction is the need for registered pairs of training images containing a set of high-quality groundtruth images. This paper addresses this limitation by proposing a novel unsupervised deep registration-augmented reconstruction method (U-Dream) for training deep neural nets to reconstruct high-quality images by directly mapping pairs of unregistered and artifact-corrupted images. The ability of U-Dream to circumvent the need for accurately registered data makes it widely applicable to many biomedical image reconstructio…
Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）生物计算研究中心）；Shenzhen Key Laboratory of Visual Object Detection and Recognition（深圳市视觉目标检测与识别重点实验室）；School of Computers, Guangdong University of Technology（广东工业大学计算机学院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University（台湾清华大学电机工程学系与通讯工程研究所）
Deep learning techniques have received much attention in the area of image denoising. However, there are substantial differences in the various types of deep learning methods dealing with image denoising. Specifically, discriminative learning based on deep learning can ably address the issue of Gaussian noise. Optimization models based on deep learning are effective in estimating the real noise. However, there has thus far been little related research to summarize the different deep learning techniques for image denoising. In this paper, we offer a comparative study of deep techniques in image…
We introduce DeepIR, a new thermal image processing framework that combines physically accurate sensor modeling with deep network-based image representation. Our key enabling observations are that the images captured by thermal sensors can be factored into slowly changing, scene-independent sensor non-uniformities (that can be accurately modeled using physics) and a scene-specific radiance flux (that is well-represented using a deep network-based regularizer). DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target--making it well suited for p…
Subsea images measured by the side scan sonars (SSSs) are necessary visual data in the process of deep-sea exploration by using the autonomous underwater vehicles (AUVs). They could vividly reflect the topography of the seabed, but usually accompanied by complex and severe noise. This paper proposes a deep denoising method for SSS images without high-quality reference data, which uses one single noise SSS image to perform self-supervised denoising. Compared with the classical artificially designed filters, the deep denoising method shows obvious advantages. The denoising experiments are perfor…
Nanjing University of Science and Technology（南京理工大学）；Dalian University of Technology（大连理工大学）；Google（谷歌）；Nankai University（南开大学）；SenseTime Research（商汤科技研究院）
We present a simple and effective image super-resolution algorithm that imposes an image formation constraint on the deep neural networks via pixel substitution. The proposed algorithm first uses a deep neural network to estimate intermediate high-resolution images, blurs the intermediate images using known blur kernels, and then substitutes values of the pixels at the un-decimated positions with those of the corresponding pixels from the low-resolution images. The output of the pixel substitution process strictly satisfies the image formation model and is further refined by the same deep neur…
The thinnest possible camera is achieved by removing all optics, leaving only the image sensor. We train deep neural networks to perform multi-class detection and binary classification (with accuracy of 92%) on optics-free images without the need for anthropocentric image reconstructions. Inferencing from optics-free images has the potential for enhanced privacy and power efficiency.
A novel deep learning-based magnetic resonance imaging reconstruction pipeline was designed to address fundamental image quality limitations of conventional reconstruction to provide high-resolution, low-noise MR images. This pipeline's unique aims were to convert truncation artifact into improved image sharpness while jointly denoising images to improve image quality. This new approach, now commercially available at AIR Recon DL (GE Healthcare, Waukesha, WI), includes a deep convolutional neural network (CNN) to aid in the reconstruction of raw data, ultimately producing clean, sharp images. …
