The primary objective of this paper is to build classification models and strategies to identify breathing sound anomalies (wheeze, crackle) for automated diagnosis of respiratory and pulmonary diseases. In this work we propose a deep CNN-RNN model that classifies respiratory sounds based on Mel-spectrograms. We also implement a patient specific model tuning strategy that first screens respiratory patients and then builds patient specific classification models using limited patient data for reliable anomaly detection. Moreover, we devise a local log quantization strategy for model weights to reduce the memory footprint for deployment in memory constrained systems such as wearable devices. The proposed hybrid CNN-RNN model achieves a score of 66.31% on four-class classification of breathing cycles for ICBHI'17 scientific challenge respiratory sound database. When the model is re-trained with patient specific data, it produces a score of 71.81% for leave-one-out validation. The proposed weight quantization technique achieves ~4X reduction in total memory cost without loss of performance. The main contribution of the paper is as follows: Firstly, the proposed model is able to achieve state of the art score on the ICBHI'17 dataset. Secondly, deep learning models are shown to successfully learn domain specific knowledge when pre-trained with breathing data and produce significantly superior performance compared to generalized models. Finally, local log quantization of trained weights is shown to be able to reduce the memory requirement significantly. This type of patient-specific re-training strategy can be very useful in developing reliable long-term automated patient monitoring systems particularly in wearable healthcare solutions.
Massachusetts General Hospital（麻省总医院）；University of Zurich（苏黎世大学）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；MyAir Inc（MyAir公司）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Objective: Sleep related respiratory abnormalities are typically detected using polysomnography. There is a need in general medicine and critical care for a more convenient method to automatically detect sleep apnea from a simple, easy-to-wear device. The objective is to automatically detect abnormal respiration and estimate the Apnea-Hypopnea-Index (AHI) with a wearable respiratory device, compared to an SpO2 signal or polysomnography using a large (n = 412) dataset serving as ground truth. Methods: Simultaneously recorded polysomnographic (PSG) and wearable respiratory effort data were used …
Developing a reliable sound detection and recognition system offers many benefits and has many useful applications in different industries. This paper examines the difficulties that exist when attempting to perform sound classification as it relates to respiratory disease classification. Some methods which have been employed such as independent component analysis and blind source separation are examined. Finally, an examination on the use of deep learning and long short-term memory networks is performed in order to identify how such a task can be implemented.
University of Kent（肯特大学）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents and explores a robust deep learning framework for auscultation analysis. This aims to classify anomalies in respiratory cycles and detect disease, from respiratory sound recordings. The framework begins with front-end feature extraction that transforms input sound into a spectrogram representation. Then, a back-end deep learning network is used to classify the spectrogram features into categories of respiratory anomaly cycles or diseases. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, confirm three main contributions towards respiratory-sound…
Respiratory rate (RR) is a clinical metric used to assess overall health and physical fitness. An individual's RR can change from their baseline due to chronic illness symptoms (e.g., asthma, congestive heart failure), acute illness (e.g., breathlessness due to infection), and over the course of the day due to physical exhaustion during heightened exertion. Remote estimation of RR can offer a cost-effective method to track disease progression and cardio-respiratory fitness over time. This work investigates a model-driven approach to estimate RR from short audio segments obtained after physical…
The recording of respiratory sounds was of significant benefit in the diagnosis of abnormalities in respiratory sounds. The duration of the sounds used in the diagnosis affects the speed of the diagnosis. In this study, the effect of window size on diagnosis of abnormalities in respiratory sounds and the most efficient recording time for diagnosis were analyzed. First, window size was applied to each sound in the data set consisting of normal and abnormal breathing sounds, 0.5 second and from 1 to 20 seconds Increased by 1 second. Then, the data applied to window size was inferred feature extr…
Ho Chi Minh City University of Technology（胡志明市理工大学）；Da Nang University of Science and Technology（岘港科技大学）
This paper proposes a robust deep learning framework used for classifying anomaly of respiratory cycles. Initially, our framework starts with front-end feature extraction step. This step aims to transform the respiratory input sound into a two-dimensional spectrogram where both spectral and temporal features are well presented. Next, an ensemble of C- DNN and Autoencoder networks is then applied to classify into four categories of respiratory anomaly cycles. In this work, we conducted experiments over 2017 Internal Conference on Biomedical Health Informatics (ICBHI) benchmark dataset. As a res…
In this study, a machine learning model was developed for automatically detecting respiratory system sounds such as sneezing and coughing in disease diagnosis. The automatic model and approach development of breath sounds, which carry valuable information, results in early diagnosis and treatment. A successful machine learning model was developed in this study, which was a strong response to the challenge called the "Pfizer digital medicine challenge" on the "OSFHOME" open access platform. "Environmental sound classification" called ESC-50 and AudioSet sound files were used to prepare the data…
Austria Institute of Technology（奥地利技术研究所）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents an inception-based deep neural network for detecting lung diseases using respiratory sound input. Recordings of respiratory sound collected from patients are firstly transformed into spectrograms where both spectral and temporal information are well presented, referred to as front-end feature extraction. These spectrograms are then fed into the proposed network, referred to as back-end classification, for detecting whether patients suffer from lung-relevant diseases. Our experiments, conducted over the ICBHI benchmark meta-dataset of respiratory sound, achieve competitive I…
Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University（国立台湾大学生物医学电子与生物信息学研究所）；Heroic Faith Medical Science Corporation Limited（恒信医学科技股份有限公司）；National Taiwan University-Stanford Joint Program Office of AI in Biotechnology, Ministry of Science and Technology Joint Research Center for Artificial Intelligence Technology and All Vista Healthcare（国立台湾大学-斯坦福大学生物技术人工智能联合项目办公室 科技部人工智能技术联合研究中心）；Department of Life Science, College of Life Science, National Taiwan University（国立台湾大学生命科学学院生命科学系）；Institute of Biomedical Sciences, Academia Sinica（中央研究院生物医学科学研究所）；Department of Critical Care, Far Eastern Memorial Hospital（远东纪念医院重症监护部）
Respiratory auscultation can help healthcare professionals detect abnormal respiratory conditions if adventitious lung sounds are heard. The state-of-the-art artificial intelligence technologies based on deep learning show great potential in the development of automated respiratory sound analysis. To train a deep learning-based model, a huge number of accurate labels of normal breath sounds and adventitious sounds are needed. In this paper, we demonstrate the work of developing a respiratory sound labeling software to help annotators identify and label the inhalation, exhalation, and adventiti…
The issue in respiratory sound classification has attained good attention from the clinical scientists and medical researcher's group in the last year to diagnosing COVID-19 disease. To date, various models of Artificial Intelligence (AI) entered into the real-world to detect the COVID-19 disease from human-generated sounds such as voice/speech, cough, and breath. The Convolutional Neural Network (CNN) model is implemented for solving a lot of real-world problems on machines based on Artificial Intelligence (AI). In this context, one dimension (1D) CNN is suggested and implemented to diagnose …
University of Kent（肯特大学）；University of Science and Technology of China（中国科学技术大学）；University of Oxford（牛津大学）；Graz University of Technology（格拉茨理工大学）
This paper presents a robust deep learning framework developed to detect respiratory diseases from recordings of respiratory sounds. The complete detection process firstly involves front end feature extraction where recordings are transformed into spectrograms that convey both spectral and temporal information. Then a back-end deep learning model classifies the features into classes of respiratory disease or anomaly. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, evaluate the ability of the framework to classify sounds. Two main contributions are made in this pa…
Sound-Dr: Reliable Sound Dataset and Baseline Artificial Intelligence System for Respiratory Illnesses
AI Center, FPT Software Company Limited（FPT软件公司人工智能中心）；Reliable Machine Learning Group（可靠机器学习小组）；School of Computer Science and Information Technology, University College Cork, Ireland（University College Cork信息科学与技术学院）
Imperial College London（帝国理工学院）；Griffith University（格里菲斯大学）；University of Augsburg（奥格斯堡大学）
Respiratory sound classification is an important tool for remote screening of respiratory-related diseases such as pneumonia, asthma, and COVID-19. To facilitate the interpretability of classification results, especially ones based on deep learning, many explanation methods have been proposed using prototypes. However, existing explanation techniques often assume that the data is non-biased and the prediction results can be explained by a set of prototypical examples. In this work, we develop a unified example-based explanation method for selecting both representative data (prototypes) and out…
Recognizing patterns in lung sounds is crucial to detecting and monitoring respiratory diseases. Current techniques for analyzing respiratory sounds demand domain experts and are subject to interpretation. Hence an accurate and automatic respiratory sound classification system is desired. In this work, we took a data-driven approach to classify abnormal lung sounds. We compared the performance using three different feature extraction techniques, which are short-time Fourier transformation (STFT), Mel spectrograms, and Wav2vec, as well as three different classifiers, including pre-trained ResNe…
University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Audio signals generated by the human body (e.g., sighs, breathing, heart, digestion, vibration sounds) have routinely been used by clinicians as indicators to diagnose disease or assess disease progression. Until recently, such signals were usually collected through manual auscultation at scheduled visits. Research has now started to use digital technology to gather bodily sounds (e.g., from digital stethoscopes) for cardiovascular or respiratory examination, which could then be used for automatic analysis. Some initial work shows promise in detecting diagnostic signals of COVID-19 from voice …
University of Coimbra（科英布拉大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；“G. Papanikolaou” General Hospital（“G.帕帕尼古拉乌”综合医院）
Mechanically ventilated patients typically exhibit abnormal respiratory sounds. Squawks are short inspiratory adventitious sounds that may occur in patients with pneumonia, such as COVID-19 patients. In this work we devised a method for squawk detection in mechanically ventilated patients by developing algorithms for respiratory cycle estimation, squawk candidate identification, feature extraction, and clustering. The best classifier reached an F1 of 0.48 at the sound file level and an F1 of 0.66 at the recording session level. These preliminary results are promising, as they were obtained in …
Monash University（莫纳什大学）；Monash Children’s Hospital（莫纳什儿童医院）；BC Children’s Hospital Research Institute（不列颠哥伦比亚省儿童医院研究所）
Neonatal respiratory distress is a common condition that if left untreated, can lead to short- and long-term complications. This paper investigates the usage of digital stethoscope recorded chest sounds taken within 1min post-delivery, to enable early detection and prediction of neonatal respiratory distress. Fifty-one term newborns were included in this study, 9 of whom developed respiratory distress. For each newborn, 1min anterior and posterior recordings were taken. These recordings were pre-processed to remove noisy segments and obtain high-quality heart and lung sounds. The random unders…
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification
KAIST AI（韩国科学技术院人工智能学院）；Department of AI, Kyungpook National University（庆北国立大学人工智能系）；SmartSound（SmartSound公司）；Dongguk University（东国大学）；MODULABS（MODULABS机构）
Singapore University of Technology and Design（新加坡科技设计大学）
Intelligent systems are transforming the world, as well as our healthcare system. We propose a deep learning-based cough sound classification model that can distinguish between children with healthy versus pathological coughs such as asthma, upper respiratory tract infection (URTI), and lower respiratory tract infection (LRTI). In order to train a deep neural network model, we collected a new dataset of cough sounds, labelled with clinician's diagnosis. The chosen model is a bidirectional long-short term memory network (BiLSTM) based on Mel Frequency Cepstral Coefficients (MFCCs) features. The…
Multi-View Spectrogram Transformer for Respiratory Sound Classification
School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机科学学院）；Nottingham Ningbo China Beacons of Excellence Research and Innovation Institute, University of Nottingham Ningbo China（宁波诺丁汉大学诺丁汉卓越研究与创新研究院）；School of Electrical & Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
The World Health Organization (WHO) has announced a COVID-19 was a global pandemic in March 2020. It was initially started in china in the year 2019 December and affected an expanding number of nations in various countries in the last few months. In this particular situation, many techniques, methods, and AI-based classification algorithms are put in the spotlight in reacting to fight against it and reduce the rate of such a global health crisis. COVID-19's main signs are heavy temperature, different cough, cold, breathing shortness, and a combination of loss of sense of smell and chest tightn…
XAI-Driven Spectral Analysis of Cough Sounds for Respiratory Disease Characterization
Universidad de Valladolid（巴利亚多利德大学）；Instituto de Neurociencias de Castilla y León (INCYL)（卡斯蒂利亚-莱昂神经科学研究所（INCYL））；Complejo Asistencial Universitario de Salamanca（萨拉曼卡大学医疗综合体）
Analysis of respiratory sounds increases its importance every day. Many different methods are available in the analysis, and new techniques are continuing to be developed to further improve these methods. Features are extracted from audio signals and trained using different machine learning techniques. The use of deep learning, which is a different method and has increased in recent years, also shows its influence in this field. Deep learning techniques applied to the image of audio signals give good results and continue to be developed. In this study, image filters were applied to the values …
Respiratory diseases are among the most common causes of severe illness and death worldwide. Prevention and early diagnosis are essential to limit or even reverse the trend that characterizes the diffusion of such diseases. In this regard, the development of advanced computational tools for the analysis of respiratory auscultation sounds can become a game changer for detecting disease-related anomalies, or diseases themselves. In this work, we propose a novel learning framework for respiratory auscultation sound data. Our approach combines state-of-the-art feature extraction techniques and adv…
Sound event localization and detection (SELD) is a combined task of identifying the sound event and its direction. Deep neural networks (DNNs) are utilized to associate them with the sound signals observed by a microphone array. Although ambisonic microphones are popular in the literature of SELD, they might limit the range of applications due to their predetermined geometry. Some applications (including those for pedestrians that perform SELD while walking) require a wearable microphone array whose geometry can be designed to suit the task. In this paper, for the development of such a wearabl…
Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection
Indian Institute of Science（印度科学学院）；Fraunhofer Institute of Integrated Circuits（弗劳恩霍夫集成电路研究所）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯科特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究学院）
Indian Institute of Science（印度科学学院）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯考特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究院）
The COVID-19 pandemic has accelerated research on design of alternative, quick and effective COVID-19 diagnosis approaches. In this paper, we describe the Coswara tool, a website application designed to enable COVID-19 detection by analysing respiratory sound samples and health symptoms. A user using this service can log into a website using any device connected to the internet, provide there current health symptom information and record few sound sampled corresponding to breathing, cough, and speech. Within a minute of analysis of this information on a cloud server the website tool will outpu…
Donghua University（东华大学）；University of Catania（卡塔尼亚大学）；PIEAS（巴基斯坦工程与应用科学学院）；Islamia University of Bahawalpur（巴哈瓦尔布尔伊斯兰大学）
With the development of computer -systems that can collect and analyze enormous volumes of data, the medical profession is establishing several non-invasive tools. This work attempts to develop a non-invasive technique for identifying respiratory sounds acquired by a stethoscope and voice recording software via machine learning techniques. This study suggests a trained and proven CNN-based approach for categorizing respiratory sounds. A visual representation of each audio sample is constructed, allowing resource identification for classification using methods like those used to effectively des…
Queen Mary University of London（伦敦玛丽女王大学）；University of Luebeck（吕贝克大学）
This thesis focuses on dealing with the task of acoustic scene classification (ASC), and then applied the techniques developed for ASC to a real-life application of detecting respiratory disease. To deal with ASC challenges, this thesis addresses three main factors that directly affect the performance of an ASC system. Firstly, this thesis explores input features by making use of multiple spectrograms (log-mel, Gamma, and CQT) for low-level feature extraction to tackle the issue of insufficiently discriminative or descriptive input features. Next, a novel Encoder network architecture is introd…
Large annotated lung sound databases are publicly available and might be used to train algorithms for diagnosis systems. However, it might be a challenge to develop a well-performing algorithm for small non-public data, which have only a few subjects and show differences in recording devices and setup. In this paper, we use transfer learning to tackle the mismatch of the recording setup. This allows us to transfer knowledge from one dataset to another dataset for crackle detection in lung sounds. In particular, a single input convolutional neural network (CNN) model is pre-trained on a source …
In this paper, we use pre-trained ResNet models as backbone architectures for classification of adventitious lung sounds and respiratory diseases. The knowledge of the pre-trained model is transferred by using vanilla fine-tuning, co-tuning, stochastic normalization and the combination of the co-tuning and stochastic normalization techniques. Furthermore, data augmentation in both time domain and time-frequency domain is used to account for the class imbalance of the ICBHI and our multi-channel lung sound dataset. Additionally, we apply spectrum correction to consider the variations of the rec…
Towards reliable respiratory disease diagnosis based on cough sounds and vision transformers
Luca Healthcare（卢卡医疗）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；Shanghai Key Laboratory of Emergency Prevention, Diagnosis and Treatment of Respiratory Infectious Disease（上海市呼吸道传染病应急防控诊治重点实验室）
Improving the Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning-Based Audio Enhancement: Algorithm Development and Validation
CycleGuardian: A Framework for Automatic RespiratorySound classification Based on Improved Deep clustering and Contrastive Learning
School of BioMedical Engineering, Hainan University（海南大学生物医学工程学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Information and Communication, Hainan University（海南大学信息与通信学院）
