Sound-Dr: Reliable Sound Dataset and Baseline Artificial Intelligence System for Respiratory Illnesses
AI Center, FPT Software Company Limited（FPT软件公司人工智能中心）；Reliable Machine Learning Group（可靠机器学习小组）；School of Computer Science and Information Technology, University College Cork, Ireland（University College Cork信息科学与技术学院）
Truong V. Hoang, Quang H. Nguyen, Cuong Q. Nguyen, Phong X. Nguyen, Hoang D. Nguyen
As the burden of respiratory diseases continues to fall on society worldwide, this paper proposes a high-quality and reliable dataset of human sounds for studying respiratory illnesses, including pneumonia and COVID-19. It consists of coughing, mouth breathing, and nose breathing sounds together with metadata on related clinical characteristics. We also develop a proof-of-concept system for establishing baselines and benchmarking against multiple datasets, such as Coswara and COUGHVID. Our comprehensive experiments show that the Sound-Dr dataset has richer features, better performance, and is more robust to dataset shifts in various machine learning tasks. It is promising for a wide range of real-time applications on mobile devices. The proposed dataset and system will serve as practical tools to support healthcare professionals in diagnosing respiratory disorders. The dataset and code are publicly available here: https://github.com/ReML-AI/Sound-Dr/.
Multi-View Spectrogram Transformer for Respiratory Sound Classification
School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机科学学院）；Nottingham Ningbo China Beacons of Excellence Research and Innovation Institute, University of Nottingham Ningbo China（宁波诺丁汉大学诺丁汉卓越研究与创新研究院）；School of Electrical & Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；School of Computer Science and Electronic Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）；Center for Digital Safety & Security, Austrian Institute of Technology（奥地利技术研究所数字安全与安保中心）；Amazon Alexa（亚马逊Alexa）；Nuffield Department of Clinical Neurosciences, University of Oxford（牛津大学纳菲尔德临床神经科学系）
The COVID-19 pandemic presents global challenges transcending boundaries of country, race, religion, and economy. The current gold standard method for COVID-19 detection is the reverse transcription polymerase chain reaction (RT-PCR) testing. However, this method is expensive, time-consuming, and violates social distancing. Also, as the pandemic is expected to stay for a while, there is a need for an alternate diagnosis tool which overcomes these limitations, and is deployable at a large scale. The prominent symptoms of COVID-19 include cough and breathing difficulties. We foresee that respira…
Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection
Indian Institute of Science（印度科学学院）；Fraunhofer Institute of Integrated Circuits（弗劳恩霍夫集成电路研究所）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯科特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究学院）
UAEU（阿拉伯联合酋长国大学）；Staffordshire University（斯塔福德郡大学）；Tianjin University of Technology（天津理工大学）
Respiratory symptoms can be a caused by different underlying conditions, and are often caused by viral infections, such as Influenza-like illnesses or other emerging viruses like the Coronavirus. These respiratory viruses, often, have common symptoms, including coughing, high temperature, congested nose, and difficulty breathing. However, early diagnosis of the type of the virus, can be crucial, especially in cases such as the recent COVID-19 pandemic. One of the factors that contributed to the spread of the pandemic, was the late diagnosis or confusing it with regular flu-like symptoms. Scien…
An Inception-Residual-Based Architecture with Multi-Objective Loss for Detecting Respiratory Anomalies
Queen Mary University of London（伦敦玛丽女王大学）；University of Essex（埃塞克斯大学）；Austrian Institute of Technology（奥地利技术研究所）；Amazon Alexa（亚马逊Alexa）；University of Oxford（牛津大学）
National Taiwan University（台湾大学）；Far Eastern Memorial Hospital（远东纪念医院）；Heroic Faith Medical Science Co. Ltd.（浩信医疗科技股份有限公司）；Avalanche Computing Inc.（雪崩计算公司）；Academia Sinica（中央研究院）；HCC Healthcare Group（HCC医疗集团）
A reliable, remote, and continuous real-time respiratory sound monitor with automated respiratory sound analysis ability is urgently required in many clinical scenarios-such as in monitoring disease progression of coronavirus disease 2019-to replace conventional auscultation with a handheld stethoscope. However, a robust computerized respiratory sound analysis algorithm has not yet been validated in practical applications. In this study, we developed a lung sound database (HF_Lung_V1) comprising 9,765 audio files of lung sounds (duration of 15 s each), 34,095 inhalation labels, 18,349 exhalati…
The present work proposes a deep-learning-based approach for the classification of COVID-19 coughs from non-COVID-19 coughs and that can be used as a low-resource-based tool for early detection of the onset of such respiratory diseases. The proposed system uses the ResNet-50 architecture, a popularly known Convolutional Neural Network (CNN) for image recognition tasks, fed with the log-Mel spectrums of the audio data to discriminate between the two types of coughs. For the training and validation of the proposed deep learning model, this work utilizes the Track-1 dataset provided by the DiCOVA…
The recording of respiratory sounds was of significant benefit in the diagnosis of abnormalities in respiratory sounds. The duration of the sounds used in the diagnosis affects the speed of the diagnosis. In this study, the effect of window size on diagnosis of abnormalities in respiratory sounds and the most efficient recording time for diagnosis were analyzed. First, window size was applied to each sound in the data set consisting of normal and abnormal breathing sounds, 0.5 second and from 1 to 20 seconds Increased by 1 second. Then, the data applied to window size was inferred feature extr…
The technology development for point-of-care tests (POCTs) targeting respiratory diseases has witnessed a growing demand in the recent past. Investigating the presence of acoustic biomarkers in modalities such as cough, breathing and speech sounds, and using them for building POCTs can offer fast, contactless and inexpensive testing. In view of this, over the past year, we launched the ``Coswara'' project to collect cough, breathing and speech sound recordings via worldwide crowdsourcing. With this data, a call for development of diagnostic tools was announced in the Interspeech 2021 as a spec…
Indian Institute of Science（印度科学研究院）；P. D. Hinduja Hospital（P. D. 欣杜贾医院）；KMC Hospital（KMC 医院）
The DiCOVA challenge aims at accelerating research in diagnosing COVID-19 using acoustics (DiCOVA), a topic at the intersection of speech and audio processing, respiratory health diagnosis, and machine learning. This challenge is an open call for researchers to analyze a dataset of sound recordings collected from COVID-19 infected and non-COVID-19 individuals for a two-class classification. These recordings were collected via crowdsourcing from multiple countries, through a website application. The challenge features two tracks, one focusing on cough sounds, and the other on using a collection…
Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University（国立台湾大学生物医学电子与生物信息学研究所）；Heroic Faith Medical Science Corporation Limited（恒信医学科技股份有限公司）；National Taiwan University-Stanford Joint Program Office of AI in Biotechnology, Ministry of Science and Technology Joint Research Center for Artificial Intelligence Technology and All Vista Healthcare（国立台湾大学-斯坦福大学生物技术人工智能联合项目办公室 科技部人工智能技术联合研究中心）；Department of Life Science, College of Life Science, National Taiwan University（国立台湾大学生命科学学院生命科学系）；Institute of Biomedical Sciences, Academia Sinica（中央研究院生物医学科学研究所）；Department of Critical Care, Far Eastern Memorial Hospital（远东纪念医院重症监护部）
Respiratory auscultation can help healthcare professionals detect abnormal respiratory conditions if adventitious lung sounds are heard. The state-of-the-art artificial intelligence technologies based on deep learning show great potential in the development of automated respiratory sound analysis. To train a deep learning-based model, a huge number of accurate labels of normal breath sounds and adventitious sounds are needed. In this paper, we demonstrate the work of developing a respiratory sound labeling software to help annotators identify and label the inhalation, exhalation, and adventiti…
The World Health Organization (WHO) has announced a COVID-19 was a global pandemic in March 2020. It was initially started in china in the year 2019 December and affected an expanding number of nations in various countries in the last few months. In this particular situation, many techniques, methods, and AI-based classification algorithms are put in the spotlight in reacting to fight against it and reduce the rate of such a global health crisis. COVID-19's main signs are heavy temperature, different cough, cold, breathing shortness, and a combination of loss of sense of smell and chest tightn…
Analysis of respiratory sounds increases its importance every day. Many different methods are available in the analysis, and new techniques are continuing to be developed to further improve these methods. Features are extracted from audio signals and trained using different machine learning techniques. The use of deep learning, which is a different method and has increased in recent years, also shows its influence in this field. Deep learning techniques applied to the image of audio signals give good results and continue to be developed. In this study, image filters were applied to the values …
Developing a reliable sound detection and recognition system offers many benefits and has many useful applications in different industries. This paper examines the difficulties that exist when attempting to perform sound classification as it relates to respiratory disease classification. Some methods which have been employed such as independent component analysis and blind source separation are examined. Finally, an examination on the use of deep learning and long short-term memory networks is performed in order to identify how such a task can be implemented.
Phonation, or the vibration of the vocal folds, is the primary source of vocalization in the production of voiced sounds by humans. It is a complex bio-mechanical process that is highly sensitive to changes in the speaker's respiratory parameters. Since most symptomatic cases of COVID-19 present with moderate to severe impairment of respiratory functions, we hypothesize that signatures of COVID-19 may be observable by examining the vibrations of the vocal folds. Our goal is to validate this hypothesis, and to quantitatively characterize the changes observed to enable the detection of COVID-19 …
University of Augsburg（奥格斯堡大学）；Universitat Pompeu Fabra（庞培法布拉大学）；Joint Research Centre, European Commission（欧盟委员会联合研究中心）；Imperial College London（伦敦帝国理工学院）
This paper aims to automatically detect COVID-19 patients by analysing the acoustic information embedded in coughs. COVID-19 affects the respiratory system, and, consequently, respiratory-related signals have the potential to contain salient information for the task at hand. We focus on analysing the spectrogram representations of coughing samples with the aim to investigate whether COVID-19 alters the frequency content of these signals. Furthermore, this work also assesses the impact of gender in the automatic detection of COVID-19. To extract deep learnt representations of the spectrograms, …
University of Coimbra（科英布拉大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；“G. Papanikolaou” General Hospital（“G.帕帕尼古拉乌”综合医院）
Mechanically ventilated patients typically exhibit abnormal respiratory sounds. Squawks are short inspiratory adventitious sounds that may occur in patients with pneumonia, such as COVID-19 patients. In this work we devised a method for squawk detection in mechanically ventilated patients by developing algorithms for respiratory cycle estimation, squawk candidate identification, feature extraction, and clustering. The best classifier reached an F1 of 0.48 at the sound file level and an F1 of 0.66 at the recording session level. These preliminary results are promising, as they were obtained in …
Queen Mary University of London（伦敦玛丽女王大学）；University of Luebeck（吕贝克大学）
This thesis focuses on dealing with the task of acoustic scene classification (ASC), and then applied the techniques developed for ASC to a real-life application of detecting respiratory disease. To deal with ASC challenges, this thesis addresses three main factors that directly affect the performance of an ASC system. Firstly, this thesis explores input features by making use of multiple spectrograms (log-mel, Gamma, and CQT) for low-level feature extraction to tackle the issue of insufficiently discriminative or descriptive input features. Next, a novel Encoder network architecture is introd…
Recognizing patterns in lung sounds is crucial to detecting and monitoring respiratory diseases. Current techniques for analyzing respiratory sounds demand domain experts and are subject to interpretation. Hence an accurate and automatic respiratory sound classification system is desired. In this work, we took a data-driven approach to classify abnormal lung sounds. We compared the performance using three different feature extraction techniques, which are short-time Fourier transformation (STFT), Mel spectrograms, and Wav2vec, as well as three different classifiers, including pre-trained ResNe…
Remote screening of respiratory diseases has been widely studied as a non-invasive and early instrument for diagnosis purposes, especially in the pandemic. The respiratory sound classification task has been realized with numerous deep neural network (DNN) models due to their superior performance. However, in the high-stake medical domain where decisions can have significant consequences, it is desirable to develop interpretable models; thus, providing understandable reasons for physicians and patients. To address the issue, we propose a prototype learning framework, that jointly generates exem…
Austria Institute of Technology（奥地利技术研究所）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents an inception-based deep neural network for detecting lung diseases using respiratory sound input. Recordings of respiratory sound collected from patients are firstly transformed into spectrograms where both spectral and temporal information are well presented, referred to as front-end feature extraction. These spectrograms are then fed into the proposed network, referred to as back-end classification, for detecting whether patients suffer from lung-relevant diseases. Our experiments, conducted over the ICBHI benchmark meta-dataset of respiratory sound, achieve competitive I…
Ho Chi Minh City University of Technology（胡志明市理工大学）；Da Nang University of Science and Technology（岘港科技大学）
This paper proposes a robust deep learning framework used for classifying anomaly of respiratory cycles. Initially, our framework starts with front-end feature extraction step. This step aims to transform the respiratory input sound into a two-dimensional spectrogram where both spectral and temporal features are well presented. Next, an ensemble of C- DNN and Autoencoder networks is then applied to classify into four categories of respiratory anomaly cycles. In this work, we conducted experiments over 2017 Internal Conference on Biomedical Health Informatics (ICBHI) benchmark dataset. As a res…
University of Kent（肯特大学）；University of Science and Technology of China（中国科学技术大学）；University of Oxford（牛津大学）；Graz University of Technology（格拉茨理工大学）
This paper presents a robust deep learning framework developed to detect respiratory diseases from recordings of respiratory sounds. The complete detection process firstly involves front end feature extraction where recordings are transformed into spectrograms that convey both spectral and temporal information. Then a back-end deep learning model classifies the features into classes of respiratory disease or anomaly. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, evaluate the ability of the framework to classify sounds. Two main contributions are made in this pa…
In this study, a machine learning model was developed for automatically detecting respiratory system sounds such as sneezing and coughing in disease diagnosis. The automatic model and approach development of breath sounds, which carry valuable information, results in early diagnosis and treatment. A successful machine learning model was developed in this study, which was a strong response to the challenge called the "Pfizer digital medicine challenge" on the "OSFHOME" open access platform. "Environmental sound classification" called ESC-50 and AudioSet sound files were used to prepare the data…
Cough audio signal classification has been successfully used to diagnose a variety of respiratory conditions, and there has been significant interest in leveraging Machine Learning (ML) to provide widespread COVID-19 screening. However, there is currently no validated database of cough sounds with which to train such ML models. The COUGHVID dataset provides over 20,000 crowdsourced cough recordings representing a wide range of subject ages, genders, geographic locations, and COVID-19 statuses. First, we filtered the dataset using our open-sourced cough detection algorithm. Second, experienced …
The issue in respiratory sound classification has attained good attention from the clinical scientists and medical researcher's group in the last year to diagnosing COVID-19 disease. To date, various models of Artificial Intelligence (AI) entered into the real-world to detect the COVID-19 disease from human-generated sounds such as voice/speech, cough, and breath. The Convolutional Neural Network (CNN) model is implemented for solving a lot of real-world problems on machines based on Artificial Intelligence (AI). In this context, one dimension (1D) CNN is suggested and implemented to diagnose …
Imperial College London（帝国理工学院）；Griffith University（格里菲斯大学）；University of Augsburg（奥格斯堡大学）
Respiratory sound classification is an important tool for remote screening of respiratory-related diseases such as pneumonia, asthma, and COVID-19. To facilitate the interpretability of classification results, especially ones based on deep learning, many explanation methods have been proposed using prototypes. However, existing explanation techniques often assume that the data is non-biased and the prediction results can be explained by a set of prototypical examples. In this work, we develop a unified example-based explanation method for selecting both representative data (prototypes) and out…
University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Audio signals generated by the human body (e.g., sighs, breathing, heart, digestion, vibration sounds) have routinely been used by clinicians as indicators to diagnose disease or assess disease progression. Until recently, such signals were usually collected through manual auscultation at scheduled visits. Research has now started to use digital technology to gather bodily sounds (e.g., from digital stethoscopes) for cardiovascular or respiratory examination, which could then be used for automatic analysis. Some initial work shows promise in detecting diagnostic signals of COVID-19 from voice …
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification
KAIST AI（韩国科学技术院人工智能学院）；Department of AI, Kyungpook National University（庆北国立大学人工智能系）；SmartSound（SmartSound公司）；Dongguk University（东国大学）；MODULABS（MODULABS机构）
University of Kent（肯特大学）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents and explores a robust deep learning framework for auscultation analysis. This aims to classify anomalies in respiratory cycles and detect disease, from respiratory sound recordings. The framework begins with front-end feature extraction that transforms input sound into a spectrogram representation. Then, a back-end deep learning network is used to classify the spectrogram features into categories of respiratory anomaly cycles or diseases. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, confirm three main contributions towards respiratory-sound…
The primary objective of this paper is to build classification models and strategies to identify breathing sound anomalies (wheeze, crackle) for automated diagnosis of respiratory and pulmonary diseases. In this work we propose a deep CNN-RNN model that classifies respiratory sounds based on Mel-spectrograms. We also implement a patient specific model tuning strategy that first screens respiratory patients and then builds patient specific classification models using limited patient data for reliable anomaly detection. Moreover, we devise a local log quantization strategy for model weights to r…
Monash University（莫纳什大学）；Monash Children’s Hospital（莫纳什儿童医院）；BC Children’s Hospital Research Institute（不列颠哥伦比亚省儿童医院研究所）
Neonatal respiratory distress is a common condition that if left untreated, can lead to short- and long-term complications. This paper investigates the usage of digital stethoscope recorded chest sounds taken within 1min post-delivery, to enable early detection and prediction of neonatal respiratory distress. Fifty-one term newborns were included in this study, 9 of whom developed respiratory distress. For each newborn, 1min anterior and posterior recordings were taken. These recordings were pre-processed to remove noisy segments and obtain high-quality heart and lung sounds. The random unders…
CycleGuardian: A Framework for Automatic RespiratorySound classification Based on Improved Deep clustering and Contrastive Learning
School of BioMedical Engineering, Hainan University（海南大学生物医学工程学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Information and Communication, Hainan University（海南大学信息与通信学院）
