This paper addresses the issue of cough detection using only audio recordings, with the ultimate goal of quantifying and qualifying the degree of pathology for patients suffering from respiratory diseases, notably mucoviscidosis. A large set of audio features describing various aspects of the audio signal is proposed. These features are assessed in two steps. First, their intrisic potential and redundancy are evaluated using mutual information-based measures. Secondly, their efficiency is confirmed relying on three classifiers: Artificial Neural Network, Gaussian Mixture Model and Support Vector Machine. The influence of both the feature dimension and the classifier complexity are also investigated.
Technological University Dublin（都柏林理工大学）；University College Dublin（都柏林大学学院）
In this paper, an algorithm designed to detect characteristic cough events in audio recordings is presented, significantly reducing the time required for manual counting. Using time-frequency representations and independent subspace analysis (ISA), sound events that exhibit characteristics of coughs are automatically detected, producing a summary of the events detected. Using a dataset created from publicly available audio recordings, this algorithm has been tested on a variety of synthesized audio scenarios representative of those likely to be encountered by subjects undergoing an ambulatory …
University of Mons（蒙斯大学）；Cliniques St Luc（圣卢克诊所）；University of Louvain（鲁汶大学）
In the framework of assessing the pathology severity in chronic cough diseases, medical literature underlines the lack of tools for allowing the automatic, objective and reliable detection of cough events. This paper describes a system based on two microphones which we developed for this purpose. The proposed approach relies on a large variety of audio descriptors, an efficient algorithm of feature selection based on their mutual information and the use of artificial neural networks. First, the possible use of a contact microphone (placed on the patient's thorax or trachea) in complement to th…
We present an automatic non-invasive way of detecting cough events based on both accelerometer and audio signals. The acceleration signals are captured by a smartphone firmly attached to the patient's bed, using its integrated accelerometer. The audio signals are captured simultaneously by the same smartphone using an external microphone. We have compiled a manually-annotated dataset containing such simultaneously-captured acceleration and audio signals for approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic. LR, SVM and MLP are evaluated as…
Cough audio signal classification has been successfully used to diagnose a variety of respiratory conditions, and there has been significant interest in leveraging Machine Learning (ML) to provide widespread COVID-19 screening. However, there is currently no validated database of cough sounds with which to train such ML models. The COUGHVID dataset provides over 20,000 crowdsourced cough recordings representing a wide range of subject ages, genders, geographic locations, and COVID-19 statuses. First, we filtered the dataset using our open-sourced cough detection algorithm. Second, experienced …
East China Normal University（华东师范大学）；Shanghai Jiao Tong University（上海交通大学）
Cough is a common symptom of respiratory and lung diseases. Cough detection is important to prevent, assess and control epidemic, such as COVID-19. This paper proposes a model to detect cough events from cough audio signals. The models are trained by the dataset combined ESC-50 dataset with self-recorded cough recordings. The test dataset contains inpatient cough recordings collected from inpatients of the respiratory disease department in Ruijin Hospital. We totally build 15 cough detection models based on different feature numbers selected by Random Frog, Uninformative Variable Elimination (…
Imperial College London（伦敦帝国理工学院）；University of Augsburg（奥格斯堡大学）；North Bristol NHS Trust（北布里斯托尔国民保健服务信托基金）
Our main contributions are as follows: (I) We demonstrate the first attempt to diagnose COVID-19 using end-to-end deep learning from a crowd-sourced dataset of audio samples, achieving ROC-AUC of 0.846; (II) Our model, the COVID-19 Identification ResNet, (CIdeR), has potential for rapid scalability, minimal cost and improving performance as more data becomes available. This could enable regular COVID-19 testing at apopulation scale; (III) We introduce a novel modelling strategy using a custom deep neural network to diagnose COVID-19 from a joint breath and cough representation; (IV) We release…
Secretaría de Innovavión y Transformación Digital（创新与数字化转型秘书处）；Subsecretaría de Planificación Sanitaria（卫生规划副秘书处）；Ministerio de Salud（卫生部）
OBJECTIVE: Our objective is to evaluate the possibility of using cough audio recordings (spontaneous or simulated) to detect sound patterns in people who are diagnosed with COVID-19. The research question that led our work was: what is the sensitivity and specificity of a machine learning based COVID-19 cough classifier, using RT-PCR tests as gold standard? SETTING: The audio samples that were collected for this study belong to individuals who were swabbed in the City of Buenos Aires in 20 public and 1 private facilities where RT-PCR studies were carried out on patients suspected of COVID, and…
Imperial College London（伦敦帝国理工学院）；North Bristol NHS Trust（北布里斯托尔国民健康服务体系信托基金）；University of Augsburg（奥格斯堡大学）
We report on cross-running the recent COVID-19 Identification ResNet (CIdeR) on the two Interspeech 2021 COVID-19 diagnosis from cough and speech audio challenges: ComParE and DiCOVA. CIdeR is an end-to-end deep learning neural network originally designed to classify whether an individual is COVID-positive or COVID-negative based on coughing and breathing audio recordings from a published crowdsourced dataset. In the current study, we demonstrate the potential of CIdeR at binary COVID-19 diagnosis from both the COVID-19 Cough and Speech Sub-Challenges of INTERSPEECH 2021, ComParE and DiCOVA. C…
University of Cambridge（剑桥大学）；University of Southampton（南安普顿大学）
Recent work has shown the potential of using audio data (eg, cough, breathing, and voice) in the screening for COVID-19. However, these approaches only focus on one-off detection and detect the infection given the current audio sample, but do not monitor disease progression in COVID-19. Limited exploration has been put forward to continuously monitor COVID-19 progression, especially recovery, through longitudinal audio data. Tracking disease progression characteristics could lead to more timely treatment. The primary objective of this study is to explore the potential of longitudinal audio sam…
Tsinghua University（清华大学）；University of Washington（华盛顿大学）
Cough monitoring can enable new individual pulmonary health applications. Subject cough event detection is the foundation for continuous cough monitoring. Recently, the rapid growth in smart hearables has opened new opportunities for such needs. This paper proposes EarCough, which enables continuous subject cough event detection on edge computing hearables by leveraging the always-on active noise cancellation (ANC) microphones. Specifically, we proposed a lightweight end-to-end neural network model -- EarCoughNet. To evaluate the effectiveness of our method, we constructed a synchronous motion…
Virufy AI Research Group（Virufy AI研究组）；University of Alexandria（亚历山大大学）；Stanford University（斯坦福大学）；Duke University（杜克大学）
Rapid and affordable methods of testing for COVID-19 infections are essential to reduce infection rates and prevent medical facilities from becoming overwhelmed. Current approaches of detecting COVID-19 require in-person testing with expensive kits that are not always easily accessible. This study demonstrates that crowdsourced cough audio samples recorded and acquired on smartphones from around the world can be used to develop an AI-based method that accurately predicts COVID-19 infection with an ROC-AUC of 77.1% (75.2%-78.3%). Furthermore, we show that our method is able to generalize to cro…
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；National University Health System, National University of Singapore（新加坡国立大学国立大学医疗系统）；National University Hospital, Singapore（新加坡国立大学医院）；Yong Loo Lin Medical School, National University of Singapore（新加坡国立大学杨潞龄医学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
We suggested a unified system with core components of data augmentation, ImageNet-pretrained ResNet-50, cost-sensitive loss, deep ensemble learning, and uncertainty estimation to quickly and consistently detect COVID-19 using acoustic evidence. To increase the model's capacity to identify a minority class, data augmentation and cost-sensitive loss are incorporated (infected samples). In the COVID-19 detection challenge, ImageNet-pretrained ResNet-50 has been found to be effective. The unified framework also integrates deep ensemble learning and uncertainty estimation to integrate predictions f…
University of Stellenbosch（斯泰伦博斯大学）；TASK Applied Science（TASK应用科学公司）
We have performed cough detection based on measurements from an accelerometer attached to the patient's bed. This form of monitoring is less intrusive than body-attached accelerometer sensors, and sidesteps privacy concerns encountered when using audio for cough detection. For our experiments, we have compiled a manually-annotated dataset containing the acceleration signals of approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic. As classifiers, we have considered convolutional neural networks (CNN), long-short-term-memory (LSTM) networks, an…
CoughPhase-CLR: Designing an acoustics-informed foundation model for coughing sound classification
Chair of Health Informatics at the TUM University Hospital（慕尼黑工业大学大学医院健康信息学教席）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；University Hospital Augsburg at the University of Augsburg（奥格斯堡大学奥格斯堡大学医院）；Medical Faculty, Ludwig Maximilians University of Munich（慕尼黑路德维希-马克西米利安大学医学院）；Group on Language, Audio, & Music at Imperial College London（帝国理工学院语言、音频与音乐组）；Technical University of Munich（慕尼黑工业大学）
Stellenbosch University（斯泰伦博斯大学）；SAMRC Centre for Tuberculosis Research（南非医学研究理事会结核病研究中心）；DSI/NRF Centre of Excellence for Biomedical Tuberculosis Research（科学与创新部/国家研究基金会生物医学结核病研究卓越中心）
In this work, we explore recurrent neural network architectures for tuberculosis (TB) cough classification. In contrast to previous unsuccessful attempts to implement deep architectures in this domain, we show that a basic bidirectional long short-term memory network (BiLSTM) can achieve improved performance. In addition, we show that by performing greedy feature selection in conjunction with a newly-proposed attention-based architecture that learns patient invariant features, substantially better generalisation can be achieved compared to a baseline and other considered architectures. Further…
University of Augsburg（奥格斯堡大学）；Imperial College London（伦敦帝国理工学院）
Aiming to automatically detect COVID-19 from cough sounds, we propose a deep attentive multi-model fusion system evaluated on the Track-1 dataset of the DiCOVA 2021 challenge. Three kinds of representations are extracted, including hand-crafted features, image-from-audio-based deep representations, and audio-based deep representations. Afterwards, the best models on the three types of features are fused at both the feature level and the decision level. The experimental results demonstrate that the proposed attention-based fusion at the feature level achieves the best performance (AUC: 77.96%) …
Northwestern University（西北大学）；Samsung Research America（三星美国研究院）
Cough is a major symptom of respiratory-related diseases. There exists a tremendous amount of work in detecting coughs from audio but there has been no effort to identify coughs from solely inertial measurement unit (IMU). Coughing causes motion across the whole body and especially on the neck and head. Therefore, head motion data during coughing captured by a head-worn IMU sensor could be leveraged to detect coughs using a template matching algorithm. In time series template matching problems, K-Nearest Neighbors (KNN) combined with elastic distance measurement (esp. Dynamic Time Warping (DTW…
Stellenbosch University（斯泰伦博斯大学）；SAMRC Centre for Tuberculosis Research（南非医学研究理事会结核病研究中心）
We present an experimental investigation into the effectiveness of transfer learning and bottleneck feature extraction in detecting COVID-19 from audio recordings of cough, breath and speech. This type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can be deployed on inexpensive consumer hardware. We use datasets that contain recordings of coughing, sneezing, speech and other noises, but do not contain COVID-19 labels, to pre-train three deep neural networks: a CNN, an LSTM and a Resnet50. These pre-trained networks are subsequently eith…
COVID-19 (coronavirus disease 2019) pandemic caused by SARS-CoV-2 has led to a treacherous and devastating catastrophe for humanity. At the time of writing, no specific antivirus drugs or vaccines are recommended to control infection transmission and spread. The current diagnosis of COVID-19 is done by Reverse-Transcription Polymer Chain Reaction (RT-PCR) testing. However, this method is expensive, time-consuming, and not easily available in straitened regions. An interpretable and COVID-19 diagnosis AI framework is devised and developed based on the cough sounds features and symptoms metadata…
We present a deep learning based automatic cough classifier which can discriminate tuberculosis (TB) coughs from COVID-19 coughs and healthy coughs. Both TB and COVID-19 are respiratory diseases, contagious, have cough as a predominant symptom and claim thousands of lives each year. The cough audio recordings were collected at both indoor and outdoor settings and also uploaded using smartphones from subjects around the globe, thus containing various levels of noise. This cough data include 1.68 hours of TB coughs, 18.54 minutes of COVID-19 coughs and 1.69 hours of healthy coughs from 47 TB pat…
The present work proposes a deep-learning-based approach for the classification of COVID-19 coughs from non-COVID-19 coughs and that can be used as a low-resource-based tool for early detection of the onset of such respiratory diseases. The proposed system uses the ResNet-50 architecture, a popularly known Convolutional Neural Network (CNN) for image recognition tasks, fed with the log-Mel spectrums of the audio data to discriminate between the two types of coughs. For the training and validation of the proposed deep learning model, this work utilizes the Track-1 dataset provided by the DiCOVA…
Korea Advanced Institute of Science and Technology（韩国高等科学技术院）；SM Instruments Inc.（SM仪器公司）
Coughing is a typical symptom of COVID-19. To detect and localize coughing sounds remotely, a convolutional neural network (CNN) based deep learning model was developed in this work and integrated with a sound camera for the visualization of the cough sounds. The cough detection model is a binary classifier of which the input is a two second acoustic feature and the output is one of two inferences (Cough or Others). Data augmentation was performed on the collected audio files to alleviate class imbalance and reflect various background noises in practical environments. For effective featuring o…
FPT Software Company Limited（FPT软件有限公司）；Austrian Institute of Technology（奥地利技术研究所）；University of Essex（埃塞克斯大学）；University of Glasgow（格拉斯哥大学）
This paper presents a deep learning framework for detecting COVID-19 positive subjects from their cough sounds. In particular, the proposed approach comprises two main steps. In the first step, we generate a feature representing the cough sound by combining an embedding extracted from a pre-trained model and handcrafted features extracted from draw audio recording, referred to as the front-end feature extraction. Then, the combined features are fed into different back-end classification models for detecting COVID-19 positive subjects in the second step. Our experiments on the Track-2 dataset o…
University of Mons（蒙斯大学）；Cliniques St Luc（圣卢克诊所）；University of Louvain（鲁汶大学）
The development of a system for the automatic, objective and reliable detection of cough events is a need underlined by the medical literature for years. The benefit of such a tool is clear as it would allow the assessment of pathology severity in chronic cough diseases. Even though some approaches have recently reported solutions achieving this task with a relative success, there is still no standardization about the method to adopt or the sensors to use. The goal of this paper is to study objectively the performance of several sensors for cough detection: ECG, thermistor, chest belt, acceler…
University of Chittagong（吉大港大学）；Charles Sturt University（查尔斯特大学）
The objectives of this research are analysing the performance of the state-of-the-art machine learning techniques for classifying COVID-19 from cough sound and identifying the model(s) that consistently perform well across different cough datasets. Different performance evaluation metrics (such as precision, sensitivity, specificity, AUC, accuracy, etc.) make it difficult to select the best performance model. To address this issue, in this paper, we propose an ensemble-based multi-criteria decision making (MCDM) method for selecting top performance machine learning technique(s) for COVID-19 co…
We present a machine learning based COVID-19 cough classifier which can discriminate COVID-19 positive coughs from both COVID-19 negative and healthy coughs recorded on a smartphone. This type of screening is non-contact, easy to apply, and can reduce the workload in testing centres as well as limit transmission by recommending early self-isolation to those who have a cough suggestive of COVID-19. The datasets used in this study include subjects from all six continents and contain both forced and natural coughs, indicating that the approach is widely applicable. The publicly available Coswara …
University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Audio signals generated by the human body (e.g., sighs, breathing, heart, digestion, vibration sounds) have routinely been used by clinicians as indicators to diagnose disease or assess disease progression. Until recently, such signals were usually collected through manual auscultation at scheduled visits. Research has now started to use digital technology to gather bodily sounds (e.g., from digital stethoscopes) for cardiovascular or respiratory examination, which could then be used for automatic analysis. Some initial work shows promise in detecting diagnostic signals of COVID-19 from voice …
Ton Duc Thang University（孙德胜大学）；ASICLAND（ASICLAND公司）；Sejong University（世宗大学）；Hanoi University of Industry（河内工业大学）；Northeastern University（东北大学）；Deakin University（迪肯大学）
SARS-CoV-2 is colloquially known as COVID-19 that had an initial outbreak in December 2019. The deadly virus has spread across the world, taking part in the global pandemic disease since March 2020. In addition, a recent variant of SARS-CoV-2 named Delta is intractably contagious and responsible for more than four million deaths over the world. Therefore, it is vital to possess a self-testing service of SARS-CoV-2 at home. In this study, we introduce Fruit-CoV, a two-stage vision framework, which is capable of detecting SARS-CoV-2 infections through recorded cough sounds. Specifically, we conv…
MIT AutoID Laboratory（麻省理工学院自动识别实验室）；Harvard University（哈佛大学）；University of Edinburgh（爱丁堡大学）；University of Central Florida（中佛罗里达大学）；University of Milano-Bicocca（米兰比可卡大学）
We introduce a novel audio processing architecture, the Open Voice Brain Model (OVBM), improving detection accuracy for Alzheimer's (AD) longitudinal discrimination from spontaneous speech. We also outline the OVBM design methodology leading us to such architecture, which in general can incorporate multimodal biomarkers and target simultaneously several diseases and other AI tasks. Key in our methodology is the use of multiple biomarkers complementing each other, and when two of them uniquely identify different subjects in a target disease we say they are orthogonal. We illustrate the methodol…
University of Cambridge（剑桥大学）；University of Southampton（南安普顿大学）
Researchers have been battling with the question of how we can identify Coronavirus disease (COVID-19) cases efficiently, affordably and at scale. Recent work has shown how audio based approaches, which collect respiratory audio data (cough, breathing and voice) can be used for testing, however there is a lack of exploration of how biases and methodological decisions impact these tools' performance in practice. In this paper, we explore the realistic performance of audio-based digital testing of COVID-19. To investigate this, we collected a large crowdsourced respiratory audio dataset through …
Objective: The automatic discrimination between the coughing sounds produced by patients with tuberculosis (TB) and those produced by patients with other lung ailments. Approach: We present experiments based on a dataset of 1358 forced cough recordings obtained in a developing-world clinic from 16 patients with confirmed active pulmonary TB and 35 patients suffering from respiratory conditions suggestive of TB but confirmed to be TB negative. Using nested cross-validation, we have trained and evaluated five machine learning classifiers: logistic regression (LR), support vector machines (SVM), …
The current outbreak of a coronavirus, has quickly escalated to become a serious global problem that has now been declared a Public Health Emergency of International Concern by the World Health Organization. Infectious diseases know no borders, so when it comes to controlling outbreaks, timing is absolutely essential. It is so important to detect threats as early as possible, before they spread. After a first successful DiCOVA challenge, the organisers released second DiCOVA challenge with the aim of diagnosing COVID-19 through the use of breath, cough and speech audio samples. This work prese…
Imperial College London（帝国理工学院）；University of Augsburg（奥格斯堡大学）；FAU Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；University of Cambridge（剑桥大学）；Delft University of Technology（代尔夫特理工大学）；Utrecht University（乌得勒支大学）
The INTERSPEECH 2021 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the COVID-19 Cough and COVID-19 Speech Sub-Challenges, a binary classification on COVID-19 infection has to be made based on coughing sounds and speech; in the Escalation SubChallenge, a three-way assessment of the level of escalation in a dialogue is featured; and in the Primates Sub-Challenge, four species vs background need to be classified. We describe the Sub-Challenges, baseline feature extraction, and classifiers ba…
We seek to evaluate the detection performance of a rapid primary screening tool of Covid-19 solely based on the cough sound from 8,380 clinically validated samples with laboratory molecular-test (2,339 Covid-19 positives and 6,041 Covid-19 negatives). Samples were clinically labeled according to the results and severity based on quantitative RT-PCR (qRT-PCR) analysis, cycle threshold, and lymphocytes count from the patients. Our proposed generic method is an algorithm based on Empirical Mode Decomposition (EMD) with subsequent classification based on a tensor of audio features and a deep artif…
Fine-tuning Pre-trained Audio Models for COVID-19 Detection: A Technical Report
Institute of Biosciences, Languages, and Exact Sciences（生物科学、语言和精确科学学院）；São Paulo State University（圣保罗州大学）；Institute of Natural Sciences and Engineering（自然科学与工程学院）；University of São Paulo（圣保罗大学）；Institute of Mathematics and Statistics（数学与统计学院）
University of Essex（埃塞克斯大学）；Austrian Institute of Technology（奥地利技术研究所）；FPT Software Company Limited（FPT软件有限公司）
This paper evaluates a wide range of audio-based deep learning frameworks applied to the breathing, cough, and speech sounds for detecting COVID-19. In general, the audio recording inputs are transformed into low-level spectrogram features, then they are fed into pre-trained deep learning models to extract high-level embedding features. Next, the dimension of these high-level embedding features are reduced before finetuning using Light Gradient Boosting Machine (LightGBM) as a back-end classification. Our experiments on the Second DiCOVA Challenge achieved the highest Area Under the Curve (AUC…
University of Brighton（布莱顿大学）；Royal Holloway University of London（伦敦大学皇家霍洛威学院）
Audio classification using breath and cough samples has recently emerged as a low-cost, non-invasive, and accessible COVID-19 screening method. However, a comprehensive survey shows that no application has been approved for official use at the time of writing, due to the stringent reliability and accuracy requirements of the critical healthcare setting. To support the development of Machine Learning classification models, we performed an extensive comparative investigation and ranking of 15 audio features, including less well-known ones. The results were verified on two independent COVID-19 so…
University of Augsburg（奥格斯堡大学）；Johannes Kepler University Linz（约翰内斯·开普勒林茨大学）；Imperial College London（帝国理工学院）
Detecting COVID-19 from audio signals, such as breathing and coughing, can be used as a fast and efficient pre-testing method to reduce the virus transmission. Due to the promising results of deep learning networks in modelling time sequences, and since applications to rapidly identify COVID in-the-wild should require low computational effort, we present a temporal-oriented broadcasting residual learning method that achieves efficient computation and high accuracy with a small model size. Based on the EfficientNet architecture, our novel network, named Temporal-oriented ResNet~(TorNet), consti…
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）
In this paper, we propose a real-time robot-based auxiliary system for risk evaluation of COVID-19 infection. It combines real-time speech recognition, temperature measurement, keyword detection, cough detection and other functions in order to convert live audio into actionable structured data to achieve the COVID-19 infection risk assessment function. In order to better evaluate the COVID-19 infection, we propose an end-to-end method for cough detection and classification for our proposed system. It is based on real conversation data from human-robot, which processes speech signals to detect …
INICTEL – UNI（秘鲁国家电信和信息技术研究所（国立工程大学））；University of Applied Sciences（应用科学大学）
ITU-R BS.1387 states a method for objective assessment of perceived audio quality. This Recommendation, known also as PEAQ (Perceptual Evaluation of Audio Quality) is based on a psychoacoustic model of the human ear and was standardized by the International Telecommunications Union as an alternative to subjective tests, which are expensive and time-consuming processes. PEAQ combines various physiological and psycho-acoustical properties of the human ear to give a measure of the quality difference between a reference audio and a test audio. The reference audio signal could be considered as a di…
