The epidemic disease, called the new coronavirus (COVID19), firstly occurred in Wuhan, China in December 2019. COVID19 was announced as an epidemic by World Health Organization soon after. Some of the symptoms of this disease are fever, cough, shortness of breath and difficulty in breathing. In more severe cases, death may occur as a result of infection. The most significant question in fighting the pandemic and controlling the epidemic is the early diagnosis of COVID19(+) patients and the follow-up of these patients. Therefore, various diagnostic mechanisms are used. Additionally to the RT-PCR test, medical imaging methods have been utilized, especially in the detection of COVID19(+) patients. In this study, an alternative approach was proposed by using cough data, which is one of the most prominent symptoms of COVID19(+) patients. The cough acoustic public dataset on the Virufy website was used. The entire data was normalized using z-normalization technique. The performance of the features obtained via the 5-layer empirical mode decomposition method and the performances of different classifiers has been compared. As the classifier algorithm, 5 different algorithms were used. The highest accuracy and F1-score performances were obtained by using Ensemble-Bagged-Trees algorithm as 90.6% and 90.5%, respectively. On the other hand, other classification algorithms used in the study are Support Vector Machines, Logistic Regression, Linear Discriminant Analysis and k-Nearest Neigbors, respectively. According to the results obtained, choosing the right classifier algorithm provides high results. Thus, it is clear that using cough acoustic data, those with COVID19(+) can be detected easily and effectively.
Cough is one of the most common symptoms in all respiratory diseases. In cases like Chronic Obstructive Pulmonary Disease, Asthma, acute and chronic Bronchitis and the recent pandemic Covid-19, the early identification of cough is important to provide healthcare professionals with useful clinical information such as frequency, severity, and nature of cough to enable better diagnosis. This paper presents and demonstrates best feature selection using MFCC which can help to determine cough events, eventually helping a neural network to learn and improve accuracy of cough detection. The paper prop…
Zonguldak Bülent Ecevit University（宗古尔达克·比伦特·埃杰维特大学）；Eregli Iron and Steel Co.（埃雷利钢铁公司）
The disease called the new coronavirus (COVID19) is a new viral respiratory disease that first appeared on January 13, 2020 in Wuhan, China. Some of the symptoms of this disease are fever, cough, shortness of breath and difficulty in breathing. In more serious cases, death may occur as a result of infection. COVID19 emerged as a pandemic that affected the whole world in a little while. The most important issue in the fight against the epidemic is the early diagnosis and follow-up of COVID19 (+) patients. Therefore, in addition to the RT-PCR test, medical imaging methods are also used when iden…
Stellenbosch University（斯泰伦博斯大学）；SAMRC Centre for Tuberculosis Research（南非医学研究理事会结核病研究中心）；DSI/NRF Centre of Excellence for Biomedical Tuberculosis Research（科学与创新部/国家研究基金会生物医学结核病研究卓越中心）
In this work, we explore recurrent neural network architectures for tuberculosis (TB) cough classification. In contrast to previous unsuccessful attempts to implement deep architectures in this domain, we show that a basic bidirectional long short-term memory network (BiLSTM) can achieve improved performance. In addition, we show that by performing greedy feature selection in conjunction with a newly-proposed attention-based architecture that learns patient invariant features, substantially better generalisation can be achieved compared to a baseline and other considered architectures. Further…
COVID-19 has resulted in over 100 million infections and caused worldwide lock downs due to its high transmission rate and limited testing options. Current diagnostic tests can be expensive, limited in availability, time-intensive and require risky in-person appointments. It has been established that symptomatic COVID-19 seriously impairs normal functioning of the respiratory system, thus affecting the coughing acoustics. The 2021 DiCOVA Challenge @ INTERSPEECH was designed to find scientific and engineering insights to the question by enabling participants to analyze an acoustic dataset gathe…
Korea Advanced Institute of Science and Technology（韩国高等科学技术院）；SM Instruments Inc.（SM仪器公司）
Coughing is a typical symptom of COVID-19. To detect and localize coughing sounds remotely, a convolutional neural network (CNN) based deep learning model was developed in this work and integrated with a sound camera for the visualization of the cough sounds. The cough detection model is a binary classifier of which the input is a two second acoustic feature and the output is one of two inferences (Cough or Others). Data augmentation was performed on the collected audio files to alleviate class imbalance and reflect various background noises in practical environments. For effective featuring o…
University of Science and Technology of China (USTC)（中国科学技术大学）；Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所）
In this work, we propose a bi-directional long short-term memory (BiLSTM) network based COVID-19 detection method using breath/speech/cough signals. By using the acoustic signals to train the network, respectively, we can build individual models for three tasks, whose parameters are averaged to obtain an average model, which is then used as the initialization for the BiLSTM model training of each task. This initialization method can significantly improve the performance on the three tasks, which surpasses the official baseline results. Besides, we also utilize a public pre-trained model wav2ve…
The current outbreak of a coronavirus, has quickly escalated to become a serious global problem that has now been declared a Public Health Emergency of International Concern by the World Health Organization. Infectious diseases know no borders, so when it comes to controlling outbreaks, timing is absolutely essential. It is so important to detect threats as early as possible, before they spread. After a first successful DiCOVA challenge, the organisers released second DiCOVA challenge with the aim of diagnosing COVID-19 through the use of breath, cough and speech audio samples. This work prese…
Samsung Research and Development Institute Bangalore（三星班加罗尔研发院）
The COVID-19 pandemic has resulted in more than 125 million infections and more than 2.7 million casualties. In this paper, we attempt to classify covid vs non-covid cough sounds using signal processing and deep learning methods. Air turbulence, the vibration of tissues, movement of fluid through airways, opening, and closure of glottis are some of the causes for the production of the acoustic sound signals during cough. Does the COVID-19 alter the acoustic characteristics of breath, cough, and speech sounds produced through the respiratory system? This is an open question waiting for answers.…
COVID-19 Diagnosis from Cough Acoustics using ConvNets and Data Augmentation
Electronics and Communication Engineering Department, National Institute of Technology, Silchar, India 788010（电子与通信工程系，印度尼特理工学院，西尔CHAR，印度788010）；Computer Science and Engineering Department, National Institute of Technology,Silchar, India 788010（计算机科学与工程系，印度尼特理工学院，西尔CHAR，印度788010）；Software Engineering Department, Paytm, Noida, India 110096（软件工程系，Paytm，印度诺伊达，印度110096）；Mathematics in Computer Science, University of Science of HCMC, Ho Chi Minh City, Vietnam 700000（计算机科学中的数学，河内科学大学，胡志明市，越南700000）
Verisk Analytics Inc.（维睿斯克分析公司）；Indian Institute of Science（印度科学学院）
This report describes the system used for detecting COVID-19 positives using three different acoustic modalities, namely speech, breathing, and cough in the second DiCOVA challenge. The proposed system is based on the combination of 4 different approaches, each focusing more on one aspect of the problem, and reaches the blind test AUCs of 86.41, 77.60, and 84.55, in the breathing, cough, and speech tracks, respectively, and the AUC of 85.37 in the fusion of these three tracks.
Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；University of the Basque Country (EHU)（巴斯克大学）
The technology development for point-of-care tests (POCTs) targeting respiratory diseases has witnessed a growing demand in the recent past. Investigating the presence of acoustic biomarkers in modalities such as cough, breathing and speech sounds, and using them for building POCTs can offer fast, contactless and inexpensive testing. In view of this, over the past year, we launched the ``Coswara'' project to collect cough, breathing and speech sound recordings via worldwide crowdsourcing. With this data, a call for development of diagnostic tools was announced in the Interspeech 2021 as a spec…
The Second Diagnosis of COVID-19 using Acoustics (DiCOVA) Challenge aimed at accelerating the research in acoustics based detection of COVID-19, a topic at the intersection of acoustics, signal processing, machine learning, and healthcare. This paper presents the details of the challenge, which was an open call for researchers to analyze a dataset of audio recordings consisting of breathing, cough and speech signals. This data was collected from individuals with and without COVID-19 infection, and the task in the challenge was a two-class classification. The development set audio recordings we…
Respiratory infections and chronic respiratory diseases impose a heavy health burden worldwide. Coughing is one of the most common symptoms of many such infections, and can be indicative of flare-ups of chronic respiratory diseases. Whether at a clinical or public health level, the capacity to identify bouts of coughing can aid understanding of population and individual health status. Developing health monitoring models in the context of respiratory diseases and also seasonal diseases with symptoms such as cough has the potential to improve quality of life, help clinicians and public health au…
University of Augsburg（奥格斯堡大学）；Universitat Pompeu Fabra（庞培法布拉大学）；Joint Research Centre, European Commission（欧盟委员会联合研究中心）；Imperial College London（伦敦帝国理工学院）
This paper aims to automatically detect COVID-19 patients by analysing the acoustic information embedded in coughs. COVID-19 affects the respiratory system, and, consequently, respiratory-related signals have the potential to contain salient information for the task at hand. We focus on analysing the spectrogram representations of coughing samples with the aim to investigate whether COVID-19 alters the frequency content of these signals. Furthermore, this work also assesses the impact of gender in the automatic detection of COVID-19. To extract deep learnt representations of the spectrograms, …
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；National University Health System, National University of Singapore（新加坡国立大学国立大学医疗系统）；National University Hospital, Singapore（新加坡国立大学医院）；Yong Loo Lin Medical School, National University of Singapore（新加坡国立大学杨潞龄医学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
We suggested a unified system with core components of data augmentation, ImageNet-pretrained ResNet-50, cost-sensitive loss, deep ensemble learning, and uncertainty estimation to quickly and consistently detect COVID-19 using acoustic evidence. To increase the model's capacity to identify a minority class, data augmentation and cost-sensitive loss are incorporated (infected samples). In the COVID-19 detection challenge, ImageNet-pretrained ResNet-50 has been found to be effective. The unified framework also integrates deep ensemble learning and uncertainty estimation to integrate predictions f…
Indian Institute of Science（印度科学学院）；P. D. Hinduja National Hospital and Medical Research Center（P. D. 印度教国家医院与医学研究中心）
The research direction of identifying acoustic bio-markers of respiratory diseases has received renewed interest following the onset of COVID-19 pandemic. In this paper, we design an approach to COVID-19 diagnostic using crowd-sourced multi-modal data. The data resource, consisting of acoustic signals like cough, breathing, and speech signals, along with the data of symptoms, are recorded using a web-application over a period of ten months. We investigate the use of statistical descriptors of simple time-frequency features for acoustic signals and binary features for the presence of symptoms. …
Leveraging cough sounds to optimize chest x-ray usage in low-resource settings
Christian Medical Center & Hospital（基督教医疗中心与医院）；Qure.ai Technologies（Qure.ai科技公司）；Hyfe Inc.（Hyfe公司）；University of Washington（华盛顿大学）；ISGlobal, Barcelona Institute for Global Health（ISGlobal巴塞罗那全球健康研究所）；University of Navarra（纳瓦拉大学）；Centro de Investigación Biomédica en Red de Enfermedades Infecciosas（西班牙传染病生物医学研究网络中心）
Institute of Information and Decision Technology, Dalian University of Technology（大连理工大学信息与决策技术研究所）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
In this paper, we propose a deep residual network-based method, namely the DiCOVA-Net, to identify COVID-19 infected patients based on the acoustic recording of their coughs. Since there are far more healthy people than infected patients, this classification problem faces the challenge of imbalanced data. To improve the model's ability to recognize minority class (the infected patients), we introduce data augmentation and cost-sensitive methods into our model. Besides, considering the particularity of this task, we deploy some fine-tuning techniques to adjust the pre-training ResNet50. Further…
Indian Institute of Science（印度科学学院）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯考特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究学院）
The COVID-19 outbreak resulted in multiple waves of infections that have been associated with different SARS-CoV-2 variants. Studies have reported differential impact of the variants on respiratory health of patients. We explore whether acoustic signals, collected from COVID-19 subjects, show computationally distinguishable acoustic patterns suggesting a possibility to predict the underlying virus variant. We analyze the Coswara dataset which is collected from three subject pools, namely, i) healthy, ii) COVID-19 subjects recorded during the delta variant dominant period, and iii) data from CO…
Acoustic event classification (AEC) and acoustic event detection (AED) refer to the task of detecting whether specific target events occur in audios. As long short-term memory (LSTM) leads to state-of-the-art results in various speech related tasks, it is employed as a popular solution for AEC as well. This paper focuses on investigating the dynamics of LSTM model on AEC tasks. It includes a detailed analysis on LSTM memory retaining, and a benchmarking of nine different pooling methods on LSTM models using 1.7M generated mixture clips of multiple events with different signal-to-noise ratios. …
National Institute of Informatics（国立信息学研究所）；Aalto University（阿尔托大学）
Recent advances in speech synthesis suggest that limitations such as the lossy nature of the amplitude spectrum with minimum phase approximation and the over-smoothing effect in acoustic modeling can be overcome by using advanced machine learning approaches. In this paper, we build a framework in which we can fairly compare new vocoding and acoustic modeling techniques with conventional approaches by means of a large scale crowdsourced evaluation. Results on acoustic models showed that generative adversarial networks and an autoregressive (AR) model performed better than a normal recurrent net…
Zonguldak Bulent Ecevit University（宗古尔达克比伦特埃杰维特大学）；Badji Mokhtar Annaba University（安纳巴巴吉莫赫塔尔大学）
The new coronavirus disease (COVID-19) has been declared a pandemic since March 2020 by the World Health Organization. It consists of an emerging viral infection with respiratory tropism that could develop atypical pneumonia. Experts emphasize the importance of early detection of those who have the COVID-19 virus. In this way, patients will be isolated from other people and the spread of the virus can be prevented. For this reason, it has become an area of interest to develop early diagnosis and detection methods to ensure a rapid treatment process and prevent the virus from spreading. Since t…
Institute for Systems Research, University of Maryland College Park（马里兰大学帕克分校系统研究所）
In this work, we incorporated acoustically derived source features, aperiodicity, periodicity and pitch as additional targets to an acoustic-to-articulatory speech inversion (SI) system. We also propose a Temporal Convolution based SI system, which uses auditory spectrograms as the input speech representation, to learn long-range dependencies and complex interactions between the source and vocal tract, to improve the SI task. The experiments are conducted with both the Wisconsin X-ray microbeam (XRMB) and Haskins Production Rate Comparison (HPRC) datasets, with comparisons done with respect to…
In this work, three lattice-free (LF) discriminative training criteria for purely sequence-trained neural network acoustic models are compared on LVCSR tasks, namely maximum mutual information (MMI), boosted maximum mutual information (bMMI) and state-level minimum Bayes risk (sMBR). We demonstrate that, analogous to LF-MMI, a neural network acoustic model can also be trained from scratch using LF-bMMI or LF-sMBR criteria respectively without the need of cross-entropy pre-training. Furthermore, experimental results on Switchboard-300hrs and Switchboard+Fisher-2100hrs datasets show that models …
Many speech processing tasks involve measuring the acoustic similarity between speech segments. Acoustic word embeddings (AWE) allow for efficient comparisons by mapping speech segments of arbitrary duration to fixed-dimensional vectors. For zero-resource speech processing, where unlabelled speech is the only available resource, some of the best AWE approaches rely on weak top-down constraints in the form of automatically discovered word-like segments. Rather than learning embeddings at the segment level, another line of zero-resource research has looked at representation learning at the short…
Institute for Informatics and Telematics of the National Research Council of Italy（意大利国家研究委员会信息学与远程通信研究所）；University of Milano（米兰大学）
Smartphones and wearable devices, along with Artificial Intelligence, can represent a game-changer in the pandemic control, by implementing low-cost and pervasive solutions to recognize the development of new diseases at their early stages and by potentially avoiding the rise of new outbreaks. Some recent works show promise in detecting diagnostic signals of COVID-19 from voice and coughs by using machine learning and hand-crafted acoustic features. In this paper, we decided to investigate the capabilities of the recently proposed deep embedding model L3-Net to automatically extract meaningful…
ParaLBench: A Large-Scale Benchmark for Computational Paralinguistics over Acoustic Foundation Models
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与电子工程学院）；Shenzhen Research Institute, Hunan University（湖南大学深圳研究院）；School of Information Science and Engineering, Hunan Normal University（湖南师范大学信息科学与工程学院）；UBTECH Robotics Corp（优必选机器人公司）
Expressive reading, considered the defining attribute of oral reading fluency, comprises the prosodic realization of phrasing and prominence. In the context of evaluating oral reading, it helps to establish the speaker's comprehension of the text. We consider a labeled dataset of children's reading recordings for the speaker-independent detection of prominent words using acoustic-prosodic and lexico-syntactic features. A previous well-tuned random forest ensemble predictor is replaced by an RNN sequence classifier to exploit potential context dependency across the longer utterance. Further, de…
In this work, we compare the performance of three selected techniques in open set acoustic scenes classification (ASC). We test thresholding of the softmax output of a deep network classifier, which is the most popular technique nowadays employed in ASC. Further we compare the results with the Openmax classifier which is derived from the computer vision field. As the third model, we use the Adapted Class-Conditioned Autoencoder (Adapted C2AE) which is our variation of another computer vision related technique called C2AE. Adapted C2AE encompasses a more fair comparison of the given experiments…
In this work, we study the association between song lyrics and mood through a data-driven analysis. Our data set consists of nearly one million songs, with song-mood associations derived from user playlists on the Spotify streaming platform. We take advantage of state-of-the-art natural language processing models based on transformers to learn the association between the lyrics and moods. We find that a pretrained transformer-based language model in a zero-shot setting -- i.e., out of the box with no further training on our data -- is powerful for capturing song-mood associations. Moreover, we…
Toyota Technological Institute at Chicago（芝加哥丰田理工学院）；Google Inc.（谷歌公司）
Attention-based recurrent neural encoder-decoder models present an elegant solution to the automatic speech recognition problem. This approach folds the acoustic model, pronunciation model, and language model into a single network and requires only a parallel corpus of speech and text for training. However, unlike in conventional approaches that combine separate acoustic and language models, it is not clear how to use additional (unpaired) text. While there has been previous work on methods addressing this problem, a thorough comparison among methods is still lacking. In this paper, we compare…
This paper presents an analysis of the Low-Complexity Acoustic Scene Classification task in DCASE 2022 Challenge. The task was a continuation from the previous years, but the low-complexity requirements were changed to the following: the maximum number of allowed parameters, including the zero-valued ones, was 128 K, with parameters being represented using INT8 numerical format; and the maximum number of multiply-accumulate operations at inference time was 30 million. The provided baseline system is a convolutional neural network which employs post-training quantization of parameters, resultin…
