This paper proposes a robust deep learning framework used for classifying anomaly of respiratory cycles. Initially, our framework starts with front-end feature extraction step. This step aims to transform the respiratory input sound into a two-dimensional spectrogram where both spectral and temporal features are well presented. Next, an ensemble of C- DNN and Autoencoder networks is then applied to classify into four categories of respiratory anomaly cycles. In this work, we conducted experiments over 2017 Internal Conference on Biomedical Health Informatics (ICBHI) benchmark dataset. As a result, we achieve competitive performances with ICBHI average score of 0.49, ICBHI harmonic score of 0.42.
Developing a reliable sound detection and recognition system offers many benefits and has many useful applications in different industries. This paper examines the difficulties that exist when attempting to perform sound classification as it relates to respiratory disease classification. Some methods which have been employed such as independent component analysis and blind source separation are examined. Finally, an examination on the use of deep learning and long short-term memory networks is performed in order to identify how such a task can be implemented.
In this study, a machine learning model was developed for automatically detecting respiratory system sounds such as sneezing and coughing in disease diagnosis. The automatic model and approach development of breath sounds, which carry valuable information, results in early diagnosis and treatment. A successful machine learning model was developed in this study, which was a strong response to the challenge called the "Pfizer digital medicine challenge" on the "OSFHOME" open access platform. "Environmental sound classification" called ESC-50 and AudioSet sound files were used to prepare the data…
Austria Institute of Technology（奥地利技术研究所）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents an inception-based deep neural network for detecting lung diseases using respiratory sound input. Recordings of respiratory sound collected from patients are firstly transformed into spectrograms where both spectral and temporal information are well presented, referred to as front-end feature extraction. These spectrograms are then fed into the proposed network, referred to as back-end classification, for detecting whether patients suffer from lung-relevant diseases. Our experiments, conducted over the ICBHI benchmark meta-dataset of respiratory sound, achieve competitive I…
The issue in respiratory sound classification has attained good attention from the clinical scientists and medical researcher's group in the last year to diagnosing COVID-19 disease. To date, various models of Artificial Intelligence (AI) entered into the real-world to detect the COVID-19 disease from human-generated sounds such as voice/speech, cough, and breath. The Convolutional Neural Network (CNN) model is implemented for solving a lot of real-world problems on machines based on Artificial Intelligence (AI). In this context, one dimension (1D) CNN is suggested and implemented to diagnose …
University of Kent（肯特大学）；University of Science and Technology of China（中国科学技术大学）；University of Oxford（牛津大学）；Graz University of Technology（格拉茨理工大学）
This paper presents a robust deep learning framework developed to detect respiratory diseases from recordings of respiratory sounds. The complete detection process firstly involves front end feature extraction where recordings are transformed into spectrograms that convey both spectral and temporal information. Then a back-end deep learning model classifies the features into classes of respiratory disease or anomaly. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, evaluate the ability of the framework to classify sounds. Two main contributions are made in this pa…
Imperial College London（帝国理工学院）；Griffith University（格里菲斯大学）；University of Augsburg（奥格斯堡大学）
Respiratory sound classification is an important tool for remote screening of respiratory-related diseases such as pneumonia, asthma, and COVID-19. To facilitate the interpretability of classification results, especially ones based on deep learning, many explanation methods have been proposed using prototypes. However, existing explanation techniques often assume that the data is non-biased and the prediction results can be explained by a set of prototypical examples. In this work, we develop a unified example-based explanation method for selecting both representative data (prototypes) and out…
University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Audio signals generated by the human body (e.g., sighs, breathing, heart, digestion, vibration sounds) have routinely been used by clinicians as indicators to diagnose disease or assess disease progression. Until recently, such signals were usually collected through manual auscultation at scheduled visits. Research has now started to use digital technology to gather bodily sounds (e.g., from digital stethoscopes) for cardiovascular or respiratory examination, which could then be used for automatic analysis. Some initial work shows promise in detecting diagnostic signals of COVID-19 from voice …
University of Kent（肯特大学）；Queen Mary University of London（伦敦玛丽女王大学）；University of Lübeck（吕贝克大学）；Singapore Institute of Technology（新加坡理工学院）
This paper presents and explores a robust deep learning framework for auscultation analysis. This aims to classify anomalies in respiratory cycles and detect disease, from respiratory sound recordings. The framework begins with front-end feature extraction that transforms input sound into a spectrogram representation. Then, a back-end deep learning network is used to classify the spectrogram features into categories of respiratory anomaly cycles or diseases. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, confirm three main contributions towards respiratory-sound…
Towards reliable respiratory disease diagnosis based on cough sounds and vision transformers
Luca Healthcare（卢卡医疗）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；Shanghai Key Laboratory of Emergency Prevention, Diagnosis and Treatment of Respiratory Infectious Disease（上海市呼吸道传染病应急防控诊治重点实验室）
Monash University（莫纳什大学）；Monash Children’s Hospital（莫纳什儿童医院）；BC Children’s Hospital Research Institute（不列颠哥伦比亚省儿童医院研究所）
Neonatal respiratory distress is a common condition that if left untreated, can lead to short- and long-term complications. This paper investigates the usage of digital stethoscope recorded chest sounds taken within 1min post-delivery, to enable early detection and prediction of neonatal respiratory distress. Fifty-one term newborns were included in this study, 9 of whom developed respiratory distress. For each newborn, 1min anterior and posterior recordings were taken. These recordings were pre-processed to remove noisy segments and obtain high-quality heart and lung sounds. The random unders…
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification
KAIST AI（韩国科学技术院人工智能学院）；Department of AI, Kyungpook National University（庆北国立大学人工智能系）；SmartSound（SmartSound公司）；Dongguk University（东国大学）；MODULABS（MODULABS机构）
The primary objective of this paper is to build classification models and strategies to identify breathing sound anomalies (wheeze, crackle) for automated diagnosis of respiratory and pulmonary diseases. In this work we propose a deep CNN-RNN model that classifies respiratory sounds based on Mel-spectrograms. We also implement a patient specific model tuning strategy that first screens respiratory patients and then builds patient specific classification models using limited patient data for reliable anomaly detection. Moreover, we devise a local log quantization strategy for model weights to r…
Singapore University of Technology and Design（新加坡科技设计大学）
Intelligent systems are transforming the world, as well as our healthcare system. We propose a deep learning-based cough sound classification model that can distinguish between children with healthy versus pathological coughs such as asthma, upper respiratory tract infection (URTI), and lower respiratory tract infection (LRTI). In order to train a deep neural network model, we collected a new dataset of cough sounds, labelled with clinician's diagnosis. The chosen model is a bidirectional long-short term memory network (BiLSTM) based on Mel Frequency Cepstral Coefficients (MFCCs) features. The…
XAI-Driven Spectral Analysis of Cough Sounds for Respiratory Disease Characterization
Universidad de Valladolid（巴利亚多利德大学）；Instituto de Neurociencias de Castilla y León (INCYL)（卡斯蒂利亚-莱昂神经科学研究所（INCYL））；Complejo Asistencial Universitario de Salamanca（萨拉曼卡大学医疗综合体）
Respiratory diseases are among the most common causes of severe illness and death worldwide. Prevention and early diagnosis are essential to limit or even reverse the trend that characterizes the diffusion of such diseases. In this regard, the development of advanced computational tools for the analysis of respiratory auscultation sounds can become a game changer for detecting disease-related anomalies, or diseases themselves. In this work, we propose a novel learning framework for respiratory auscultation sound data. Our approach combines state-of-the-art feature extraction techniques and adv…
Indian Institute of Science（印度科学学院）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯考特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究院）
The COVID-19 pandemic has accelerated research on design of alternative, quick and effective COVID-19 diagnosis approaches. In this paper, we describe the Coswara tool, a website application designed to enable COVID-19 detection by analysing respiratory sound samples and health symptoms. A user using this service can log into a website using any device connected to the internet, provide there current health symptom information and record few sound sampled corresponding to breathing, cough, and speech. Within a minute of analysis of this information on a cloud server the website tool will outpu…
Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection
Indian Institute of Science（印度科学学院）；Fraunhofer Institute of Integrated Circuits（弗劳恩霍夫集成电路研究所）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯科特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究学院）
Tri-MTL: A Triple Multitask Learning Approach for Respiratory Disease Diagnosis
Department of Psychiatry, Wonkwang University Hospital（精神医学科，Wonkwang大学医院）；RSC LAB, MODULABS（RSC实验室，MODULABS）；The State University of New York, Korea, Republic of Korea（纽约州立大学，韩国）；Department of Pediatrics, Seoul National University Bundang Hospital（儿科，首尔国立大学医院 Bundang 分院）；Department of Pediatrics, Seoul National University College of Medicine（儿科，首尔国立大学医学院）
Donghua University（东华大学）；University of Catania（卡塔尼亚大学）；PIEAS（巴基斯坦工程与应用科学学院）；Islamia University of Bahawalpur（巴哈瓦尔布尔伊斯兰大学）
With the development of computer -systems that can collect and analyze enormous volumes of data, the medical profession is establishing several non-invasive tools. This work attempts to develop a non-invasive technique for identifying respiratory sounds acquired by a stethoscope and voice recording software via machine learning techniques. This study suggests a trained and proven CNN-based approach for categorizing respiratory sounds. A visual representation of each audio sample is constructed, allowing resource identification for classification using methods like those used to effectively des…
Improving Deep Learning-based Respiratory Sound Analysis with Frequency Selection and Attention Mechanism
TICLab, International University of Rabat（国际拉巴特大学TIC实验室）；College of Computing, Mohammed VI Polytechnic University（摩洛哥穆莱·伊斯梅尔polytechnic大学计算机学院）；Faculty of Engineering, University of Leeds（利兹大学工程学院）
Indian Institute of Science（印度科学学院）；Ramaiah Medical College Hospital（拉马亚医学院医院）；General Hospital, Hoskote（霍斯考特综合医院）；PSG Institute of Medical Sciences and Research（PSG医学科学与研究学院）
The COVID-19 outbreak resulted in multiple waves of infections that have been associated with different SARS-CoV-2 variants. Studies have reported differential impact of the variants on respiratory health of patients. We explore whether acoustic signals, collected from COVID-19 subjects, show computationally distinguishable acoustic patterns suggesting a possibility to predict the underlying virus variant. We analyze the Coswara dataset which is collected from three subject pools, namely, i) healthy, ii) COVID-19 subjects recorded during the delta variant dominant period, and iii) data from CO…
An Inception-Residual-Based Architecture with Multi-Objective Loss for Detecting Respiratory Anomalies
Queen Mary University of London（伦敦玛丽女王大学）；University of Essex（埃塞克斯大学）；Austrian Institute of Technology（奥地利技术研究所）；Amazon Alexa（亚马逊Alexa）；University of Oxford（牛津大学）
AeroSpectra Sentinel: An Auditable LLM Prompt-Chaining Decision-Support Workflow for Acute Asthma Risk Assessment from Respiratory Sounds and Clinical Signals
School of Information, Computer, and Communication Technology（信息、计算机与通信技术学院）；Sirindhorn International Institute of Technology, Thammasat University（泰国朱拉隆梭国际技术学院）
Patients with respiratory conditions typically exhibit adventitious respiratory sounds, such as wheezes. Wheeze events have variable duration. In this work we studied the influence of event duration on wheeze classification, namely how the creation of the non-wheeze class affected the classifiers' performance. First, we evaluated several classifiers on an open access respiratory sound database, with the best one reaching sensitivity and specificity values of 98% and 95%, respectively. Then, by changing one parameter in the design of the non-wheeze class, i.e., event duration, the best classifi…
Meta-Ensemble Learning with Diverse Data Splits for Improved Respiratory Sound Classification
Department of Electronic Engineering, Wonkwang University（Wonkwang大学电子工程系）；RSC LAB, MODULABS（MODULABS的RSC实验室）；AICU Global Inc.（AICU Global公司）；Seoul National University Bundang Hospital（首尔国立大学柏阳医院）
Auscultation of respiratory sounds is the primary tool for screening and diagnosing lung diseases. Automated analysis, coupled with digital stethoscopes, can play a crucial role in enabling tele-screening of fatal lung diseases. Deep neural networks (DNNs) have shown a lot of promise for such problems, and are an obvious choice. However, DNNs are extremely data hungry, and the largest respiratory dataset ICBHI has only 6898 breathing cycles, which is still small for training a satisfactory DNN model. In this work, RespireNet, we propose a simple CNN-based model, along with a suite of novel tec…
The usage of smartphone-collected respiratory sound, trained with deep learning models, for detecting and classifying COVID-19 becomes popular recently. It removes the need for in-person testing procedures especially for rural regions where related medical supplies, experienced workers, and equipment are limited. However, existing sound-based diagnostic approaches are trained in a fully supervised manner, which requires large scale well-labelled data. It is critical to discover new methods to leverage unlabelled respiratory data, which can be obtained more easily. In this paper, we propose a n…
National Taiwan University（台湾大学）；Far Eastern Memorial Hospital（远东纪念医院）；Heroic Faith Medical Science Co. Ltd.（浩信医疗科技股份有限公司）；Avalanche Computing Inc.（雪崩计算公司）；Academia Sinica（中央研究院）；HCC Healthcare Group（HCC医疗集团）
A reliable, remote, and continuous real-time respiratory sound monitor with automated respiratory sound analysis ability is urgently required in many clinical scenarios-such as in monitoring disease progression of coronavirus disease 2019-to replace conventional auscultation with a handheld stethoscope. However, a robust computerized respiratory sound analysis algorithm has not yet been validated in practical applications. In this study, we developed a lung sound database (HF_Lung_V1) comprising 9,765 audio files of lung sounds (duration of 15 s each), 34,095 inhalation labels, 18,349 exhalati…
Massachusetts General Hospital（麻省总医院）；University of Zurich（苏黎世大学）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；MyAir Inc（MyAir公司）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Objective: Sleep related respiratory abnormalities are typically detected using polysomnography. There is a need in general medicine and critical care for a more convenient method to automatically detect sleep apnea from a simple, easy-to-wear device. The objective is to automatically detect abnormal respiration and estimate the Apnea-Hypopnea-Index (AHI) with a wearable respiratory device, compared to an SpO2 signal or polysomnography using a large (n = 412) dataset serving as ground truth. Methods: Simultaneously recorded polysomnographic (PSG) and wearable respiratory effort data were used …
NTT Media Intelligence Laboratories（NTT媒体情报实验室）；NTT Communication Science Laboratories（NTT通信科学实验室）
Use of an autoencoder (AE) as a normal model is a state-of-the-art technique for unsupervised-anomaly detection in sounds (ADS). The AE is trained to minimize the sample mean of the anomaly score of normal sounds in a mini-batch. One problem with this approach is that the anomaly score of rare-normal sounds becomes higher than that of frequent-normal sounds, because the sample mean is strongly affected by frequent-normal samples, resulting in preferentially decreasing the anomaly score of frequent-normal samples. To decrease anomaly scores for both frequent- and rare-normal sounds, we propose …
In industrial applications, the early detection of malfunctioning factory machinery is crucial. In this paper, we consider acoustic malfunction detection via transfer learning. Contrary to the majority of current approaches which are based on deep autoencoders, we propose to extract features using neural networks that were pretrained on the task of image classification. We then use these features to train a variety of anomaly detection models and show that this improves results compared to convolutional autoencoders in recordings of four different factory machines in noisy environments. Moreov…
CCG ZGDV Institute（CCG ZGDV研究所）；Bosch Car Multimedia, Portugal（博世汽车多媒体（葡萄牙）公司）
This technical report describes two methods that were developed for Task 2 of the DCASE 2020 challenge. The challenge involves an unsupervised learning to detect anomalous sounds, thus only normal machine working condition samples are available during the training process. The two methods involve deep autoencoders, based on dense and convolutional architectures that use melspectogram processed sound features. Experiments were held, using the six machine type datasets of the challenge. Overall, competitive results were achieved by the proposed dense and convolutional AE, outperforming the basel…
Lung auscultation is the most effective and indispensable method for diagnosing various respiratory disorders by using the sounds from the airways during inspirium and exhalation using a stethoscope. In this study, the statistical features are calculated from intrinsic mode functions that are extracted by applying the HilbertHuang Transform to the lung sounds from 12 different auscultation regions on the chest and back. The classification of the lung sounds from asthma and healthy subjects is performed using Deep Belief Networks (DBN). The DBN classifier model with two hidden layers has been t…
A Dual-Purpose Deep Learning Model for Auscultated Lung and Tracheal Sound Analysis Based on Mixed Set Training
National Taiwan University（台湾大学）；Far Eastern Memorial Hospital（远东纪念医院）；Heroic Faith Medical Science Co., Ltd.（鸿信医学科技股份有限公司）；En Chu Kong Hospital（恩主公医院）；Oriental Institute of Technology（东方科技大学）；Avalanche Computing Inc.（雪崩计算公司）；Academia Sinica（中央研究院）；National Taiwan University Hospital（台湾大学医院）
University of Cambridge（剑桥大学）；University of Southampton（南安普顿大学）
Researchers have been battling with the question of how we can identify Coronavirus disease (COVID-19) cases efficiently, affordably and at scale. Recent work has shown how audio based approaches, which collect respiratory audio data (cough, breathing and voice) can be used for testing, however there is a lack of exploration of how biases and methodological decisions impact these tools' performance in practice. In this paper, we explore the realistic performance of audio-based digital testing of COVID-19. To investigate this, we collected a large crowdsourced respiratory audio dataset through …
We have developed an unsupervised anomalous sound detection method for machine condition monitoring that utilizes an auxiliary task -- detecting when the target machine is active. First, we train a model that detects machine activity by using normal data with machine activity labels and then use the activity-detection error as the anomaly score for a given sound clip if we have access to the ground-truth activity labels in the inference phase. If these labels are not available, the anomaly score is calculated through outlier detection on the embedding vectors obtained by the activity-detection…
Seoul National University of Science and Technology（首尔科学技术大学）
Lung sound classification is vital for early diagnosis of respiratory diseases. However, biomedical signals often exhibit inter-patient variability even among patients with the same symptoms, requiring a learning approach that considers individual differences. We propose a Patient-Aware Feature Alignment (PAFA) framework with two novel losses, Patient Cohesion-Separation Loss (PCSL) and Global Patient Alignment Loss (GPAL). PCSL clusters features of the same patient while separating those from other patients to capture patient variability, whereas GPAL draws each patient's centroid toward a gl…
Queensland University of Technology（昆士兰科技大学）；M3DICINE Pty Ltd（M3DICINE私人有限公司）
Traditionally, abnormal heart sound classification is framed as a three-stage process. The first stage involves segmenting the phonocardiogram to detect fundamental heart sounds; after which features are extracted and classification is performed. Some researchers in the field argue the segmentation step is an unwanted computational burden, whereas others embrace it as a prior step to feature extraction. When comparing accuracies achieved by studies that have segmented heart sounds before analysis with those who have overlooked that step, the question of whether to segment heart sounds before f…
NTT Media Intelligence Laboratories（NTT媒体情报实验室）；NTT Corporation（NTT公司）
This paper proposes a novel optimization principle and its implementation for unsupervised anomaly detection in sound (ADS) using an autoencoder (AE). The goal of unsupervised-ADS is to detect unknown anomalous sound without training data of anomalous sound. Use of an AE as a normal model is a state-of-the-art technique for unsupervised-ADS. To decrease the false positive rate (FPR), the AE is trained to minimize the reconstruction error of normal sounds and the anomaly score is calculated as the reconstruction error of the observed sound. Unfortunately, since this training procedure does not …
As the labor force decreases, the demand for labor-saving automatic anomalous sound detection technology that conducts maintenance of industrial equipment has grown. Conventional approaches detect anomalies based on the reconstruction errors of an autoencoder. However, when the target machine sound is non-stationary, a reconstruction error tends to be large independent of an anomaly, and its variations increased because of the difficulty of predicting the edge frames. To solve the issue, we propose an approach to anomalous detection in which the model utilizes multiple frames of a spectrogram …
