Dementia is a syndrome, generally of a chronic nature characterized by a deterioration in cognitive function, especially in the geriatric population and is severe enough to impact their daily activities. Early diagnosis of dementia is essential to provide timely treatment to alleviate the effects and sometimes to slow the progression of dementia. Speech has been known to provide an indication of a person's cognitive state. The objective of this work is to use speech processing and machine learning techniques to automatically identify the stage of dementia such as mild cognitive impairment (MCI) or Alzheimers disease (AD). Non-linguistic acoustic parameters are used for this purpose, making this a language independent approach. We analyze the patients audio excerpts from a clinician-participant conversations taken from the Pitt corpus of DementiaBank database, to identify the speech parameters that best distinguish between MCI, AD and healthy (HC) speech. We analyze the contribution of various types of acoustic features such as spectral, temporal, cepstral their feature-level fusion and selection towards the identification of dementia stage. Additionally, we compare the performance of using feature-level fusion and score-level fusion. An accuracy of 82% is achieved using score-level fusion with an absolute improvement of 5% over feature-level fusion.
In this paper, we exploit a Fully Convolutional Network (FCN) to analyze the audio data of spontaneous speech for dementia detection. A fully convolutional network accommodates speech samples with varying lengths, thus enabling us to analyze the speech sample without manual segmentation. Specifically, we first obtain the Mel Frequency Cepstral Coefficient (MFCC) feature map from each participant's audio data and convert the speech classification task on audio data to an image classification task on MFCC feature maps. Then, to solve the data insufficiency problem, we apply transfer learning by …
University of Vigo（维戈大学）；AtlanTTic Research Center（AtlanTTic研究中心）；acceXible（acceXible公司）
Automatic detection of Alzheimer's dementia by speech processing is enhanced when features of both the acoustic waveform and the content are extracted. Audio and text transcription have been widely used in health-related tasks, as spectral and prosodic speech features, as well as semantic and linguistic content, convey information about various diseases. Hence, this paper describes the joint work of the GTM-UVIGO research group and acceXible startup to the ADDReSS challenge at INTERSPEECH 2020. The submitted systems aim to detect patterns of Alzheimer's disease from both the patient's voice an…
University of Wollongong（伍伦贡大学）；University of Wollongong in Dubai（迪拜伍伦贡大学）；University of Oslo（奥斯陆大学）
Access to informative databases is a crucial part of notable research developments. In the field of domestic audio classification, there have been significant advances in recent years. Although several audio databases exist, these can be limited in terms of the amount of information they provide, such as the exact location of the sound sources, and the associated noise levels. In this work, we detail our approach on generating an unbiased synthetic domestic audio database, consisting of sound scenes and events, emulated in both quiet and noisy environments. Data is carefully curated such that …
Dementia is a growing problem as our society ages, and detection methods are often invasive and expensive. Recent deep-learning techniques can offer a faster diagnosis and have shown promising results. However, they require large amounts of labelled data which is not easily available for the task of dementia detection. One effective solution to sparse data problems is data augmentation, though the exact methods need to be selected carefully. To date, there has been no empirical study of data augmentation on Alzheimer's disease (AD) datasets for NLP and speech processing. In this work, we inves…
University of Massachusetts Boston（马萨诸塞大学波士顿分校）；Geisel School of Medicine at Dartmouth（达特茅斯学院盖泽尔医学院）
Speech pause is an effective biomarker in dementia detection. Recent deep learning models have exploited speech pauses to achieve highly accurate dementia detection, but have not exploited the interpretability of speech pauses, i.e., what and how positions and lengths of speech pauses affect the result of dementia detection. In this paper, we will study the positions and lengths of dementia-sensitive pauses using adversarial learning approaches. Specifically, we first utilize an adversarial attack approach by adding the perturbation to the speech pauses of the testing samples, aiming to reduce…
Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data
South China University of Technology（华南理工大学）；University of Georgia（佐治亚大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital（麻省总医院）
Department of Computer Science（计算机科学系）；Sheffield Institute for Translational Neuroscience (SITraN)（谢菲尔德转化神经科学研究所（SITraN））；Department of Human Communication Sciences（人类传播科学系）；Department of Neuroscience（神经科学系）；Royal Hallamshire Hospital（皇家哈勒姆郡医院）；Academic Neurology Unit, Royal Hallamshire Hospital（皇家哈勒姆郡医院学术神经科）
Data limitation is one of the most common issues in training machine learning classifiers for medical applications. Due to ethical concerns and data privacy, the number of people that can be recruited to such experiments is generally smaller than the number of participants contributing to non-healthcare datasets. Recent research showed that generative models can be used as an effective approach for data augmentation, which can ultimately help to train more robust classifiers sparse data domains. A number of studies proved that this data augmentation technique works for image and audio data set…
MIT AutoID Laboratory（麻省理工学院自动识别实验室）；Harvard University（哈佛大学）；University of Edinburgh（爱丁堡大学）；University of Central Florida（中佛罗里达大学）；University of Milano-Bicocca（米兰比可卡大学）
We introduce a novel audio processing architecture, the Open Voice Brain Model (OVBM), improving detection accuracy for Alzheimer's (AD) longitudinal discrimination from spontaneous speech. We also outline the OVBM design methodology leading us to such architecture, which in general can incorporate multimodal biomarkers and target simultaneously several diseases and other AI tasks. Key in our methodology is the use of multiple biomarkers complementing each other, and when two of them uniquely identify different subjects in a target disease we say they are orthogonal. We illustrate the methodol…
Dementia classification from spontaneous speech using wrapper-based feature selection
Faculty of Information Technology, University of Jyväskylä（信息科技学院，于韦斯屈莱大学）；Public Health Programme, Folkhälsan Research Center（公共卫生计划，福尔哈兰研究中心）；Faculty of Sport and Health Sciences and Gerontology Research Center, University of Jyväskylä（体育与健康科学学院及老年学研究中心，于韦斯屈莱大学）；Wellbeing Services County of Central Finland（中央芬兰福祉服务县）
Tokyo Institute of Technology（东京工业大学）；Keio University School of Medicine（庆应义塾大学医学部）
We propose a non-invasive and cost-effective method to automatically detect dementia by utilizing solely speech audio data. We extract paralinguistic features for a short speech segment and use Gated Convolutional Neural Networks (GCNN) to classify it into dementia or healthy. We evaluate our method on the Pitt Corpus and on our own dataset, the PROMPT Database. Our method yields the accuracy of 73.1% on the Pitt Corpus using an average of 114 seconds of speech data. In the PROMPT Database, our method yields the accuracy of 74.7% using 4 seconds of speech data and it improves to 80.8% when we …
Alzheimer's disease (AD) is a progressive neurological disorder, meaning that the symptoms develop gradually throughout the years. It is also the main cause of dementia, which affects memory, thinking skills, and mental abilities. Nowadays, researchers have moved their interest towards AD detection from spontaneous speech, since it constitutes a time-effective procedure. However, existing state-of-the-art works proposing multimodal approaches do not take into consideration the inter- and intra-modal interactions and propose early and late fusion approaches. To tackle these limitations, we prop…
Introduction: Alzheimer's disease is a type of dementia in which early diagnosis plays a major rule in the quality of treatment. Among new works in the diagnosis of Alzheimer's disease, there are many of them analyzing the voice stream acoustically, syntactically or both. The mostly used tools to perform these analysis usually include machine learning techniques. Objective: Designing an automatic machine learning based diagnosis system will help in the procedure of early detection. Also, systems, using noninvasive data are preferable. Methods: We used are classification system based on spoken …
Benchmarking Foundation Speech and Language Models for Alzheimer's Disease and Related Dementia Detection from Spontaneous Speech
Georgia Institute of Technology（佐治亚理工学院）；Shandong Mental Health Center（山东精神卫生中心）；University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；Kennesaw State University（肯尼索州立大学）
Swin-BERT: A Feature Fusion System designed for Speech-based Alzheimer's Dementia Detection
College of Artificial Intelligence, Dalian Maritime University（大连海事大学人工智能学院）；University of Sheffield（谢菲尔德大学）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）
From Black Box to Biomarker: Sparse Autoencoders for Interpreting Speech Models of Parkinson's Disease
McGill University（麦吉尔大学）；Centre for Research on Brain, Language, and Music（大脑、语言和音乐研究中心）；Mila - Quebec AI Institute（魁北克人工智能研究所）；Douglas Research Centre（道格拉斯研究中心）；Concordia University（Concordia大学）
This work explores speech as a biomarker and investigates the detection of respiratory insufficiency (RI) by analyzing speech samples. Previous work \cite{spira2021} constructed a dataset of respiratory insufficiency COVID-19 patient utterances and analyzed it by means of a convolutional neural network achieving an accuracy of $87.04\%$, validating the hypothesis that one can detect RI through speech. Here, we study how Transformer neural network architectures can improve the performance on RI detection. This approach enables construction of an acoustic model. By choosing the correct pretraini…
Decoding while Adapting: Zero-Shot Online Speaker Adaptation via Audio-Textual Prompts for Elderly Speech Recognition
The Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学）；Institute of Software, Chinese Academy of Sciences, China（中国科学院软件研究所）；National Research Council Canada, Canada（加拿大国家研究理事会）
University of Augsburg（奥格斯堡大学）；University of Regensburg（雷根斯堡大学）；Hume AI
Stress is a major threat to well-being that manifests in a variety of physiological and mental symptoms. Utilising speech samples collected while the subject is undergoing an induced stress episode has recently shown promising results for the automatic characterisation of individual stress responses. In this work, we introduce new findings that shed light onto whether speech signals are suited to model physiological biomarkers, as obtained via cortisol measurements, or self-assessed appraisal and affect measurements. Our results show that different indicators impact acoustic features in a dive…
Discriminant audio properties in deep learning based respiratory insufficiency detection in Brazilian Portuguese
Universidade de São Paulo（圣保罗大学）；Universidade Estadual Paulista（圣保罗州立大学）；Center for Artificial Intelligence (C4AI-USP)（人工智能中心（C4AI-USP））；IBM Corporation（国际商业机器公司（IBM））
Universidad del País Vasco/Euskal Herriko Unibertsitatea（巴斯克大学）；Fundación CITA Alzheimer（CITA阿尔茨海默基金会）
Alzheimer's disease (AD) is one the main causes of dementia in the world and the patients develop severe disability and sometime full dependence. In previous stages Mild Cognitive Impairment (MCI) produces cognitive loss but not severe enough to interfere with daily life. This work, on selection of biomarkers from speech for the detection of AD, is part of a wide-ranging cross study for the diagnosis of Alzheimer. Specifically in this work a task for detection of MCI has been used. The task analyzes Categorical Verbal Fluency. The automatic classification is carried out by SVM over classical l…
Paris Brain Institute - ICM（巴黎大脑研究所 - ICM）；Institut Polytechnique de Paris（巴黎综合理工学院）；Sorbonne University（索邦大学）；Concordia University（康考迪亚大学）
Many articles have used voice analysis to detect Parkinson's disease (PD), but few have focused on the early stages of the disease and the gender effect. In this article, we have adapted the latest speaker recognition system, called x-vectors, in order to detect an early stage of PD from voice analysis. X-vectors are embeddings extracted from a deep neural network, which provide robust speaker representations and improve speaker recognition when large amounts of training data are used. Our goal was to assess whether, in the context of early PD detection, this technique would outperform the mor…
Usher Institute, Edinburgh Medical School, The University of Edinburgh（爱丁堡大学爱丁堡医学院厄舍研究所）；Carnegie Mellon University（卡内基梅隆大学）
Building on the success of the ADReSS Challenge at Interspeech 2020, which attracted the participation of 34 teams from across the world, the ADReSSo Challenge targets three difficult automatic prediction problems of societal and medical relevance, namely: detection of Alzheimer's Dementia, inference of cognitive testing scores, and prediction of cognitive decline. This paper presents these prediction tasks in detail, describes the datasets used, and reports the results of the baseline classification and regression models we developed for each task. A combination of acoustic and linguistic fea…
University of Toronto（多伦多大学）；Winterlight Labs（Winterlight实验室）
Traditional screening practices for anxiety and depression pose an impediment to monitoring and treating these conditions effectively. However, recent advances in NLP and speech modelling allow textual, acoustic, and hand-crafted language-based features to jointly form the basis of future mental health screening and condition detection. Speech is a rich and readily available source of insight into an individual's cognitive state and by leveraging different aspects of speech, we can develop new digital biomarkers for depression and anxiety. To this end, we propose a multi-modal system for the s…
Identification of Cognitive Decline from Spoken Language through Feature Selection and the Bag of Acoustic Words Model
Faculty of Information Technology, University of Jyväskylä（于韦斯屈莱大学信息技术学院）；Faculty of Sport and Health Sciences, University of Jyväskylä（于韦斯屈莱大学体育与健康科学学院）
Contrasting Deep Learning Models for Direct Respiratory Insufficiency Detection Versus Blood Oxygen Saturation Estimation
IME - University of São Paulo（圣保罗大学IME学院）；Federal Technological University of Parana（巴拉那联邦技术大学）；IBILCE - São Paulo State University（圣保罗州立大学IBILCE学院）；São Paulo State University（圣保罗州立大学）；FMUSP - University of São Paulo（圣保罗大学FMUSP学院）；FFLCH - University of São Paulo（圣保罗大学FFLCH学院）
The Unreliability of Acoustic Systems in Alzheimer's Speech Datasets with Heterogeneous Recording Conditions
Instituto de Investigación en Ciencias de la Computación, UBA-CONICET（布宜诺斯艾利斯大学-国家科学技术研究委员会计算机科学研究所）；Departamento de Computación, Facultad de Ciencias Exactas y Naturales, UBA（布宜诺斯艾利斯大学精确与自然科学学院计算机系）；Centro de Neurociencias Cognitivas, Universidad de San Andrés（圣安德烈斯大学认知神经科学中心）
CognoSpeak: an automatic, remote assessment of early cognitive decline in real-world conversational speech
School of Computer Science, University of Sheffield, Sheffield, S1 4DP, UK（1 计算机科学学院，谢菲尔德大学，谢菲尔德，S1 4DP，英国）；Therapy Box, London, UK（2 Therapy Box，伦敦，英国）；NIHR Devices for Dignity HTC, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, S10 2JF, UK（3 NIHR尊严设备HTC，谢菲尔德教学医院国家健康服务基金会信托，谢菲尔德，S10 2JF，英国）；Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield, Sheffield, S10 2HQ, UK（4 谢菲尔德转化神经科学研究所（SITraN），谢菲尔德大学，谢菲尔德，S10 2HQ，英国）
BenSParX: A Robust Explainable Machine Learning Framework for Parkinson's Disease Detection from Bengali Conversational Speech
Chittagong University of Engineering and Technology（吉大港工程技术大学）；East Delta University（东三角洲大学）；Charles Sturt University（查尔斯特大学）；Bangabandhu Sheikh Mujib Medical University（班加班杜·谢赫·穆吉布医科大学）
