The ADReSS Challenge at INTERSPEECH 2020 defines a shared task through which different approaches to the automated recognition of Alzheimer's dementia based on spontaneous speech can be compared. ADReSS provides researchers with a benchmark speech dataset which has been acoustically pre-processed and balanced in terms of age and gender, defining two cognitive assessment tasks, namely: the Alzheimer's speech classification task and the neuropsychological score regression task. In the Alzheimer's speech classification task, ADReSS challenge participants create models for classifying speech as dementia or healthy control speech. In the the neuropsychological score regression task, participants create models to predict mini-mental state examination scores. This paper describes the ADReSS Challenge in detail and presents a baseline for both tasks, including feature extraction procedures and results for classification and regression models. ADReSS aims to provide the speech and language Alzheimer's research community with a platform for comprehensive methodological comparisons. This will hopefully contribute to addressing the lack of standardisation that currently affects the field and shed light on avenues for future research and clinical applicability.
Collecting and accessing a large amount of medical data is very time-consuming and laborious, not only because it is difficult to find specific patients but also because it is required to resolve the confidentiality of a patient's medical records. On the other hand, there are deep learning models, trained on easily collectible, large scale datasets such as Youtube or Wikipedia, offering useful representations. It could therefore be very advantageous to utilize the features from these pre-trained networks for handling a small amount of data at hand. In this work, we exploit various multi-modal …
BITS Pilani, K.K. Birla Goa Campus（Birla技术与科学学院皮拉尼校区 K.K. Birla果阿校区）；IEMN Université de Lille（里尔大学微电子与纳米技术研究所）
Alzheimers disease is a fatal progressive brain disorder that worsens with time. It is high time we have inexpensive and quick clinical diagnostic techniques for early detection and care. In previous studies, various Machine Learning techniques and Pre-trained Deep Learning models have been used in conjunction with the extraction of various acoustic and linguistic features. Our study focuses on three models for the classification task in the ADReSS (The Alzheimers Dementia Recognition through Spontaneous Speech) 2021 Challenge. We use the well-balanced dataset provided by the ADReSS Challenge …
Swin-BERT: A Feature Fusion System designed for Speech-based Alzheimer's Dementia Detection
College of Artificial Intelligence, Dalian Maritime University（大连海事大学人工智能学院）；University of Sheffield（谢菲尔德大学）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）
Usher Institute, Edinburgh Medical School, The University of Edinburgh（爱丁堡大学爱丁堡医学院厄舍研究所）；Carnegie Mellon University（卡内基梅隆大学）
Building on the success of the ADReSS Challenge at Interspeech 2020, which attracted the participation of 34 teams from across the world, the ADReSSo Challenge targets three difficult automatic prediction problems of societal and medical relevance, namely: detection of Alzheimer's Dementia, inference of cognitive testing scores, and prediction of cognitive decline. This paper presents these prediction tasks in detail, describes the datasets used, and reports the results of the baseline classification and regression models we developed for each task. A combination of acoustic and linguistic fea…
Benchmarking Foundation Speech and Language Models for Alzheimer's Disease and Related Dementia Detection from Spontaneous Speech
Georgia Institute of Technology（佐治亚理工学院）；Shandong Mental Health Center（山东精神卫生中心）；University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；Kennesaw State University（肯尼索州立大学）
DementiaBank-Emotion: A Multi-Rater Emotion Annotation Corpus for Alzheimer's Disease Speech (Version 1.0)
Sue & Bill Gross School of Nursing, University of California, Irvine (UCI)（加州大学厄尔布雷特分校护理学院）；Donald Bren School of Information and Computer Sciences, UCI, USA（加州大学厄尔布雷特分校信息与计算机科学学院）；Smart Forward, Rancho Palos Verdes, USA（Smart Forward公司）；Dept. of Psychiatry and Human Behavior, UCI, USA（加州大学厄尔布雷特分校精神病学与人类行为系）；Amore Senior Living, Laguna Niguel, USA（Amore Senior Living公司）；School of Medicine, University of California, Irvine（加州大学厄尔布雷特分校医学院）；Purdue University, West Lafayette, USA（普渡大学）；The George Washington University, Washington, D.C., USA（乔治华盛顿大学）
In this paper, we exploit a Fully Convolutional Network (FCN) to analyze the audio data of spontaneous speech for dementia detection. A fully convolutional network accommodates speech samples with varying lengths, thus enabling us to analyze the speech sample without manual segmentation. Specifically, we first obtain the Mel Frequency Cepstral Coefficient (MFCC) feature map from each participant's audio data and convert the speech classification task on audio data to an image classification task on MFCC feature maps. Then, to solve the data insufficiency problem, we apply transfer learning by …
Winterlight Labs（Winterlight实验室）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（Vector人工智能研究所）；Li Ka Shing Knowledge Institute（李嘉诚知识研究院）；St Michael’s Hospital（圣迈克尔医院）；Surgical Safety Technologies Inc.（手术安全技术公司）
A significant number of studies apply acoustic and linguistic characteristics of human speech as prominent markers of dementia and depression. However, studies on discriminating depression from dementia are rare. Co-morbid depression is frequent in dementia and these clinical conditions share many overlapping symptoms, but the ability to distinguish between depression and dementia is essential as depression is often curable. In this work, we investigate the ability of clustering approaches in distinguishing between depression and dementia from human speech. We introduce a novel aggregated data…
A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning
Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens（决策支持系统实验室，电气与计算机工程学院，国家技术大学雅典）
Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADRe…
Alzheimer's disease (AD) is a progressive neurological disorder, meaning that the symptoms develop gradually throughout the years. It is also the main cause of dementia, which affects memory, thinking skills, and mental abilities. Nowadays, researchers have moved their interest towards AD detection from spontaneous speech, since it constitutes a time-effective procedure. However, existing state-of-the-art works proposing multimodal approaches do not take into consideration the inter- and intra-modal interactions and propose early and late fusion approaches. To tackle these limitations, we prop…
Massachusetts Institute of Technology（麻省理工学院）；BITS Pilani Goa Campus（贝拉理工学院皮拉尼果阿校区）
Reliability in Neural Networks (NNs) is crucial in safety-critical applications like healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of NNs in deployment. In this work, we propose an uncertainty-aware boosting technique for multi-modal ensembling to predict Alzheimer's Dementia Severity. The propagation of uncertainty across acoustic, cognitive, and linguistic features produces an ensemble system robust to heteroscedasticity in the data. Weighing the different modalities based on the uncertainty estimates, we experiment on the benchmark ADReSS d…
Alzheimer's Dementia (AD) is an incurable, debilitating, and progressive neurodegenerative condition that affects cognitive function. Early diagnosis is important as therapeutics can delay progression and give those diagnosed vital time. Developing models that analyse spontaneous speech could eventually provide an efficient diagnostic modality for earlier diagnosis of AD. The Alzheimer's Dementia Recognition through Spontaneous Speech task offers acoustically pre-processed and balanced datasets for the classification and prediction of AD and associated phenotypes through the modelling of spont…
Introduction: Alzheimer's disease is a type of dementia in which early diagnosis plays a major rule in the quality of treatment. Among new works in the diagnosis of Alzheimer's disease, there are many of them analyzing the voice stream acoustically, syntactically or both. The mostly used tools to perform these analysis usually include machine learning techniques. Objective: Designing an automatic machine learning based diagnosis system will help in the procedure of early detection. Also, systems, using noninvasive data are preferable. Methods: We used are classification system based on spoken …
Usher Institute, Edinburgh Medical School, The University of Edinburgh（爱丁堡大学爱丁堡医学院厄舍研究所）；Carnegie Mellon University（卡内基梅隆大学）；Information Technologies Institute, CERTH（希腊研究与技术基金会信息技术研究所）
This Signal Processing Grand Challenge (SPGC) targets a difficult automatic prediction problem of societal and medical relevance, namely, the detection of Alzheimer's Dementia (AD). Participants were invited to employ signal processing and machine learning methods to create predictive models based on spontaneous speech data. The Challenge has been designed to assess the extent to which predictive models built based on speech in one language (English) generalise to another language (Greek). To the best of our knowledge no work has investigated acoustic features of the speech signal in multiling…
University of Vigo（维戈大学）；AtlanTTic Research Center（AtlanTTic研究中心）；acceXible（acceXible公司）
Automatic detection of Alzheimer's dementia by speech processing is enhanced when features of both the acoustic waveform and the content are extracted. Audio and text transcription have been widely used in health-related tasks, as spectral and prosodic speech features, as well as semantic and linguistic content, convey information about various diseases. Hence, this paper describes the joint work of the GTM-UVIGO research group and acceXible startup to the ADDReSS challenge at INTERSPEECH 2020. The submitted systems aim to detect patterns of Alzheimer's disease from both the patient's voice an…
Queen Mary University of London（伦敦玛丽女王大学）；Jožef Stefan Institute（约热夫·斯特凡研究所）
We present two multimodal fusion-based deep learning models that consume ASR transcribed speech and acoustic data simultaneously to classify whether a speaker in a structured diagnostic task has Alzheimer's Disease and to what degree, evaluating the ADReSSo challenge 2021 data. Our best model, a BiLSTM with highway layers using words, word probabilities, disfluency features, pause information, and a variety of acoustic features, achieves an accuracy of 84% and RSME error prediction of 4.26 on MMSE cognitive scores. While predicting cognitive decline is more challenging, our models show improve…
This paper describes a multi-modal approach for the automatic detection of Alzheimer's disease proposed in the context of the INESC-ID Human Language Technology Laboratory participation in the ADReSS 2020 challenge. Our classification framework takes advantage of both acoustic and textual feature embeddings, which are extracted independently and later combined. Speech signals are encoded into acoustic features using DNN speaker embeddings extracted from pre-trained models. For textual input, contextual embedding vectors are first extracted using an English Bert model and then used either to di…
Freelance（自由职业者）；longterm care centre (EHPAD)（长期护理中心（EHPAD））
Sound environment is a prime source of conscious and unconscious information which allows listeners to place themselves, to communicate, to feel, to remember. The author describes the process of designing a new audio interactive apparatus for Alzheimer's care, in the context of an active multidisciplinary research project led by the author in collaboration with a longterm care centre (EHPAD) in Burgundy (France), a geriatrician, a gerontologist, psychologists and caregivers. The apparatus, named Madeleines Sonores in reference to Proust's madeleine, have provided virtual soundscapes sounding f…
We present an approach to automatic detection of Alzheimer's type dementia based on characteristics of spontaneous spoken language dialogue consisting of interviews recorded in natural settings. The proposed method employs additive logistic regression (a machine learning boosting method) on content-free features extracted from dialogical interaction to build a predictive model. The model training data consisted of 21 dialogues between patients with Alzheimer's and interviewers, and 17 dialogues between patients with other health conditions and interviewers. Features analysed included speech ra…
RWTH Aachen University（亚琛工业大学）；University of Amsterdam（阿姆斯特丹大学）
In this paper, we combined linguistic complexity and (dis)fluency features with pretrained language models for the task of Alzheimer's disease detection of the 2021 ADReSSo (Alzheimer's Dementia Recognition through Spontaneous Speech) challenge. An accuracy of 83.1% was achieved on the test set, which amounts to an improvement of 4.23% over the baseline model. Our best-performing model that integrated component models using a stacking ensemble technique performed equally well on cross-validation and test data, indicating that it is robust against overfitting.
Universitat Pompeu Fabra（庞培法布拉大学）；Universitat de Barcelona（巴塞罗那大学）
Alzheimer's Disease is the most common form of dementia. Automatic detection from speech could help to identify symptoms at early stages, so that preventive actions can be carried out. This research is a contribution to the ADReSSo Challenge, we analyze the usage of a SotA ASR system to transcribe participant's spoken descriptions from a picture. We analyse the loss of performance regarding the use of human transcriptions (measured using transcriptions from the 2020 ADReSS Challenge). Furthermore, we study the influence of a language model -- which tends to correct non-standard sequences of wo…
Winterlight Labs（温特莱特实验室）；University of Toronto（多伦多大学）
Robust strategies for Alzheimer's disease (AD) detection are important, given the high prevalence of AD. In this paper, we study the performance and generalizability of three approaches for AD detection from speech on the recent ADReSSo challenge dataset: 1) using conventional acoustic features 2) using novel pre-trained acoustic embeddings 3) combining acoustic features and embeddings. We find that while feature-based approaches have a higher precision, classification approaches relying on pre-trained embeddings prove to have a higher, and more balanced cross-validated performance across mult…
Dementia classification from spontaneous speech using wrapper-based feature selection
Faculty of Information Technology, University of Jyväskylä（信息科技学院，于韦斯屈莱大学）；Public Health Programme, Folkhälsan Research Center（公共卫生计划，福尔哈兰研究中心）；Faculty of Sport and Health Sciences and Gerontology Research Center, University of Jyväskylä（体育与健康科学学院及老年学研究中心，于韦斯屈莱大学）；Wellbeing Services County of Central Finland（中央芬兰福祉服务县）
Alzheimer's disease (AD) is a progressive neurodegenerative disease most often associated with memory deficits and cognitive decline. With the aging population, there has been much interest in automated methods for cognitive impairment detection. One approach that has attracted attention in recent years is AD detection through spontaneous speech. While the results are promising, it is not certain whether the learned speech features can be generalized across languages. To fill this gap, the ADReSS-M challenge was organized. This paper presents our submission to this ICASSP-2023 Signal Processin…
Dementia is a syndrome, generally of a chronic nature characterized by a deterioration in cognitive function, especially in the geriatric population and is severe enough to impact their daily activities. Early diagnosis of dementia is essential to provide timely treatment to alleviate the effects and sometimes to slow the progression of dementia. Speech has been known to provide an indication of a person's cognitive state. The objective of this work is to use speech processing and machine learning techniques to automatically identify the stage of dementia such as mild cognitive impairment (MCI…
University of Massachusetts Boston（马萨诸塞大学波士顿分校）；Geisel School of Medicine at Dartmouth（达特茅斯学院盖泽尔医学院）
Speech pause is an effective biomarker in dementia detection. Recent deep learning models have exploited speech pauses to achieve highly accurate dementia detection, but have not exploited the interpretability of speech pauses, i.e., what and how positions and lengths of speech pauses affect the result of dementia detection. In this paper, we will study the positions and lengths of dementia-sensitive pauses using adversarial learning approaches. Specifically, we first utilize an adversarial attack approach by adding the perturbation to the speech pauses of the testing samples, aiming to reduce…
Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data
South China University of Technology（华南理工大学）；University of Georgia（佐治亚大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital（麻省总医院）
Soft-Weighted CrossEntropy Loss for Continous Alzheimer's Disease Detection
Beijing JiaoTong University（北京交通大学）；Research Center of 6G Mobile Communications（6G移动通信研究中心）；School of Cyber Science and Engineering（网络空间与工程学院）；Huazhong University of Science and Technology（华中科技大学）
Alzheimer's Disease (AD) is nowadays the most common form of dementia, and its automatic detection can help to identify symptoms at early stages, so that preventive actions can be carried out. Moreover, non-intrusive techniques based on spoken data are crucial for the development of AD automatic detection systems. In this light, this paper is presented as a contribution to the ADReSS Challenge, aiming at improving AD automatic detection from spontaneous speech. To this end, recordings from 108 participants, which are age-, gender-, and AD condition-balanced, have been used as training set to p…
Institute of Health Informatics, University of Minnesota（明尼苏达大学健康信息学研究所）；Biomedical Informatics and Medical Education, University of Washington（华盛顿大学生物医学信息学与医学教育系）；Department of Pharmaceutical Care & Health Systems, University of Minnesota（明尼苏达大学药物护理与卫生系统系）
