Collecting and accessing a large amount of medical data is very time-consuming and laborious, not only because it is difficult to find specific patients but also because it is required to resolve the confidentiality of a patient's medical records. On the other hand, there are deep learning models, trained on easily collectible, large scale datasets such as Youtube or Wikipedia, offering useful representations. It could therefore be very advantageous to utilize the features from these pre-trained networks for handling a small amount of data at hand. In this work, we exploit various multi-modal features extracted from pre-trained networks to recognize Alzheimer's Dementia using a neural network, with a small dataset provided by the ADReSS Challenge at INTERSPEECH 2020. The challenge regards to discern patients suspicious of Alzheimer's Dementia by providing acoustic and textual data. With the multi-modal features, we modify a Convolutional Recurrent Neural Network based structure to perform classification and regression tasks simultaneously and is capable of computing conversations with variable lengths. Our test results surpass baseline's accuracy by 18.75%, and our validation result for the regression task shows the possibility of classifying 4 classes of cognitive impairment with an accuracy of 78.70%.
BITS Pilani, K.K. Birla Goa Campus（Birla技术与科学学院皮拉尼校区 K.K. Birla果阿校区）；IEMN Université de Lille（里尔大学微电子与纳米技术研究所）
Alzheimers disease is a fatal progressive brain disorder that worsens with time. It is high time we have inexpensive and quick clinical diagnostic techniques for early detection and care. In previous studies, various Machine Learning techniques and Pre-trained Deep Learning models have been used in conjunction with the extraction of various acoustic and linguistic features. Our study focuses on three models for the classification task in the ADReSS (The Alzheimers Dementia Recognition through Spontaneous Speech) 2021 Challenge. We use the well-balanced dataset provided by the ADReSS Challenge …
The University of Edinburgh（爱丁堡大学）；Carnegie Mellon University（卡内基梅隆大学）
The ADReSS Challenge at INTERSPEECH 2020 defines a shared task through which different approaches to the automated recognition of Alzheimer's dementia based on spontaneous speech can be compared. ADReSS provides researchers with a benchmark speech dataset which has been acoustically pre-processed and balanced in terms of age and gender, defining two cognitive assessment tasks, namely: the Alzheimer's speech classification task and the neuropsychological score regression task. In the Alzheimer's speech classification task, ADReSS challenge participants create models for classifying speech as de…
Swin-BERT: A Feature Fusion System designed for Speech-based Alzheimer's Dementia Detection
College of Artificial Intelligence, Dalian Maritime University（大连海事大学人工智能学院）；University of Sheffield（谢菲尔德大学）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）
Benchmarking Foundation Speech and Language Models for Alzheimer's Disease and Related Dementia Detection from Spontaneous Speech
Georgia Institute of Technology（佐治亚理工学院）；Shandong Mental Health Center（山东精神卫生中心）；University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；Kennesaw State University（肯尼索州立大学）
DementiaBank-Emotion: A Multi-Rater Emotion Annotation Corpus for Alzheimer's Disease Speech (Version 1.0)
Sue & Bill Gross School of Nursing, University of California, Irvine (UCI)（加州大学厄尔布雷特分校护理学院）；Donald Bren School of Information and Computer Sciences, UCI, USA（加州大学厄尔布雷特分校信息与计算机科学学院）；Smart Forward, Rancho Palos Verdes, USA（Smart Forward公司）；Dept. of Psychiatry and Human Behavior, UCI, USA（加州大学厄尔布雷特分校精神病学与人类行为系）；Amore Senior Living, Laguna Niguel, USA（Amore Senior Living公司）；School of Medicine, University of California, Irvine（加州大学厄尔布雷特分校医学院）；Purdue University, West Lafayette, USA（普渡大学）；The George Washington University, Washington, D.C., USA（乔治华盛顿大学）
Alzheimer's disease (AD) constitutes a neurodegenerative disease with serious consequences to peoples' everyday lives, if it is not diagnosed early since there is no available cure. Alzheimer's is the most common cause of dementia, which constitutes a general term for loss of memory. Due to the fact that dementia affects speech, existing research initiatives focus on detecting dementia from spontaneous speech. However, little work has been done regarding the conversion of speech data to Log-Mel spectrograms and Mel-frequency cepstral coefficients (MFCCs) and the usage of pretrained models. Con…
Winterlight Labs（Winterlight实验室）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（Vector人工智能研究所）；Li Ka Shing Knowledge Institute（李嘉诚知识研究院）；St Michael’s Hospital（圣迈克尔医院）；Surgical Safety Technologies Inc.（手术安全技术公司）
A significant number of studies apply acoustic and linguistic characteristics of human speech as prominent markers of dementia and depression. However, studies on discriminating depression from dementia are rare. Co-morbid depression is frequent in dementia and these clinical conditions share many overlapping symptoms, but the ability to distinguish between depression and dementia is essential as depression is often curable. In this work, we investigate the ability of clustering approaches in distinguishing between depression and dementia from human speech. We introduce a novel aggregated data…
In this paper, we exploit a Fully Convolutional Network (FCN) to analyze the audio data of spontaneous speech for dementia detection. A fully convolutional network accommodates speech samples with varying lengths, thus enabling us to analyze the speech sample without manual segmentation. Specifically, we first obtain the Mel Frequency Cepstral Coefficient (MFCC) feature map from each participant's audio data and convert the speech classification task on audio data to an image classification task on MFCC feature maps. Then, to solve the data insufficiency problem, we apply transfer learning by …
Massachusetts Institute of Technology（麻省理工学院）；BITS Pilani Goa Campus（贝拉理工学院皮拉尼果阿校区）
Reliability in Neural Networks (NNs) is crucial in safety-critical applications like healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of NNs in deployment. In this work, we propose an uncertainty-aware boosting technique for multi-modal ensembling to predict Alzheimer's Dementia Severity. The propagation of uncertainty across acoustic, cognitive, and linguistic features produces an ensemble system robust to heteroscedasticity in the data. Weighing the different modalities based on the uncertainty estimates, we experiment on the benchmark ADReSS d…
A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning
Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens（决策支持系统实验室，电气与计算机工程学院，国家技术大学雅典）
Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADRe…
Alzheimer's Dementia (AD) is an incurable, debilitating, and progressive neurodegenerative condition that affects cognitive function. Early diagnosis is important as therapeutics can delay progression and give those diagnosed vital time. Developing models that analyse spontaneous speech could eventually provide an efficient diagnostic modality for earlier diagnosis of AD. The Alzheimer's Dementia Recognition through Spontaneous Speech task offers acoustically pre-processed and balanced datasets for the classification and prediction of AD and associated phenotypes through the modelling of spont…
VisTA: Vision-Text Alignment Model with Contrastive Learning using Multimodal Data for Evidence-Driven, Reliable, and Explainable Alzheimer's Disease Diagnosis
Centre hospitalier universitaire vaudois (CHUV)（洛桑大学医院中心（CHUV））；Université de Lausanne (UNIL)（洛桑大学（UNIL））；VNU University of Engineering and Technology, Vietnam National University（越南国家大学下属越南工程技术大学）；University of Science, Vietnam National University Ho Chi Minh City（胡志明市越南国家大学下属科学大学）；Military Hospital 175（175军事医院）；School of Biomedical Engineering, International University, Vietnam National University（越南国家大学下属国际大学生物医学工程学院）
Queen Mary University of London（伦敦玛丽女王大学）；Jožef Stefan Institute（约热夫·斯特凡研究所）
We present two multimodal fusion-based deep learning models that consume ASR transcribed speech and acoustic data simultaneously to classify whether a speaker in a structured diagnostic task has Alzheimer's Disease and to what degree, evaluating the ADReSSo challenge 2021 data. Our best model, a BiLSTM with highway layers using words, word probabilities, disfluency features, pause information, and a variety of acoustic features, achieves an accuracy of 84% and RSME error prediction of 4.26 on MMSE cognitive scores. While predicting cognitive decline is more challenging, our models show improve…
University of Vigo（维戈大学）；AtlanTTic Research Center（AtlanTTic研究中心）；acceXible（acceXible公司）
Automatic detection of Alzheimer's dementia by speech processing is enhanced when features of both the acoustic waveform and the content are extracted. Audio and text transcription have been widely used in health-related tasks, as spectral and prosodic speech features, as well as semantic and linguistic content, convey information about various diseases. Hence, this paper describes the joint work of the GTM-UVIGO research group and acceXible startup to the ADDReSS challenge at INTERSPEECH 2020. The submitted systems aim to detect patterns of Alzheimer's disease from both the patient's voice an…
Freelance（自由职业者）；longterm care centre (EHPAD)（长期护理中心（EHPAD））
Sound environment is a prime source of conscious and unconscious information which allows listeners to place themselves, to communicate, to feel, to remember. The author describes the process of designing a new audio interactive apparatus for Alzheimer's care, in the context of an active multidisciplinary research project led by the author in collaboration with a longterm care centre (EHPAD) in Burgundy (France), a geriatrician, a gerontologist, psychologists and caregivers. The apparatus, named Madeleines Sonores in reference to Proust's madeleine, have provided virtual soundscapes sounding f…
We present an approach to automatic detection of Alzheimer's type dementia based on characteristics of spontaneous spoken language dialogue consisting of interviews recorded in natural settings. The proposed method employs additive logistic regression (a machine learning boosting method) on content-free features extracted from dialogical interaction to build a predictive model. The model training data consisted of 21 dialogues between patients with Alzheimer's and interviewers, and 17 dialogues between patients with other health conditions and interviewers. Features analysed included speech ra…
Linguistic and Audio Embedding-Based Machine Learning for Alzheimer's Dementia and Mild Cognitive Impairment Detection: Insights from the PROCESS Challenge
Singapore University of Technology and Design（新加坡科技设计大学）；Hochschule für Musik Detmold（音乐学院Detmold）
RWTH Aachen University（亚琛工业大学）；University of Amsterdam（阿姆斯特丹大学）
In this paper, we combined linguistic complexity and (dis)fluency features with pretrained language models for the task of Alzheimer's disease detection of the 2021 ADReSSo (Alzheimer's Dementia Recognition through Spontaneous Speech) challenge. An accuracy of 83.1% was achieved on the test set, which amounts to an improvement of 4.23% over the baseline model. Our best-performing model that integrated component models using a stacking ensemble technique performed equally well on cross-validation and test data, indicating that it is robust against overfitting.
Dementia is a growing problem as our society ages, and detection methods are often invasive and expensive. Recent deep-learning techniques can offer a faster diagnosis and have shown promising results. However, they require large amounts of labelled data which is not easily available for the task of dementia detection. One effective solution to sparse data problems is data augmentation, though the exact methods need to be selected carefully. To date, there has been no empirical study of data augmentation on Alzheimer's disease (AD) datasets for NLP and speech processing. In this work, we inves…
Institute of Health Informatics, University of Minnesota（明尼苏达大学健康信息学研究所）；Biomedical Informatics and Medical Education, University of Washington（华盛顿大学生物医学信息学与医学教育系）；Department of Pharmaceutical Care & Health Systems, University of Minnesota（明尼苏达大学药物护理与卫生系统系）
Alzheimer's Disease (AD) is nowadays the most common form of dementia, and its automatic detection can help to identify symptoms at early stages, so that preventive actions can be carried out. Moreover, non-intrusive techniques based on spoken data are crucial for the development of AD automatic detection systems. In this light, this paper is presented as a contribution to the ADReSS Challenge, aiming at improving AD automatic detection from spontaneous speech. To this end, recordings from 108 participants, which are age-, gender-, and AD condition-balanced, have been used as training set to p…
Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data
South China University of Technology（华南理工大学）；University of Georgia（佐治亚大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital（麻省总医院）
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care to delay further progression. This paper presents the development of a state-of-the-art Conformer based speech recognition system built on the DementiaBank Pitt corpus for automatic AD detection. The baseline Conformer system trained with speed perturbation and SpecAugment based data augmentation is significantly improved by incorporating a set of purposefully designed modeling features, including neural architecture search based auto-configuration of domain-specific Conformer hyper-parameters in addition to…
Soft-Weighted CrossEntropy Loss for Continous Alzheimer's Disease Detection
Beijing JiaoTong University（北京交通大学）；Research Center of 6G Mobile Communications（6G移动通信研究中心）；School of Cyber Science and Engineering（网络空间与工程学院）；Huazhong University of Science and Technology（华中科技大学）
Dementia classification from spontaneous speech using wrapper-based feature selection
Faculty of Information Technology, University of Jyväskylä（信息科技学院，于韦斯屈莱大学）；Public Health Programme, Folkhälsan Research Center（公共卫生计划，福尔哈兰研究中心）；Faculty of Sport and Health Sciences and Gerontology Research Center, University of Jyväskylä（体育与健康科学学院及老年学研究中心，于韦斯屈莱大学）；Wellbeing Services County of Central Finland（中央芬兰福祉服务县）
