Language-Agnostic Analysis of Speech Depression Detection
Saintgits College of Engineering（圣吉茨工程学院）；IIIT Hyderabad（印度信息技术研究所海得拉巴分校）；Cognitive Science Lab, IIIT Hyderabad（印度信息技术研究所海得拉巴分校认知科学实验室）；Language Technologies Research Center, IIIT Hyderabad（印度信息技术研究所海得拉巴分校语言技术研究中心）
Sona Binu, Jismi Jose, Fathima Shimna K, Alino Luke Hans, Reni K. Cherian, Starlet Ben Alex, Priyanka Srivastava, Chiranjeevi Yarra
The people with Major Depressive Disorder (MDD) exhibit the symptoms of tonal variations in their speech compared to the healthy counterparts. However, these tonal variations not only confine to the state of MDD but also on the language, which has unique tonal patterns. This work analyzes automatic speech-based depression detection across two languages, English and Malayalam, which exhibits distinctive prosodic and phonemic characteristics. We propose an approach that utilizes speech data collected along with self-reported labels from participants reading sentences from IViE corpus, in both English and Malayalam. The IViE corpus consists of five sets of sentences: simple sentences, WH-questions, questions without morphosyntactic markers, inversion questions and coordinations, that can naturally prompt speakers to speak in different tonal patterns. Convolutional Neural Networks (CNNs) are employed for detecting depression from speech. The CNN model is trained to identify acoustic features associated with depression in speech, focusing on both languages. The model's performance is evaluated on the collected dataset containing recordings from both depressed and non-depressed speakers, analyzing its effectiveness in detecting depression across the two languages. Our findings and collected data could contribute to the development of language-agnostic speech-based depression detection systems, thereby enhancing accessibility for diverse populations.
Interdisciplinary School of Doctoral Studies（跨学科博士研究院）；Department of Psychology（心理学系）
In this work, we provide an extensive part-of-speech analysis of the discourse of social media users with depression. Research in psychology revealed that depressed users tend to be self-focused, more preoccupied with themselves and ruminate more about their lives and emotions. Our work aims to make use of large-scale datasets and computational methods for a quantitative exploration of discourse. We use the publicly available depression dataset from the Early Risk Prediction on the Internet Workshop (eRisk) 2018 and extract part-of-speech features and several indices based on them. Our results…
Winterlight Labs（Winterlight实验室）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（Vector人工智能研究所）；Li Ka Shing Knowledge Institute（李嘉诚知识研究院）；St Michael’s Hospital（圣迈克尔医院）；Surgical Safety Technologies Inc.（手术安全技术公司）
A significant number of studies apply acoustic and linguistic characteristics of human speech as prominent markers of dementia and depression. However, studies on discriminating depression from dementia are rare. Co-morbid depression is frequent in dementia and these clinical conditions share many overlapping symptoms, but the ability to distinguish between depression and dementia is essential as depression is often curable. In this work, we investigate the ability of clustering approaches in distinguishing between depression and dementia from human speech. We introduce a novel aggregated data…
DepFlow: Disentangled Speech Generation to Mitigate Semantic Bias in Depression Detection
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Electrical Engineering and Telecommunications, UNSW（新南威尔士大学电信工程学院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Center for AI in Medicine (C-AIM), Lee Kong Chian School of Medicine, NTU（人工智能医学中心（C-AIM）、李光耀医学院、南洋理工大学）
Speech-Based Depressive Mood Detection in the Presence of Multiple Sclerosis: A Cross-Corpus and Cross-Lingual Study
audEERING GmbH（audEERING公司）；TUM University Hospital（技术大学医院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Hasso-Plattner Institute（哈索-普拉特纳研究所）；Inselspital, Bern University Hospital（伯尔尼大学医院）；Agile Robots（敏捷机器人公司）；Imperial College（帝国学院）
University of Toronto（多伦多大学）；Winterlight Labs（Winterlight实验室）
Traditional screening practices for anxiety and depression pose an impediment to monitoring and treating these conditions effectively. However, recent advances in NLP and speech modelling allow textual, acoustic, and hand-crafted language-based features to jointly form the basis of future mental health screening and condition detection. Speech is a rich and readily available source of insight into an individual's cognitive state and by leveraging different aspects of speech, we can develop new digital biomarkers for depression and anxiety. To this end, we propose a multi-modal system for the s…
The Relationship Between Speech Features Changes When You Get Depressed: Feature Correlations for Improving Speed and Performance of Depression Detection
University of Glasgow（格拉斯哥大学）；Università degli Studi della Campania “Luigi Vanvitelli”（坎帕尼亚大学“路易吉·万维泰利”）
Identifying depression-related topics in smartphone-collected free-response speech recordings using an automatic speech recognition system and a deep learning topic model
King's College London（伦敦国王学院）；University College London（伦敦大学学院）；South London and Maudsley NHS Foundation Trust（南伦敦和莫兹利国民保健基金会信托）；Health Data Research UK London, University College London（英国健康数据研究伦敦中心，伦敦大学学院）；University of Augsburg（奥格斯堡大学）；University of Sussex（萨塞克斯大学）；Amsterdam UMC, Vrije Universiteit（阿姆斯特丹大学医学中心，自由大学）；Parc Sanitari Sant Joan de Déu（圣胡安·德杜健康园区）；Universitat de Barcelona（巴塞罗那大学）；Janssen Research and Development LLC（杨森研发有限责任公司）；Amsterdam University Medical Centre, Vrije Universiteit（阿姆斯特丹大学医学中心，自由大学）；GGZ inGeest（GGZ inGeest精神卫生机构）；Davos Alzheimer’s Collaborative（达沃斯阿尔茨海默病协作组织）
Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction
The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada（电气与计算机工程系，多伦多大学）；Division of Engineering Science, University of Toronto, Toronto, Canada（工程科学系，多伦多大学）；Cundill Centre for Child and Youth Depression, Centre for Addiction and Mental Health, Toronto, Canada（儿童与青少年抑郁研究中心，成瘾与心理健康中心）；Department of Psychology, York University, Toronto, Canada（心理学系，约克大学）；The Hospital for Sick Children, Toronto, ON, Canada（多伦多儿童医院）；Department of Psychiatry, University of Toronto, Toronto, Canada（精神病学系，多伦多大学）
Laboratoire d’analyse statistique des textes - Statistical Analysis of Text Laboratory (LAST - SATLab)（文本统计分析实验室（LAST - SATLab））；Institut de recherche en sciences psychologiques（心理科学研究所）；Université catholique de Louvain（鲁汶天主教大学）
For automatically identifying hate speech and offensive content in tweets, a system based on a classical supervised algorithm only fed with character n-grams, and thus completely language-agnostic, is proposed by the SATLab team. After its optimization in terms of the feature weighting and the classifier parameters, it reached, in the multilingual HASOC 2021 challenge, a medium performance level in English, the language for which it is easy to develop deep learning approaches relying on many external linguistic resources, but a far better level for the two less resourced language, Hindi and Ma…
Institute for Natural Language Processing（自然语言处理研究所）；University of Stuttgart（斯图加特大学）
While neural text-to-speech systems perform remarkably well in high-resource scenarios, they cannot be applied to the majority of the over 6,000 spoken languages in the world due to a lack of appropriate training data. In this work, we use embeddings derived from articulatory vectors rather than embeddings derived from phoneme identities to learn phoneme representations that hold across languages. In conjunction with language agnostic meta learning, this enables us to fine-tune a high-quality text-to-speech model on just 30 minutes of data in a previously unseen language spoken by a previously…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Carnegie Mellon University（卡内基梅隆大学）
We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increasingly used interchangeably during fluent speech. Such CS complicates traditional speech recognition and translation, as we must recognize which language was spoken first and then apply a language-dependent recognizer and subsequent translation component to generate the desired target language output. Such a pipeline introduces latency and errors. In this paper, we eliminate the…
University of Saskatchewan（萨斯喀彻温大学）；Singapore University of Technology and Design（新加坡科技设计大学）；IIT Delhi（印度德里理工学院）；IIIT-Delhi（德里英迪拉·甘地信息技术学院）；DTU-Delhi（德里理工大学）
Hate speech in social media is a growing phenomenon, and detecting such toxic content has recently gained significant traction in the research community. Existing studies have explored fine-tuning language models (LMs) to perform hate speech detection, and these solutions have yielded significant performance. However, most of these studies are limited to detecting hate speech only in English, neglecting the bulk of hateful content that is generated in other languages, particularly in low-resource languages. Developing a classifier that captures hate speech and nuances in a low-resource languag…
Continuous speech separation using a microphone array was shown to be promising in dealing with the speech overlap problem in natural conversation transcription. This paper proposes VarArray, an array-geometry-agnostic speech separation neural network model. The proposed model is applicable to any number of microphones without retraining while leveraging the nonlinear correlation between the input channels. The proposed method adapts different elements that were proposed before separately, including transform-average-concatenate, conformer speech separation, and inter-channel phase differences…
Interpretable Symptom Vectors for Depression in a Large Language Model
Stanford University School of Medicine（斯坦福大学医学院）；Wu Tsai Neurosciences Institute, Stanford University（斯坦福大学吴蔡神经科学研究所）；Kaiser Permanente（凯撒医疗机构）；BigCommerce（BigCommerce公司）；Veterans Affairs Palo Alto Healthcare System（帕洛阿尔托退伍军人事务医疗系统）；Sierra Pacific Mental Illness, Research, Education, and Clinical Center (MIRECC)（塞拉太平洋精神疾病研究、教育和临床中心（MIRECC））
Microsoft Research India（微软印度研究院）；Microsoft Corporation（微软公司）
Recognizing code-switched speech is challenging for Automatic Speech Recognition (ASR) for a variety of reasons, including the lack of code-switched training data. Recently, we showed that monolingual ASR systems fine-tuned on code-switched data deteriorate in performance on monolingual speech recognition, which is not desirable as ASR systems deployed in multilingual scenarios should recognize both monolingual and code-switched speech with high accuracy. Our experiments indicated that this loss in performance could be mitigated by using certain strategies for fine-tuning and regularization, l…
The identification of syllables within phonetic sequences is known as syllabification. This task is thought to play an important role in natural language understanding, speech production, and the development of speech recognition systems. The concept of the syllable is cross-linguistic, though formal definitions are rarely agreed upon, even within a language. In response, data-driven syllabification methods have been developed to learn from syllabified examples. These methods often employ classical machine learning sequence labeling models. In recent years, recurrence-based neural networks hav…
Recent advancement in Generative Adversarial Networks in speech synthesis domain[3],[2] have shown, that it's possible to train GANs [8] in a reliable manner for high quality coherent waveform generation from mel-spectograms. We propose that it is possible to transfer the MelGAN's [3] robustness in learning speech features to speech enhancement and noise reduction domain without any model modification tasks. Our proposed method generalizes over multi-speaker speech dataset and is able to robustly handle unseen background noises during the inference. Also, we show that by increasing the batch s…
End-to-end Speech Translation (ST) models have several advantages such as lower latency, smaller model size, and less error compounding over conventional pipelines that combine Automatic Speech Recognition (ASR) and text Machine Translation (MT) models. However, collecting large amounts of parallel data for ST task is more difficult compared to the ASR and MT tasks. Previous studies have proposed the use of transfer learning approaches to overcome the above difficulty. These approaches benefit from weakly supervised training data, such as ASR speech-to-transcript or MT text-to-text translation…
Hate speech detection has become an important research topic within the past decade. More private corporations are needing to regulate user generated content on different platforms across the globe. In this paper, we introduce a study of multilingual hate speech classification. We compile a dataset of 11 languages and resolve different taxonomies by analyzing the combined data with binary labels: hate speech or not hate speech. Defining hate speech in a single way across different languages and datasets may erase cultural nuances to the definition, therefore, we utilize language agnostic embed…
Self-supervised learning (SSL) is seen as a very promising approach with high performance for several speech downstream tasks. Since the parameters of SSL models are generally so large that training and inference require a lot of memory and computational cost, it is desirable to produce compact SSL models without a significant performance degradation by applying compression methods such as knowledge distillation (KD). Although the KD approach is able to shrink the depth and/or width of SSL model structures, there has been little research on how varying the depth and width impacts the internal …
In this paper, we proposed to apply meta learning approach for low-resource automatic speech recognition (ASR). We formulated ASR for different languages as different tasks, and meta-learned the initialization parameters from many pretraining languages to achieve fast adaptation on unseen target language, via recently proposed model-agnostic meta learning algorithm (MAML). We evaluated the proposed approach using six languages as pretraining tasks and four languages as target tasks. Preliminary results showed that the proposed method, MetaASR, significantly outperforms the state-of-the-art mul…
