Accurately extracting clinical information from speech is critical to the diagnosis and treatment of many neurological conditions. As such, there is interest in leveraging AI for automatic, objective assessments of clinical speech to facilitate diagnosis and treatment of speech disorders. We explore transfer learning using foundation models, focusing on the impact of layer selection for the downstream task of predicting pathological speech features. We find that selecting an optimal layer can greatly improve performance (~15.8% increase in balanced accuracy per feature as compared to worst layer, ~13.6% increase as compared to final layer), though the best layer varies by predicted feature and does not always generalize well to unseen data. A learned weighted sum offers comparable performance to the average best layer in-distribution (only ~1.2% lower) and had strong generalization for out-of-distribution data (only 1.5% lower than the average best layer).
Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
Intron(英创公司);BioRAMP(BioRAMP机构);Brown University(布朗大学);Indian Institute of Information Technology Allahabad(阿拉哈巴德印度信息技术学院);University of Minnesota-Twin Cities(明尼苏达大学双城分校);University of Glasgow(格拉斯哥大学);University of Florida(佛罗里达大学);Johns Hopkins University(约翰斯·霍普金斯大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Georgia Institute of Technology(佐治亚理工学院)
该工作提出 Afrispeech-Dialog 基准数据集,通过 50 段非洲口音英语对话评估 ASR 和说话人分离系统,发现口音导致性能下降超 10%,并揭示 ASR 错误对医疗摘要的影响,呼吁构建更具包容性的低资源语音数据集。
An Objective Evaluation Framework for Pathological Speech Synthesis
University of Amsterdam(阿姆斯特丹大学);Delft University of Technology(代尔夫特理工大学);Netherlands Cancer Institute(荷兰癌症研究所);Idiap Research Institute(伊迪亚普研究所);École polytechnique fédérale de Lausanne(洛桑联邦理工学院)
Pathological voice adaptation with autoencoder-based voice conversion
Delft University of Technology(代尔夫特理工大学);Universitat Politecnica de Catalunya(加泰罗尼亚理工大学);University of Amsterdam(阿姆斯特丹大学);Netherlands Cancer Institute(荷兰癌症研究所);Johns Hopkins University(约翰斯·霍普金斯大学)
Robust and Complex Approach of Pathological Speech Signal Analysis
Brno University of Technology(布尔诺理工大学);Masaryk University(马萨里克大学);Universidad Politecnica de Madrid(马德里理工大学);St. Anne’s University Hospital(圣安妮大学医院);Central European Institute of Technology(中欧理工学院)
Multi-class Detection of Pathological Speech with Latent Features: How does it perform on unseen data?
Technische Hochschule Nürnberg Georg Simon Ohm(纽伦堡乔治·西蒙·欧姆应用科学大学);Friedrich-Alexander-University Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Intel Labs(英特尔研究院)
本研究利用预训练 wav2vec 2.0 的潜在表示,结合神经网络、大间隔分类器和树提升机区分四种病理语音,在噪声和未见数据下展现出强鲁棒性与高 F1 分数。
How to Leverage Synthetic Speech for LLM-Based ASR Systems?
Idiap Research Institute(Idiap研究 institute);Universite de Toulon(里昂大学);EPFL(瑞士联邦理工学院);University of Zurich(苏黎世大学);Uniphore(Uniphore公司);Brno University of Technology(布拉格技术大学)
Perceptual implications of automatic anonymization in pathological speech
Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany. Department of Urology, Stanford University, Stanford, CA, USA. Department of Radiology, Stanford University, Stanford, CA, USA. Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany. Department of Diagnostic;Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. Institute of Radiology, University Hospital Erlangen, Erlangen, Germany. Department of Foreign Language Education, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany. Department of Otorhinolaryngology, Head;Neck Surgery, Ludwig-Maximilians-Universität München, Munich, Germany. Speech \& Language Processing Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
Robustness against the channel effect in pathological voice detection
Academia Sinica(中央研究院);University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所);Far Eastern Memorial Hospital(远东纪念医院);Yuan Ze University(元智大学);St. Michael’s Hospital(圣迈克尔医院);Surgical Safety Technologies Inc.(手术安全技术公司)
Towards Robust Assessment of Pathological Voices via Combined Low-Level Descriptors and Foundation Model Representations
Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology(国家交通大学科学与信息工程系);Research Center for Information Technology Innovation, Academia Sinica(学术院资讯科技创新研究中心);University of Palermo(巴勒莫大学);Institute of Information Science, Academia Sinica(学术院资讯科学研究所);Department of Electrical Engineering, Chung Yuan Christian University(Chung Yuan Christian University电子工程系)
The Effect of Heterogeneous Data for Alzheimer's Disease Detection from Speech
Winterlight Labs(Winterlight实验室);University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(Vector人工智能研究所);Li Ka Shing Knowledge Institute, St Michael’s Hospital(圣迈克尔医院李嘉诚知识研究所);Surgical Safety Technologies Inc.(外科安全技术公司)