arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01796eess.AScs.CLcs.LG
收藏

医疗领域的语音基础模型:层选择对病理性语音特征预测的影响

Speech foundation models in healthcare: Effect of layer selection on pathological speech feature prediction

Mayo Clinic(梅奥诊所)

Daniela A. Wiepert, Rene L. Utianski, Joseph R. Duffy, John L. Stricker, Leland R. Barnard, David T. Jones, Hugo Botha

AI总结:

本研究探索语音基础模型迁移学习中层选择对病理性语音特征预测的影响,发现最优层可显著提升性能但泛化性有限,而学习加权和方法兼顾了分布内性能与分布外泛化能力。

摘要:

从语音中准确提取临床信息对许多神经系统疾病的诊断和治疗至关重要。因此,研究者们有兴趣利用AI对临床语音进行自动、客观的评估,以促进言语障碍的诊断和治疗。我们探索了使用基础模型的迁移学习,重点研究层选择对预测病理性语音特征这一下游任务的影响。我们发现,选择最优层可以大幅提升性能(与最差层相比,每个特征的平衡准确率提升约15.8%,与最后一层相比提升约13.6%),不过最佳层因预测特征而异,且并非总能很好地泛化到未见过的数据。一种学习得到的加权和方法在分布内数据上的性能与平均最佳层相当(仅低约1.2%),并且对分布外数据具有很强的泛化能力(仅比平均最佳层低1.5%)。

英文摘要:

Accurately extracting clinical information from speech is critical to the diagnosis and treatment of many neurological conditions. As such, there is interest in leveraging AI for automatic, objective assessments of clinical speech to facilitate diagnosis and treatment of speech disorders. We explore transfer learning using foundation models, focusing on the impact of layer selection for the downstream task of predicting pathological speech features. We find that selecting an optimal layer can greatly improve performance (~15.8% increase in balanced accuracy per feature as compared to worst layer, ~13.6% increase as compared to final layer), though the best layer varies by predicted feature and does not always generalize well to unseen data. A learned weighted sum offers comparable performance to the average best layer in-distribution (only ~1.2% lower) and had strong generalization for out-of-distribution data (only 1.5% lower than the average best layer).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑