arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.15261cs.SD

非规范语音识别的两阶段适应:重新审视针对个性化 Speaker-Independent 初始化

Two-Stage Adaptation for Non-Normative Speech Recognition: Revisiting Speaker-Independent Initialization for Personalization

Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands(莱顿先进计算机科学研究所,莱顿大学,荷兰);Penobscot Community Health Care, United States(彭纳波克社区医疗服务,美国);Department of Computational Linguistics, University of Zurich, Switzerland(计算语言学系,苏黎世大学,瑞士)

Shan Jiang, Jiawen Qi, Chuanbing Huo, Yingqiang Gao, Qinyu Chen

AI总结:

本文提出两阶段适应框架,通过多说话人非规范数据进行Speaker-Independent微调,再进行Speaker-Specific微调,实验表明在AphasiaBank和UA-Speech数据集上,该方法在保持可控的领域外偏差的同时提升了个性化性能。

摘要:

对非规范语音(如构音障碍和失语症语音)进行自动语音识别(ASR)系统个性化是具有挑战性的。虽然speaker-specific微调(SS-FT)被广泛使用,但通常直接从通用预训练模型初始化。在这样的不匹配情况下,Speaker-Independent适应是否能提供更强的初始化先验仍不明确。本文提出一个两阶段适应框架,包括在多说话人非规范数据上进行Speaker-Independent微调(SI-FT),随后进行SS-FT,并通过在相同per-speaker条件下与直接SS-FT的受控比较进行评估。在AphasiaBank和UA-Speech上使用Whisper-Large-v3和Qwen3-ASR,以及在典型语音数据集TED-LIUM v3和FLEURS上的评估表明,两阶段适应在保持可控的领域外(OOD)偏差的同时,一致提升了个性化性能。

英文摘要:

Personalizing automatic speech recognition (ASR) systems for non-normative speech, such as dysarthric and aphasic speech, is challenging. While speaker-specific fine-tuning (SS-FT) is widely used, it is typically initialized directly from a generic pre-trained model. Whether speaker-independent adaptation provides a stronger initialization prior under such mismatch remains unclear. In this work, we propose a two-stage adaptation framework consisting of speaker-independent fine-tuning (SI-FT) on multi-speaker non-normative data followed by SS-FT, and evaluate it through a controlled comparison with direct SS-FT under identical per-speaker conditions. Experiments on AphasiaBank and UA-Speech with Whisper-Large-v3 and Qwen3-ASR, alongside evaluation on typical-speech datasets TED-LIUM v3 and FLEURS, show that two-stage adaptation consistently improves personalization while maintaining manageable out-of-domain (OOD) trade-offs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑