非规范语音识别的两阶段适应:重新审视针对个性化 Speaker-Independent 初始化
Two-Stage Adaptation for Non-Normative Speech Recognition: Revisiting Speaker-Independent Initialization for Personalization
AI总结:
本文提出两阶段适应框架,通过多说话人非规范数据进行Speaker-Independent微调,再进行Speaker-Specific微调,实验表明在AphasiaBank和UA-Speech数据集上,该方法在保持可控的领域外偏差的同时提升了个性化性能。
摘要:
对非规范语音(如构音障碍和失语症语音)进行自动语音识别(ASR)系统个性化是具有挑战性的。虽然speaker-specific微调(SS-FT)被广泛使用,但通常直接从通用预训练模型初始化。在这样的不匹配情况下,Speaker-Independent适应是否能提供更强的初始化先验仍不明确。本文提出一个两阶段适应框架,包括在多说话人非规范数据上进行Speaker-Independent微调(SI-FT),随后进行SS-FT,并通过在相同per-speaker条件下与直接SS-FT的受控比较进行评估。在AphasiaBank和UA-Speech上使用Whisper-Large-v3和Qwen3-ASR,以及在典型语音数据集TED-LIUM v3和FLEURS上的评估表明,两阶段适应在保持可控的领域外(OOD)偏差的同时,一致提升了个性化性能。
英文摘要:
Personalizing automatic speech recognition (ASR) systems for non-normative speech, such as dysarthric and aphasic speech, is challenging. While speaker-specific fine-tuning (SS-FT) is widely used, it is typically initialized directly from a generic pre-trained model. Whether speaker-independent adaptation provides a stronger initialization prior under such mismatch remains unclear. In this work, we propose a two-stage adaptation framework consisting of speaker-independent fine-tuning (SI-FT) on multi-speaker non-normative data followed by SS-FT, and evaluate it through a controlled comparison with direct SS-FT under identical per-speaker conditions. Experiments on AphasiaBank and UA-Speech with Whisper-Large-v3 and Qwen3-ASR, alongside evaluation on typical-speech datasets TED-LIUM v3 and FLEURS, show that two-stage adaptation consistently improves personalization while maintaining manageable out-of-domain (OOD) trade-offs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。