arXiv 2602.043072026-03-01eess.AScs.CLcs.LGcs.SD

通用鲁棒语音适应用于跨域语音识别与增强

Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement

Department of Computer Science and Information Engineering, National Taiwan Normal University（计算机科学与信息工程系，台湾国立正常大学）；United Link Co., Ltd.（联合链接有限公司）；Institute of Information Science, Academia Sinica（学术院信息科学研究所）

Chien-Chun Wang, Hung-Shin Lee, Hsin-Min Wang, Berlin Chen

备注：: Accepted to IEEE Transactions on Audio, Speech and Language Processing (IEEE TASLP)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

URSA-GAN通过双嵌入架构和动态随机扰动技术，提升跨域语音识别与增强的鲁棒性和泛化能力。

摘要：

预训练模型在自动语音识别（ASR）和语音增强（SE）中表现出色，但在匹配噪声和信道条件下表现优异。然而，这些模型在面对领域偏移时往往会出现严重的性能下降，特别是在存在未见过的噪声和信道畸变的情况下。为此，本文提出了URSA-GAN，一种统一且领域感知的生成框架，专门设计用于减轻噪声和信道条件下的不匹配问题。URSA-GAN利用双嵌入架构，包括一个噪声编码器和一个信道编码器，每个都用有限的领域内数据预训练，以捕捉领域相关的表示。这些嵌入条件一个基于GAN的语音生成器，促进合成在声学上与目标领域对齐的语音，同时保持语音内容。为了进一步增强泛化能力，我们提出动态随机扰动，一种新的正则化技术，它在生成过程中引入受控的变异性，促进对未见过的领域的鲁棒性。实验证明，URSA-GAN在ASR中有效降低了字符错误率，并在SE中改进了感知指标，跨多样化的嘈杂和不匹配信道场景。值得注意的是，在具有信道和噪声退化复合测试条件的评估中，URSA-GAN的泛化能力得到验证，其在ASR性能上实现了16.16%的相对提升，在SE指标上实现了15.58%的提升。

英文摘要：

Pre-trained models for automatic speech recognition (ASR) and speech enhancement (SE) have exhibited remarkable capabilities under matched noise and channel conditions. However, these models often suffer from severe performance degradation when confronted with domain shifts, particularly in the presence of unseen noise and channel distortions. In view of this, we in this paper present URSA-GAN, a unified and domain-aware generative framework specifically designed to mitigate mismatches in both noise and channel conditions. URSA-GAN leverages a dual-embedding architecture that consists of a noise encoder and a channel encoder, each pre-trained with limited in-domain data to capture domain-relevant representations. These embeddings condition a GAN-based speech generator, facilitating the synthesis of speech that is acoustically aligned with the target domain while preserving phonetic content. To enhance generalization further, we propose dynamic stochastic perturbation, a novel regularization technique that introduces controlled variability into the embeddings during generation, promoting robustness to unseen domains. Empirical results demonstrate that URSA-GAN effectively reduces character error rates in ASR and improves perceptual metrics in SE across diverse noisy and mismatched channel scenarios. Notably, evaluations on compound test conditions with both channel and noise degradations confirm the generalization ability of URSA-GAN, yielding relative improvements of 16.16% in ASR performance and 15.58% in SE metrics.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。