arXiv 2602.086962026-09-15cs.SDcs.CL

基于原型的解耦方法用于可控性口吃语音合成

Controllable Dysarthric Speech Synthesis with Patient-Specific Conditioning for Speaker-Diverse ASR Augmentation

Department of Language Science and Technology, The Hong Kong Polytechnic University（语言科学与技术系，香港理工大学）；College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine（康复医学学院，福建中医药大学）；Department of Imaging, Rehabilitation Hospital affiliated to Fujian University of Traditional Chinese Medicine（影像科，福建中医药大学附属康复医院）

Haoshen Wang, Xueli Zhong, Bingbing Lin, Jia Huang, Xingduo Pan, Shengxiang Liang, Nizhuan Wang, Wai Ting Siok

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出ProtoDisent-TTS框架，通过原型解耦技术实现健康与口吃语音的可控转换，提升语音合成的可控性和鲁棒性。

摘要：

口吃语音表现出高度的变异性以及有限的标注数据，给自动语音识别（ASR）和辅助语音技术带来了重大挑战。现有方法依赖于合成数据增强或语音重建，但往往将说话人身份与病理语音融合，限制了可控性和鲁棒性。本文提出ProtoDisent-TTS，一种基于原型的解耦TTS框架，该框架基于预训练的文本到语音骨干网络，在统一的潜在空间中分解说话人音色和口吃语音。一个病理原型代码本提供了可解释且可控的健康与口吃语音模式表示，而双重分类器目标与梯度反转层强制说话人嵌入对病理属性的不变性。在TORGO数据集上的实验表明，这种设计使健康语音与口吃语音之间能够实现双向转换，从而实现一致的ASR性能提升和鲁棒、说话人感知的语音重建。

英文摘要：

Dysarthric speech recognition is limited by high speaker variability and scarce labeled data. Existing synthesis methods often couple speaker identity with dysarthric articulation, reducing control over generated speech. We propose a controllable dysarthric speech synthesis framework for ASR augmentation with separate prompt-derived timbre prefixes and learnable patient-specific pathology prefixes. Built on a pre-trained neural codec language model adapted using LoRA, the framework combines both prefixes through additive conditioning. A dual-classifier objective with gradient reversal and voice-conversion-based counterfactual augmentation promotes factor separation, allowing learned pathology conditions to be paired with different target speakers, including healthy speakers. Experiments on TORGO show that generated speech can partially replace real dysarthric training data and provides effective speaker-diverse augmentation when combined with real data. Objective ASR, perceptual, factor-separation, and phoneme-level analyses indicate preservation of target-speaker timbre and pathology-dependent patterns consistent with real dysarthric speech.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。