arXiv 2607.039282026-07-07cs.SDcs.AIeess.AS

TokAN：使用自监督语音令牌进行口音归一化

TokAN: Accent Normalization Using Self-Supervised Speech Tokens

School of Data Science (SDS), The Chinese University of Hong Kong, Shenzhen (CUHKSZ)（香港中文大学（深圳）数据科学学院）；Tencent Ethereal Audio Lab, Tencent（腾讯虚幻音频实验室）；School of Intelligence Science and Technology, Nanjing University, Suzhou（南京大学苏州校区智能科学与技术学院）；Shenzhen Loop Area Institute（深圳河套学院）；X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学与工程学院X-LANCE实验室）；School of Artificial Intelligence (SAI), CUHKSZ（香港中文大学（深圳）人工智能学院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）

Qibing Bai, Shuai Wang, Yuhan Du, Bohan Li, Yannan Wang, Haizhou Li

备注：: Submitted to IEEE Transactions on Audio, Speech, and Language Processing (TASLP)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究提出TokAN框架，基于自监督离散语音令牌，用自回归编解码器转换口音，引入强化学习后训练，还用流匹配合成器和持续时间预测器，实验表明其在降低字错误率和提升可懂度上优于基线。

摘要：

口音归一化（AN）旨在将非母语（L2）口音语音转换为标准（L1）语音并保留说话者身份。当前技术存在局限，本文提出TokAN，基于自监督离散语音令牌，通过自回归编码器 - 解码器模型转换口音，引入强化学习后训练，还有流匹配合成器和持续时间预测器，实验证明其性能优越。

英文摘要：

Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity. The current techniques either require naturally recorded parallel L1-L2 speech for training, or suffer from quality degradation when supervised by synthesized targets. In this paper, we present TokAN, a token-based accent normalization framework that operates on self-supervised discrete speech tokens extracted from a L1-L2 jointly trained vector-quantization (VQ) tokenizer, without the need of synthetic supervisory speech. An autoregressive encoder-decoder model performs token-to-token conversion, translating L2-accented token sequences into the tokens of standard voice. We also introduce reinforcement learning (RL) post-training based on Group Relative Policy Optimization (GRPO), using word error rate and accent classifier confidence as complementary rewards. A non-autoregressive flow-matching synthesizer recovers the Mel-spectrogram from the converted tokens, conditioned on the source speaker embedding. We also develop a flow-matching duration predictor that supports total-duration-aware synthesis, making TokAN applicable to duration-critical tasks such as voice dubbing and live casting. Experiments on seven English accents demonstrate that TokAN reduced the word error rate from 12.40% to 9.89% after supervised fine-tuning, and further to 9.23% after RL post-training, consistently outperforming frame-to-frame, direct flow-matching, and prompt-based token-conversion baselines in terms of accent reduction and intelligibility.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。