面向并行化TTS前端建模的无先验多尺度对比文本-音频预训练
Prior-agnostic Multi-scale Contrastive Text-Audio Pre-training for Parallelized TTS Frontend Modeling
AI总结：
针对TTS前端监督学习受标注数据不足、依赖同质文本信号限制的问题，提出两阶段TAP-FM流水线，通过无监督多尺度对比文本-音频预训练提取多粒度表征，并行完成三类前端任务，性能达SOTA。
摘要：
过去十年间，研究界持续投入大量精力开发高表现力、可控的文本转语音（text-to-speech, TTS）系统。整体而言，完整的TTS系统包含两个相互关联的组件：前端模块与后端模块。前端模块擅长从原始文本输入中提取语言学表征，后端模块则将语言学线索转换为语音。研究界日益关注前端组件的研究，认可其在TTS系统中的核心作用，前端任务涵盖文本规范化（Text Normalization, TN）、韵律边界预测（Prosody Boundary Prediction, PBP）和多音字消歧（Polyphone Disambiguation, PD）。然而，标注文本数据不足以及对同质文本信号的依赖，严重削弱了其监督学习的效果。为规避这一障碍，本文提出了一种名为TAP-FM的新型两阶段TTS前端预测流水线。具体而言，在第一学习阶段，我们提出了多尺度对比文本-音频预训练协议（Multi-scale Contrastive Text-audio Pre-training, MC-TAP），旨在通过无监督方式的多粒度对比预训练获取更丰富的特征信息。与现有预训练方法挖掘同质特征的思路不同，我们的框架能够深入挖掘全局与局部的文本-音频语义及声学表征。此外，我们精心设计了一个并行化TTS前端模型，用于在第二阶段分别执行TN、PD和PBP预测任务。最后，大量实验验证了所提方法的优越性，其性能达到了当前最优水平（state-of-the-art）。
英文摘要：
Over the past decade, a series of unflagging efforts have been dedicated to developing highly expressive and controllable text-to-speech (TTS) systems. In general, the holistic TTS comprises two interconnected components: the frontend module and the backend module. The frontend excels in capturing linguistic representations from the raw text input, while the backend module converts linguistic cues to speech. The research community has shown growing interest in the study of the frontend component, recognizing its pivotal role in text-to-speech systems, including Text Normalization (TN), Prosody Boundary Prediction (PBP), and Polyphone Disambiguation (PD). Nonetheless, the limitations posed by insufficient annotated textual data and the reliance on homogeneous text signals significantly undermine the effectiveness of its supervised learning. To evade this obstacle, a novel two-stage TTS frontend prediction pipeline, named TAP-FM, is proposed in this paper. Specifically, during the first learning phase, we present a Multi-scale Contrastive Text-audio Pre-training protocol (MC-TAP), which hammers at acquiring richer insights via multi-granularity contrastive pre-training in an unsupervised manner. Instead of mining homogeneous features in prior pre-training approaches, our framework demonstrates the ability to delve deep into both global and local text-audio semantic and acoustic representations. Furthermore, a parallelized TTS frontend model is delicately devised to execute TN, PD, and PBP prediction tasks, respectively in the second stage. Finally, extensive experiments illustrate the superiority of our proposed method, achieving state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
