AutoSIFT：用于可控语音生成的自动风格筛选与任意风格填充

中文摘要 AI 辅助

当前最先进的文本到语音（TTS）模型在自然度和表现力方面表现出色，但对说话风格进行细粒度、解耦控制仍具有挑战性。在电影配音、游戏语音表演和视频内容生成等专业场景中，用户常需修改特定风格类别，同时保留其他风格。现有方法难以联合控制显式语义属性并保留细微的韵律细节。我们提出AutoSIFT，一个用于类别级风格编辑的可控语音生成框架。它将说话风格分解为已知的可文本描述类别和未知的残余风格，通过广义风格解缠器和任意风格填充器，在保留残余语音风格的同时替换文本指定的风格类别，实现自然、富有表现力和高度可定制的语音生成。

英文摘要

State-of-the-art text-to-speech (TTS) models achieve impressive naturalness and expressiveness, yet fine-grained, disentangled control over speaking styles remains challenging. In professional scenarios such as film dubbing, game voice acting, and video content generation, users often need to modify a specific style category, such as emotion, age, or gender, while preserving all others. Existing style-controllable TTS methods typically rely on either text-described styles or speech-reference style transfer, making it difficult to jointly control explicit semantic attributes and preserve subtle, text-undescribed prosodic details. We propose AutoSIFT, a controllable speech generation framework for category-level style editing. AutoSIFT decomposes speaking style into known text-describable categories and unknown residual styles that capture non-verbal prosody and speaker-specific nuances. It consists of a generalized Style Disentangler, which extracts category-aware style prototypes from reference speech, and an Arbitrary Style Infiller, which selectively infills unspecified style categories from the reference. By replacing only text-specified style categories while preserving residual speech-derived styles, AutoSIFT enables natural, expressive, and highly customizable speech generation.

发表机构

• UNSW Sydney（新南威尔士大学悉尼分校）
• University of California San Diego（加利福尼亚大学圣地亚哥分校）
• Macquarie University（麦考瑞大学）
• Adobe Research（Adobe 研究院）

机构由 AI 辅助整理，请以论文原文为准。