arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.19798cs.SDcs.CLeess.AS
收藏

无边界的长语音合成

Borderless Long Speech Synthesis

MiLM Plus, Xiaomi Inc., China(小米公司MiLM Plus,中国);Nanjing University, China(南京大学,中国);WeNet Open Source Community(WeNet开源社区)

Xingchen Song, Di Wu, Dinghao Zhou, Pengyu Cheng, Hongwu Ding, Yunchao He, Jie Wang, Shengfan Shen, Sixiang Lv, Lichun Fan, Hang Su, Yifeng Wang, Shuai Wang, Meng Meng, Jian Luan

AI总结:

本文提出无边界的长语音合成框架,通过统一能力集实现多说话人合成和长文本生成,采用全局-句子-token注释策略和连续分词器提升指令遵循能力,构建分层控制协议栈实现多模态输入到结构化生成命令的转换。

摘要:

现有文本到语音系统要么逐句合成再拼接,要么仅基于纯文本对话驱动合成。这两种方法使模型难以理解全局上下文或非语言线索,难以捕捉现实中的多说话人交互(打断、重叠)、情感变化和多样的声音环境。我们引入无边界的长语音合成框架,面向以代理为中心的无边界长音频合成。该系统不是针对单一狭窄任务,而是设计为涵盖VoiceDesigner、多说话人合成、Instruct TTS和长文本生成的统一能力集。在数据方面,我们提出

英文摘要:

Most existing text-to-speech (TTS) systems either synthesize speech sentence by sentence and stitch the results together, or drive synthesis from plain-text dialogues alone. Both approaches leave models with little understanding of global context or paralinguistic cues, making it hard to capture real-world phenomena such as multi-speaker interactions (interruptions, overlapping speech), evolving emotional arcs, and varied acoustic environments. We introduce the Borderless Long Speech Synthesis framework for agent-centric, borderless long audio synthesis. Rather than targeting a single narrow task, the system is designed as a unified capability set spanning VoiceDesigner, multi-speaker synthesis, Instruct TTS, and long-form text synthesis. On the data side, we propose a "Labeling over filtering/cleaning" strategy and design a top-down, multi-level annotation schema we call Global-Sentence-Token. On the model side, we adopt a backbone with a continuous tokenizer and add Chain-of-Thought (CoT) reasoning together with Dimension Dropout, both of which markedly improve instruction following under complex conditions. We further show that the system is Native Agentic by design: the hierarchical annotation doubles as a Structured Semantic Interface between the LLM Agent and the synthesis engine, creating a layered control protocol stack that spans from scene semantics down to phonetic detail. Text thereby becomes an information-complete, wide-band control channel, enabling a front-end LLM to convert inputs of any modality into structured generation commands, extending the paradigm from Text2Speech to borderless long speech synthesis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑