arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.08233cs.SDcs.AI
收藏

Tutti:通过结构级音色控制和语音纹理建模实现表达性多声部合成

Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling

Wuhan University of Technology, School of Computer Science and Artificial Intelligence(武汉理工大学计算机科学与人工智能学院)

Jiatao Chen, Xing Tang, Xiaoyue Duan, Yutang Feng, Jinchao Zhang, Jie Zhou

AI总结:

Tutti通过结构级音色控制和语音纹理建模,实现了多声部合成的高精度调度和声学真实性的提升。

摘要:

尽管现有的歌唱语音合成系统能够实现高保真的独唱表演,但它们受到全局音色控制的限制,无法解决动态多声部编排和歌曲中的语音纹理问题。为此,我们提出了Tutti,一个统一的框架,用于结构化的多声部生成。具体来说,我们引入了结构感知的歌手提示,以实现随着音乐结构演变的灵活歌手调度,并提出了通过条件引导的VAE进行互补纹理学习,以捕捉隐含的声学纹理(例如空间混响和频谱融合),这些纹理与显式控制互补。实验表明,Tutti在精确的多声部调度和显著提高合唱生成的声学真实性方面表现出色,为复杂的多声部编排提供了新的范式。音频样本可在https://annoauth123-ctrl.github.io/Tutii_Demo/上获取。

英文摘要:

While existing Singing Voice Synthesis systems achieve high-fidelity solo performances, they are constrained by global timbre control, failing to address dynamic multi-singer arrangement and vocal texture within a single song. To address this, we propose Tutti, a unified framework designed for structured multi-singer generation. Specifically, we introduce a Structure-Aware Singer Prompt to enable flexible singer scheduling evolving with musical structure, and propose Complementary Texture Learning via Condition-Guided VAE to capture implicit acoustic textures (e.g., spatial reverberation and spectral fusion) that are complementary to explicit controls. Experiments demonstrate that Tutti excels in precise multi-singer scheduling and significantly enhances the acoustic realism of choral generation, offering a novel paradigm for complex multi-singer arrangement. Audio samples are available at https://annoauth123-ctrl.github.io/Tutii_Demo/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑