发表机构
Nagoya University; National Institute of Informatics; Keio University; Waseda University（名古屋大学; 信息学研究所; 庆应义塾大学; 早稻田大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对全双工对话中重叠等同时现象，提出KABURI-TTS，以每说话人音素栅格为输入，在语音活动条件下双声道渲染语音，实现可控双人对话生成，提升自然度并产生更多重叠与轮换。
AI 中文摘要
实现全双工口语对话需要大量双声道、每声道一位说话人的对话语音数据。尽管会话式文本到语音（TTS）引擎已被开发出来，但它们对于对话者在说话时发生的诸如反馈、打断和重叠等双方同时发生的现象并不一定具有鲁棒性。在这项工作中，为了生成能够再现类似人类重叠的会话语音合成，我们提出了KABURI-TTS。KABURI-TTS将每位说话人的音素栅格作为输入，并在由每帧音素及其衍生的语音活动作为条件下，在两个独立声道上渲染两位说话人的语音。由于音素栅格由单独模块提供，所提方法能够实现可控的每声道一位说话人的双人口语对话生成。用户评估表明，与强基线相比，所提方法在话语层面和交互层面均获得了更高的自然度。此外，对语音活动的分析证实，所提方法产生了更多的重叠和更频繁的轮换。
英文摘要
Realizing full-duplex spoken dialogue requires large amounts of two-channel, one-speaker-per-channel conversational speech data. Although conversational text-to-speech (TTS) engines have been developed, they are not necessarily robust to two-party simultaneous phenomena such as backchannels, interruptions, and overlaps that occur while the interlocutor is speaking. In this work, aiming at conversational speech synthesis that reproduces human-like overlap, we propose KABURI-TTS. KABURI-TTS takes a per-speaker phoneme raster as input and renders the speech of the two speakers on separate channels, conditioned on the per-frame phonemes and the voice activity derived from them. Because the phoneme raster is supplied by a separate module, the proposed method enables controllable generation of one-speaker-per-channel, two-party spoken dialogue. A user evaluation shows that, compared with strong baselines, the proposed method attains higher naturalness at both the utterance and the interaction level. Furthermore, an analysis of voice activity confirms that the proposed method produces more overlap and more frequent turn-taking.
CommentsAccepted at APSIPA ASC 2026
