基于书面对话实现AI智能体之间类人口语对话生成
Towards human-like spoken dialogue generation between AI agents from written dialogue
AI总结：
针对从书面对话生成类人口语对话的难题，研究提出基于离散token的CHATS系统，可同时生成双方语音、实现自然话轮转换，效果优于基线，对话更互动流畅且清晰可懂。
摘要：
大语言模型（LLMs）的出现已使生成两个智能体之间的自然书面对话成为可能。然而，从这些书面对话生成类人口语对话仍具挑战性。口语对话有若干独特特征：它们频繁包含反馈语（backchannels）和笑声，且话轮转换的流畅度显著影响对话的流畅性。本研究提出CHATS——CHatty Agents Text-to-Speech（健谈智能体文本转语音），这是一种基于离散token的系统，旨在基于书面对话生成口语对话。我们的系统可仅使用说话方的转录文本，同时为说话方和倾听方生成语音，无需反馈语或笑声的转录文本。此外，CHATS可促进自然的话轮转换：在无重叠的情况下，它确定每段话语后合适的沉默时长；在有重叠的情况下，它基于下一段话语的音素序列启动重叠语音的生成。实验评估表明，CHATS优于文本转语音基线，生成的口语对话更具互动性和流畅性，同时保持了清晰度和可懂度。
英文摘要：
The advent of large language models (LLMs) has made it possible to generate natural written dialogues between two agents. However, generating human-like spoken dialogues from these written dialogues remains challenging. Spoken dialogues have several unique characteristics: they frequently include backchannels and laughter, and the smoothness of turn-taking significantly influences the fluidity of conversation. This study proposes CHATS - CHatty Agents Text-to-Speech - a discrete token-based system designed to generate spoken dialogues based on written dialogues. Our system can generate speech for both the speaker side and the listener side simultaneously, using only the transcription from the speaker side, which eliminates the need for transcriptions of backchannels or laughter. Moreover, CHATS facilitates natural turn-taking; it determines the appropriate duration of silence after each utterance in the absence of overlap, and it initiates the generation of overlapping speech based on the phoneme sequence of the next utterance in case of overlap. Experimental evaluations indicate that CHATS outperforms the text-to-speech baseline, producing spoken dialogues that are more interactive and fluid while retaining clarity and intelligibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
