PSLM：利用大语言模型并行生成文本与语音以实现低延迟口语对话系统
PSLM: Parallel Generation of Text and Speech with LLMs for Low-Latency Spoken Dialogue Systems
AI总结：
本文提出PSLM，通过扩展大语言模型输入输出序列实现文本与语音并行生成，在口语问答中降低延迟并保持内容质量，并进一步利用多序列语音生成减少延迟。
摘要：
同时处理文本和语音的多模态语言模型在口语对话系统中具有应用潜力。然而，当前模型在响应生成延迟方面面临两大挑战：（1）生成语音响应需要先生成书面响应；（2）语音序列明显长于文本序列。本研究通过扩展语言模型的输入和输出序列以支持文本与语音的并行生成来解决这些问题。我们在口语问答任务上的实验表明，该方法在保持响应内容质量的同时改善了延迟。此外，我们还证明通过以多个序列生成语音可以进一步降低延迟。演示样例可在 https://rinnakk.github.io/research/publications/PSLM 获取。
英文摘要：
Multimodal language models that process both text and speech have a potential for applications in spoken dialogue systems. However, current models face two major challenges in response generation latency: (1) generating a spoken response requires the prior generation of a written response, and (2) speech sequences are significantly longer than text sequences. This study addresses these issues by extending the input and output sequences of the language model to support the parallel generation of text and speech. Our experiments on spoken question answering tasks demonstrate that our approach improves latency while maintaining the quality of response content. Additionally, we show that latency can be further reduced by generating speech in multiple sequences. Demo samples are available at https://rinnakk.github.io/research/publications/PSLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
