RelayS2S: 一种双路径推测生成用于实时对话
RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue
AI总结：
RelayS2S通过双路径架构平衡实时对话中的延迟与响应质量，利用快速路径生成低延迟音频起始，慢路径生成高质量延续，通过轻量验证器实现无缝切换。
摘要：
实时语音对话系统面临延迟与响应质量之间的根本矛盾。端到端语音到语音（S2S）模型能立即响应并自然处理轮流发言、背通道和打断，但生成的语义较弱。级联流水线（ASR -> LLM）虽然响应质量更高，但延迟随模型规模增加而增长。我们提出了RelayS2S，一种混合架构，在检测到轮次时并行运行两条路径。快速路径——一个双工S2S模型——推测性地生成短响应前缀，立即流式传输到TTS以实现低延迟音频起始，同时继续监控实时音频事件。慢路径——一个级联的ASR -> LLM流水线——根据已承诺的前缀生成高质量的延续，产生无缝的发言。一个轻量级学习验证器控制切换，适当时承诺前缀或优雅回退到仅慢路径。实验表明，RelayS2S在P90起始延迟方面与S2S模型相当，同时在平均得分上保留99%的级联响应质量，随着慢路径模型的扩展，优势增长。由于前缀切换不需要对任一组件进行架构修改，RelayS2S作为现有级联流水线的轻量级补充。我们的代码和数据可在：https://github.com/mailong25/relays2s 公开获取。
英文摘要：
Real-time spoken dialogue systems face a fundamental tension between latency and response quality. End-to-end speech-to-speech (S2S) models respond immediately and naturally handle turn-taking, backchanneling, and interruption, but produce semantically weaker outputs. Cascaded pipelines (ASR -> LLM) deliver stronger responses at the cost of latency that grows with model size. We present RelayS2S, a hybrid architecture that runs two paths in parallel upon turn detection. The fast path - a duplex S2S model - speculatively drafts a short response prefix that is streamed immediately to TTS for low-latency response onset, while continuing to monitor live audio events. The slow path - a cascaded ASR -> LLM pipeline - generates a higher-quality continuation conditioned on the committed prefix, producing an uninterrupted utterance. A lightweight learned verifier gates the handoff, committing the prefix when appropriate or falling back gracefully to the cascaded pipeline. With GPT-4.1 as the back-end, RelayS2S substantially reduces response latency while preserving nearly all of the cascaded pipeline's textual quality. On synthetic voice dialogues, it achieves a P90 first-chunk latency of 81 ms, excluding TTS and network latency, compared with 1,006 ms for the cascaded baseline. On real voice dialogues, RelayS2S reduces average first-chunk latency by 479 ms while retaining 99% of the cascaded pipeline's textual quality. These benefits become larger as the slow-path model scales. Because the prefix handoff requires no architectural modification to either component, RelayS2S serves as a lightweight, drop-in addition to existing cascaded pipelines. Our code is publicly available at: https://github.com/mailong25/relays2s
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
