arXiv 2602.232662026-02-26cs.CL

面向 discourse 的双轨流式响应用于低延迟语音对话系统

Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Department of Language Science and Technology, Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology（深圳先进技术大学人工智能研究院）；School of Data Science, Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；School of Artificial Intelligence, Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）

Siyuan Liu, Jiahui Xu, Feng Jiang, Kuang Wang, Zefeng Zhao, Chu-Ren Huang, Jinghang Gu, Changqing Yin, Haizhou Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

DDTSR 提出了一种低延迟语音对话系统框架，通过双轨流式响应机制实现听中思考和说中思考，显著降低响应延迟并保持 discourse 质量。

摘要：

实现类人响应是级联语音对话系统的关键但具有挑战性的目标。传统 ASR-LLM-TTS 管道遵循严格顺序范式，要求完成转录和完整推理后才能开始语音合成，这导致高响应延迟。我们提出了面向 discourse 的双轨流式响应（DDTSR）框架，一种低延迟架构，能够实现听中思考和说中思考。DDTSR 基于三个关键机制：（1）连接词引导的小大模型协同，其中辅助小模型生成最小承诺的 discourse 连接词，而大模型并行进行知识密集型推理；（2）基于流式的跨模态协作，动态重叠 ASR、LLM 推理和 TTS 以推进最早的可说话时刻；（3）基于课程学习的 discourse 连续性增强，维持早期响应与后续推理输出之间的连贯性和逻辑一致性。在两个语音对话基准测试中，DDTSR 将响应延迟降低了 19%-51%，同时保持 discourse 质量。进一步分析显示，DDTSR 作为插件式模块，可与多种 LLM 后端兼容，并在不同 utterance 长度下保持稳健，表明其在实时语音交互中的强大实用性和可扩展性。

英文摘要：

Achieving human-like responsiveness is a critical yet challenging goal for cascaded spoken dialogue systems. Conventional ASR-LLM-TTS pipelines follow a strictly sequential paradigm, requiring complete transcription and full reasoning before speech synthesis can begin, which results in high response latency. We propose the Discourse-Aware Dual-Track Streaming Response (DDTSR) framework, a low-latency architecture that enables listen-while-thinking and speak-while-thinking. DDTSR is built upon three key mechanisms: (1) connective-guided small-large model synergy, where an auxiliary small model generates minimal-committal discourse connectives while a large model performs knowledge-intensive reasoning in parallel; (2) streaming-based cross-modal collaboration, which dynamically overlaps ASR, LLM inference, and TTS to advance the earliest speakable moment; and (3) curriculum-learning-based discourse continuity enhancement, which maintains coherence and logical consistency between early responses and subsequent reasoning outputs. Experiments on two spoken dialogue benchmarks demonstrate that DDTSR reduces response latency by 19%-51% while preserving discourse quality. Further analysis shows that DDTSR functions as a plug-and-play module compatible with diverse LLM backbones, and remains robust across varying utterance lengths, indicating strong practicality and scalability for real-time spoken interaction.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。