arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2510.09592cs.CL

有意识说话:一种双脑方法用于语音语言模型中的实时推理

Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in Spoken Language Models

StepFun;Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学)

Donghang Wu, Haoyang Zhang, Jun Chen, Xiangyu, Zhang, Hexin Liu, Eng Siong Chng, Fei Tian, Xuerui Yang, Xiangyu Zhang, Daxin Jiang, Gang Yu

AI总结:

本文提出Mind-Paced Speaking方法,通过双脑架构实现语音语言模型的实时推理,显著提升推理性能并降低延迟。

摘要:

实时语音语言模型(SLMs)在利用链式思维(CoT)推理时面临生成整个推理过程的高延迟问题。本文首次提出Mind-Paced Speaking(MPS)方法,受大脑启发,采用双脑架构,通过'构思大脑'进行高层次推理以引导'表达大脑'生成流畅语音。该方法消除了模式切换,保持推理过程的完整性。实验表明,MPS显著优于现有的思考-说话方法,并在数学推理任务Spoken-MQA上达到92.8%的准确率,在语音对话任务URO-Bench上获得82.5分。MPS是Step-Audio R1.1系统的核心方法,有效弥合了高质量推理与实时交互之间的差距。

英文摘要:

Real-time Spoken Language Models (SLMs) struggle to leverage Chain-of-Thought (CoT) reasoning due to the prohibitive latency of generating the entire thought process sequentially. Enabling SLMs to think while speaking, similar to humans, is attracting increasing attention. We present, for the first time, Mind-Paced Speaking (MPS), a brain-inspired framework that enables high-fidelity, real-time reasoning. Similar to how humans utilize distinct brain regions for thinking and responding, we propose a novel dual-brain approach, employing a "Formulation Brain" for high-level reasoning to pace and guide a separate "Articulation Brain" for fluent speech generation. This division of labor eliminates mode-switching, preserving the integrity of the reasoning process. Experiments show that MPS significantly outperforms existing think-while-speaking methods and achieves reasoning performance comparable to models that pre-compute the full CoT before speaking, while drastically reducing latency. Under a zero-latency configuration, the proposed method achieves an accuracy of 92.8% on the mathematical reasoning task Spoken-MQA and attains a score of 82.5 on the speech conversation task URO-Bench. MPS is the methodology underlying our released Step-Audio R1.1 system, effectively bridging the gap between high-quality reasoning and real-time interaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑