Mini-Omni-Reasoner：大型语音模型中的词元级边说边思考
Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models
AI总结：
针对大型语音模型顺序式推理导致响应延迟的问题，提出Mini-Omni-Reasoner框架，在词元级交错静默推理与语音响应，并构建Spoken-Math-Problems-3M数据集，在Spoken-MQA上实现算术推理+19.1%、上下文理解+6.4%的提升且零解码延迟。
摘要：
推理对于有效沟通和决策至关重要。尽管大语言模型（LLMs）和多模态大语言模型（MLLMs）的最新进展表明，引入显式推理能显著提升理解与泛化能力，但大型语音模型（LSMs）中的推理仍处于起步阶段。早期工作尝试将文本模型中的“先思考后说话”范式迁移到语音领域。然而，这种顺序式表述引入了明显的延迟，因为语音响应必须等到推理完全完成后才能开始，从而损害了实时交互和通信效率。为解决这一问题，我们提出了 Mini-Omni-Reasoner，这是一个通过新颖的“边说边思考”表述在语音内部实现推理的框架。Mini-Omni-Reasoner 并非在产生任何语言输出之前完成推理，而是在词元级别将静默推理词元与语音响应词元交错排列。这种设计允许连续生成语音，同时嵌入结构化的内部推理，充分利用模型的高频词元处理能力。尽管采用交错方式，仍强制实施局部语义对齐，以确保每个响应词元都受到其前置推理的指导。为支持该框架，我们引入了 Spoken-Math-Problems-3M，这是一个专为交错推理与响应而定制的大规模数据集。该数据集确保语音词元始终跟随相关推理内容，从而实现语音耦合推理的准确高效学习。Mini-Omni-Reasoner 建立在分层的 Thinker-Talker 架构之上，能够输出流畅且逻辑有据的语音响应，同时保持自然性和精确性。在 Spoken-MQA 基准上，它在算术推理上取得了 +19.1% 的提升，在上下文理解上取得了 +6.4% 的提升，同时输出更短且解码延迟为零。
英文摘要：
Reasoning is essential for effective communication and decision-making. While recent advances in LLMs and MLLMs have shown that incorporating explicit reasoning significantly improves understanding and generalization, reasoning in LSMs remains in a nascent stage. Early efforts attempt to transfer the "Thinking-before-Speaking" paradigm from textual models to speech. However, this sequential formulation introduces notable latency, as spoken responses are delayed until reasoning is fully completed, impairing real-time interaction and communication efficiency. To address this, we propose Mini-Omni-Reasoner, a framework that enables reasoning within speech via a novel "Thinking-in-Speaking" formulation. Rather than completing reasoning before producing any verbal output, Mini-Omni-Reasoner interleaves silent reasoning tokens with spoken response tokens at the token level. This design allows continuous speech generation while embedding structured internal reasoning, leveraging the model's high-frequency token processing capability. Although interleaved, local semantic alignment is enforced to ensure that each response token is informed by its preceding reasoning. To support this framework, we introduce Spoken-Math-Problems-3M, a large-scale dataset tailored for interleaved reasoning and response. The dataset ensures that verbal tokens consistently follow relevant reasoning content, enabling accurate and efficient learning of speech-coupled reasoning. Built on a hierarchical Thinker-Talker architecture, Mini-Omni-Reasoner delivers fluent yet logically grounded spoken responses, maintaining both naturalness and precision. On the Spoken-MQA benchmark, it achieves a +19.1% gain in arithmetic reasoning and +6.4% in contextual understanding, with shorter outputs and zero decoding latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
