STITCH：基于分块推理的 spoken language models 的同时思考与说话
STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models
AI总结：
STITCH通过分块推理实现语音语言模型的同时思考与说话，提升推理效率与响应速度。
摘要：
语音语言模型（SLMs）旨在将语音输入转换为语音响应。然而，当前的SLMs缺乏在回应前进行内部、非语音思考过程的能力。相比之下，人类通常会进行复杂的内部推理，使他们能够清晰而简洁地交流思想。因此，将内部思考过程整合到SLMs中是高度理想的。虽然简单地在开始说话前生成完整的推理链（CoT）可以为SLMs提供思考能力，但这样做会增加语音响应的延迟，因为CoT推理可以任意长。为了解决这个问题，我们提出了一种新的生成方法Stitch，该方法在生成非语音推理块和语音响应块之间交替。由于语音块的音频持续时间比生成语音响应块中的token所需时间要长得多，我们利用剩余的自由时间来生成非语音推理token。当一个音频块播放给用户时，模型继续生成下一个非语音推理块，从而实现同时的思考和说话。值得注意的是，Stitch在延迟上与无法生成非语音CoT的基线模型相当，但在数学推理数据集上比基线模型高出15%；Stitch在非推理数据集上的表现与基线模型相当。一些动画和演示在项目页面上：https://d223302.github.io/STITCH。
英文摘要：
Spoken Language Models (SLMs) are designed to take speech inputs and produce spoken responses. However, current SLMs lack the ability to perform an internal, unspoken thinking process before responding. In contrast, humans typically engage in complex mental reasoning internally, enabling them to communicate ideas clearly and concisely. Thus, integrating an unspoken thought process into SLMs is highly desirable. While naively generating a complete chain-of-thought (CoT) reasoning before starting to talk can enable thinking for SLMs, this induces additional latency for the speech response, as the CoT reasoning can be arbitrarily long. To solve this issue, we propose Stitch, a novel generation method that alternates between the generation of unspoken reasoning chunks and spoken response chunks. Since the audio duration of a chunk of spoken response is much longer than the time to generate the tokens in a chunk of spoken response, we use the remaining free time to generate the unspoken reasoning tokens. When a chunk of audio is played to the user, the model continues to generate the next unspoken reasoning chunk, achieving simultaneous thinking and talking. Remarkably, Stitch matches the latency of baselines that cannot generate unspoken CoT by design while outperforming those baselines by 15% on math reasoning datasets; Stitch also performs equally well on non-reasoning datasets as those baseline models. Some animations and demonstrations are on the project page: https://d223302.github.io/STITCH.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
