RUBATO：解耦时钟使全双工模型能以自身最快速度思考
RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
AI总结：
本文提出RUBATO，通过为全双工模型各流分配独立时钟实现解耦，仅用两个极端速度训练即可适配任意解码速率，在5项口语推理任务中大幅优于基线模型，尤其在复杂中断场景下表现突出。
摘要：
近期的语音全双工（FDX）模型能够通过同时听和说来进行自然对话。在此基础上，近期的架构引入了内部思考流以提升其智能。为保证这些流能实时成功处理，现有方法采用固定解码率，该方法假设硬件配置能达到特定延迟。我们认为这种静态方法严重次优。由于内部思考本质上是异步的，理想的FDX模型应在维持实时对话流的同时，允许每个流以自身硬件自适应速率解码。思考流必须以硬件允许的最快速度处理，以更快提供准确答案。为解决该问题，我们提出RUBATO，一种新型多流位置编码，它为完全解耦的流分配独立时钟。这种结构隔离使模型能零样本插值任意推理解码速度。它大幅简化数据集构建，仅需在两个极端解码速度上训练，部署时即可平滑适配任意中间速率。RUBATO在5项口语推理任务上展现出一致鲁棒性。在Spoken GSM8K任务中，基线模型在插值速度下准确率降至1.7%；相反，RUBATO-Full在所有速率下维持48.7%至51.1%的准确率，利用更快硬件实现6.5倍的答案生成时间缩短，同时保持恒定的首次响应时间。该优势在BIG-Bench Audio任务上依然存在，且在复杂全双工中断时显著放大。具体而言，当用户在模型主动说话和推理时插话，RUBATO-Full达到40.0%至45.1%的准确率，而基线模型完全失效。
英文摘要：
Recent Speech Full-Duplex (FDX) models are capable of engaging in naturalistic conversation by listening and speaking at the same time. Building on this foundation, recent architectures introduce internal thinking streams to enhance their intelligence. To guarantee these streams process successfully in real time, existing methods enforce a fixed decoding rate, which assumes a hardware profile capable of a specific latency. We argue this static approach is severely suboptimal. Because internal thinking is intrinsically asynchronous, an ideal FDX model should sustain real-time conversational flow while allowing each stream to decode at its own hardware-adaptive rate. Thinking streams must process as fast as the hardware allows to deliver accurate answers faster. To solve this, we propose RUBATO, a novel multistream positional encoding that assigns independent clocks to entirely decouple the streams. This structural isolation empowers the model to zero-shot interpolate across any inference decoding speed. It drastically simplifies dataset construction by requiring training on only two extreme decoding speeds while smoothly adapting to any intermediate rate during deployment. RUBATO demonstrates consistent robustness across five spoken reasoning tasks. On Spoken GSM8K, a baseline model collapses to 1.7% accuracy at interpolated speeds. Conversely, RUBATO-Full sustains 48.7% to 51.1% accuracy across all rates, leveraging faster hardware to achieve a 6.5x shorter time-to-answer with a constant time-to-first-response. This advantage holds on BIG-Bench Audio and amplifies significantly during complex full-duplex interruptions. Specifically, when a user interjects while the model is actively speaking and reasoning, RUBATO-Full achieves 40.0% to 45.1% accuracy, whereas the baseline fails completely.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
