arXiv 2609.14005cs.SDeess.AS

StepAudio 3 实时技术报告

StepAudio 3 Realtime Technical Report

Bin Lin, Bo Zhao, Boyang Zhang, Boyong Wu, Chao Yan, Chen Geng, Chen Wu, Cheng Yi, Chengli Feng, Chenglin Zhu, Chengting Feng, Chengyuan Yao, Daijiao Liu, DanNi… 展开作者

Bin Lin, Bo Zhao, Boyang Zhang, Boyong Wu, Chao Yan, Chen Geng, Chen Wu, Cheng Yi, Chengli Feng, Chenglin Zhu, Chengting Feng, Chengyuan Yao, Daijiao Liu, DanNi Wan, Daxin Jiang, Dongjian Li, Dongqing Pang, Fei Tian, Feng Tian, Future Li, Gang Yu, Guanglong Yang, Haoyang Zhang, Hongyuan Wang, Jia Peng, Jiahao Song, Jialong Xue, Jiamin Fan, Jiangjie Zhen, Jianzheng Gao, Jincheng Wen, Jinghua Liang, Jinglan Gong, Jun Chen, Li Xie, Liang Zhao, Lifang Zhang, Lingli Ji, Lun Cai, Min Xu, Peilin Li, Peng Yang, Pengfei Tan, Qingjian Lin, Qinxin Du, Ruijie Xiong, Runze Li, Shenghua Hu, Shengqian Qin, Shi Qiu, Siqi Tu, Siyi Zhou, Tianjiao Deng, Wanying Lu, Weiming Niu, Wen Sun, WenWen Qu, Xiangyu Zhang, Xianwei Zhang, Xiaosu Su, Xing Chen, Xinyu Liu, Xuerui Yang, Yan Wu, Yang Li, Yang Yang, Yechang Huang, Yibo Zhu, Yifan Zhang, Yinuo Yan, Youjun Chen, Yu Fu, Yu Luo, Yu Zhou, Yujie Chen, Yumang Wang, Yunzhou Ju, Yuxiang Yang, Yuxin Li, Yuxin Zhang, Zekai Liu, Zengwei Yao, Zhaoxin Yuan, Zhenwei Mou, Zhiquan Zhang, Zhiyue Wu, Zichao Li, Zichao Zhou, Ziqi Ren, Zixuan Wang

首次发表 2026-09-22

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

浏览论文内容

中文总结 AI 辅助

本文提出 StepAudio 3 Realtime，一种基于听-说-思-行循环的音频语言模型，通过边说边想机制实现实时深度推理，在多个基准上取得领先性能。

中文摘要 AI 辅助

实时口语交互需要深度推理、快速响应和流畅的轮换发言。我们提出了 StepAudio 3 Realtime，这是一个围绕连续的“听-说-思-行”循环组织的音频语言基础模型。深度感知（Deep Perception）捕获丰富的声学线索以理解用户意图，而无缝双工（Seamless Duplex）则对同步音频流进行建模，以自然处理停顿、反馈语和打断。关键的是，我们通过“边说边想”（Think-While-Speaking）解决了深度思考与延迟之间的冲突，在口语输出的同时并行执行私有推理。在推理模式下，StepAudio 3 在 StepAudioChat 上达到了 73.0 的宏平均分数。通过“边说边想”，它在实时说话的同时实现了与专用推理模型相当的对话和推理性能。此外，集成的语音代理（Voice Agent）处理异步工具执行，而不会中断对话流程。StepAudio 3 Realtime 在关键维度上达到了顶级性能：在 MMSU 基准上取得了出色的 90.6 分，在 Artificial Analysis 全双工基准上取得了 98.9 的总分，以及在 τ-Voice 上取得了 56.0% 的宏任务成功率。

英文摘要

Realtime spoken interaction demands deep reasoning, prompt responses, and fluid turn-taking. We present StepAudio 3 Realtime, an audio-language foundation model organized around a continuous listen-converse-think-act loop. Deep Perception captures rich acoustic cues to interpret user intent, while Seamless Duplex models synchronized audio streams to handle pauses, backchannels, and interruptions naturally. Crucially, we resolve the tension between deep deliberation and latency via Think-While-Speaking, executing private reasoning in parallel with spoken delivery. In reasoning mode, StepAudio 3 reaches a 73.0 macro average on StepAudioChat. With Think-While-Speaking, it achieves dialogue and reasoning performance comparable to dedicated reasoning models while speaking in real time. Furthermore, an integrated Voice Agent handles asynchronous tool execution without disrupting the dialogue flow. StepAudio 3 Realtime achieves top-tier performance across key dimensions: an exceptional 90.6 on the MMSU benchmark, 98.9 Overall on the Artificial Analysis Full-Duplex Bench, and a 56.0% macro task-success rate on $τ$-Voice.