arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2601.22889cs.CLcs.AIcs.LGcs.SD

DiffuSpeech: 通过统一的语音-文本扩散实现无声思考, spoken 答案

DiffuSpeech: Silent Thought, Spoken Answer via Unified Speech-Text Diffusion

National University of Singapore(新加坡国立大学);Tencent(腾讯)

Yuxuan Lou, Ziming Wu, Yaochen Wang, Yong Liu, Yingxuan Ren, Fuming Lai, Shaobing Lian, Jie Tang, Yang You

AI总结:

DiffuSpeech通过统一的语音-文本扩散模型,实现语音生成的同时生成内部推理,提升语音问答的准确性和语音质量。

摘要:

当前的语音语言模型在生成响应时没有显式推理,导致生成的错误无法在音频生成后进行纠正。我们引入了

英文摘要:

Current speech language models generate responses directly without explicit reasoning, leading to errors that cannot be corrected once audio is produced. We introduce \textbf{``Silent Thought, Spoken Answer''} -- a paradigm where speech LLMs generate internal text reasoning alongside spoken responses, with thinking traces informing speech quality. To realize this, we present \method{}, the first diffusion-based speech-text language model supporting both understanding and generation, unifying discrete text and tokenized speech under a single masked diffusion framework. Unlike autoregressive approaches, \method{} jointly generates reasoning traces and speech tokens through iterative denoising, with modality-specific masking schedules. We also construct \dataset{}, the first speech QA dataset with paired text reasoning traces, containing 26K samples totaling 319 hours. Experiments show \method{} achieves state-of-the-art speech-to-speech QA accuracy, outperforming the best baseline by up to 9 points, while attaining the best TTS quality among generative models (6.2\% WER) and preserving language understanding (66.2\% MMLU). Ablations confirm that both the diffusion architecture and thinking traces contribute to these gains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑