arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2605.20946cs.CL

思考-言语:一种受控交错推理方法用于实时语音生成

Thinking-while-speaking: A Controlled, Interleaved Reasoning Method for Real-Time Speech Generation

Huawei Technologies(华为技术)

Xuan Du, Qiangyu Yan, Wenshuo Li, Borui Jiang, Changming Xiao, Han Shu, Xinghao Chen

AI总结:

本文提出了一种受控交错推理方法InterRS,用于实时语音生成,通过在自然语音生成过程中插入推理步骤,提高了语音流畅性和推理深度,实验表明其在数学和逻辑基准测试中表现更优,并生成更自然流畅的答案。

摘要:

思考-言语范式旨在使AI交流更人性化。关键挑战是保持流畅的语音生成同时进行深度推理。我们的方法InterRS通过在自然语音生成过程中插入推理步骤来解决这一问题。这需要高质量的数据,其中推理和语音精确对齐且长度比例受控。我们引入了一种新的管道来生成无缝交错的音频数据。为了训练我们的模型,我们结合了交错SFT与精炼数据以及强化学习,使用两种新的奖励:TA-Balance Reward用于管理时间与思考-回答比例,以及Linguistic Quality Reward用于优化表达。实验表明,我们的方法在数学和逻辑基准测试中实现了13%的性能提升,同时生成像口语指令模型一样快速的响应。此外,我们的方法生成的答案例句比先前方法更自然流畅。

英文摘要:

The thinking-while-speaking paradigm aims to make AI communication more human. A key challenge is maintaining fluent speech while performing deep reasoning. Our method, InterRS, tackles this by inserting reasoning steps only during natural speech generation. This requires high-quality data where reasoning and speech are precisely aligned, and the length ratio are under controlled. We introduce a novel pipeline to generate such seamlessly interleaved audio data. To train our model, we combine interleaved SFT with refined data and reinforcement learning with two new rewards: a TA-Balance Reward to manage timing and thinking-answer ratio, and a Linguistic Quality Reward to refine expression. Experiments show our approach achieves 13% better performance on mathmatical and logic benchmarks while generating instant response like a spoken-language instruct model which outputs fast CoT response. Furthermore, our method generates more natural and fluent answers than prior methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑