arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.05933eess.AS
收藏

语音世界模型:基于显式推理的语音因果状态-动作规划

Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech

Zhejiang University(浙江大学);UC Berkeley(加州大学伯克利分校)

Xuanru Zhou, Jiachen Lian, Henry Hong, Xinyi Yang, Gopala Anumanchipalli

AI总结:

研究旨在改进语音语言模型推理弱的问题,采用模块化视角和世界模型观点,将语音理解分解为四个模块通过因果图通信,建立认知状态搜索空间,实现显式推理,让语音理解更透明可控。

摘要:

当前的语音语言模型(SLMs)通常采用语音编码器和大语言模型的级联,将语音理解视为一个单一的黑箱。它们能很好地分析语音内容,但在其他方面的推理能力较弱,尤其是在稀疏监督下。因此,我们主张对语音状态和动作进行显式推理,做出模块化和透明的决策。受认知科学启发,我们采用模块化视角和世界模型观点,系统在潜在状态上学习前向动力学。我们将语音理解分解为四个通过因果图通信的模块,建立认知状态搜索空间。在该空间的后验轨迹引导下,指令调整后的语言模型产生简洁的因果分析和面向用户的响应,在部分监督下实现反事实干预和可解释性。我们提出了一种基于图的模块化语音模型用于显式推理,突出了一条通向更透明和可控语音理解的路径。

英文摘要:

Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the content of speech well but reason weakly about other aspects, especially under sparse supervision. Thus, we argue for explicit reasoning over speech states and actions with modular and transparent decisions. Inspired by cognitive science we adopt a modular perspective and a world model view in which the system learns forward dynamics over latent states. We factorize speech understanding into four modules that communicate through a causal graph, establishing a cognitive state search space. Guided by posterior traces from this space, an instruction-tuned language model produces a concise causal analysis and a user-facing response, enabling counterfactual interventions and interpretability under partial supervision. We present a graph-based modular speech model for explicit reasoning, highlighting a path toward more transparent and controllable speech understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑