Current speech-language models (SLMs) typically use a cascade of speech encoder and large language model, treating speech understanding as a single black box. They analyze the content of speech well but reason weakly about other aspects, especially under sparse supervision. Thus, we argue for explicit reasoning over speech states and actions with modular and transparent decisions. Inspired by cognitive science we adopt a modular perspective and a world model view in which the system learns forward dynamics over latent states. We factorize speech understanding into four modules that communicate through a causal graph, establishing a cognitive state search space. Guided by posterior traces from this space, an instruction-tuned language model produces a concise causal analysis and a user-facing response, enabling counterfactual interventions and interpretability under partial supervision. We present a graph-based modular speech model for explicit reasoning, highlighting a path toward more transparent and controllable speech understanding.
A Distilled Low-Latency Neural Vocoder with Explicit Amplitude and Phase Prediction
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China(中国科学技术大学语音和语言信息处理国家工程研究中心)
A Fast and Lightweight Model for Causal Audio-Visual Speech Separation
School of Computer Technology and Application(计算机技术与应用学院);Intelligent Computing and Application Laboratory of Qinghai Province(青海省智能计算与应用实验室);Department of Computer Science and Technology(计算机科学与技术系);Institute for AI(人工智能研究院);Tsinghua Laboratory of Brain and Intelligence (THBI)(清华大学脑与智能实验室);IDG/McGovern Institute for Brain Research(IDG/麦克戈维脑研究学院);Tsinghua University(清华大学);Chinese Institute for Brain Research (CIBR)(中国脑科学研究院)
Embedding and Beamforming: All-neural Causal Beamformer for Multichannel Speech Enhancement
Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);University of Chinese Academy of Sciences(中国科学院大学)
空间协方差矩阵已被认为是重要的波束形成器。基于传统波束形成器和深度神经网络的交叉点,提出了一种因果神经波束形成器的范式Embedding and Beamforming,并设计了两个核心模块EM和BM。对于EM,不是显式地估计空间协方差矩阵,而是使用网络学习3-D嵌入张量,其中可以表示光谱和空间判别信息。对于BM,直接利用网络来导出波束成形权重,以便实现滤波求和操作。为了进一步提高语音质量,引入后处理模块进一步抑制残余噪声。基于DNS-Challenge数据集,我们进行了多通道语音增强的实验,结果表明,该系统在多个评价指标上大大优于以前的先进基线。
Inference and Denoise: Causal Inference-based Neural Speech Enhancement
Research Center for Information Technology Innovation, Academia Sinica(中央研究院资讯科技创新研究中心);Georgia Institute of Technology(佐治亚理工学院);IBM Research(IBM研究院);Norwegian University of Science and Technology(挪威科技大学)
On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition
DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action
StepFun(阶跃星辰);Peking University(北京大学);Nanyang Technological University(南洋理工大学);Shanghai Jiao Tong University(上海交通大学);University of New South Wales(新南威尔士大学);Imperial College London(帝国理工学院)
Time-Frequency-Based Attention Cache Memory Model for Real-Time Speech Separation
Department of Computer Science and Technology, BNRist, THBI, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China(清华大学计算机科学与技术系,BNRist,THBI,IDG/麦克戈文脑研究 institute,清华大学,北京,中国);Chinese Institute for Brain Research (CIBR), Beijing, China(中国脑科学研究院(CIBR),北京,中国)
An Empirical Study on the Impact of Positional Encoding in Transformer-based Monaural Speech Enhancement
University of New South Wales(新南威尔士大学);National University of Singapore(新加坡国立大学);Alibaba Group(阿里巴巴集团);Meta;The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Virtual Speech Therapist: A Clinician-in-the-Loop AI Agent for Speech Therapy Planning
The Kashmir Hub for Artficial Intelligence(喀布尔人工智能中心);Microsoft / Vocametrix(微软 / Vocametrix);IAI, TCG CREST(IAI,TCG CREST);Université de Lorraine, CNRS, Inria, LORIA(洛林大学,CNRS,Inria,LORIA);Laboratoire Praxiling, UMR5267, CNRS et Université Paul-Valéry Montpellier 3(Praxiling实验室,UMR5267,CNRS及蒙彼利埃Paul-Valéry大学);Speechcare iStutter, Portuguese Catholic University(Speechcare iStutter,葡萄牙天主教大学);CHI – Chair of Health Informatics, TUM University Hospital(健康信息学系,TUM大学医院);GLAM – Group on Language, Audio, & Music, Imperial College London(语言、音频与音乐小组,伦敦帝国理工学院)
ForkNet: Simultaneous Time and Time-Frequency Domain Modeling for Speech Enhancement
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Institute of Acoustics, Chinese Academy of Science(中国科学院声学研究所)