Recent advances in spoken dialogue language models (SDLMs) reflect growing interest in shifting from turn-based to full-duplex systems, where the models continuously perceive user speech streams while generating responses. This simultaneous listening and speaking design enables real-time interaction and the agent can handle dynamic conversational behaviors like user barge-in. However, during the listening phase, existing systems keep the agent idle by repeatedly predicting the silence token, which departs from human behavior: we usually engage in lightweight thinking during conversation rather than remaining absent-minded. Inspired by this, we propose Chronological Thinking, an on-the-fly conversational thinking mechanism that aims to improve response quality in full-duplex SDLMs. Specifically, chronological thinking presents a paradigm shift from conventional LLM thinking approaches, such as Chain-of-Thought, purpose-built for streaming acoustic input. (1) Strictly causal: the agent reasons incrementally while listening, updating internal hypotheses only from past audio with no lookahead. (2) No additional latency: reasoning is amortized during the listening window; once the user stops speaking, the agent halts thinking and begins speaking without further delay. Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations show consistent improvements in response quality. Furthermore, chronological thinking robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.
Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
Graduate Institute of Communication Engineering, National Taiwan University（国立台湾大学通信工程研究所）；UC Berkeley（加州大学伯克利分校）；University of Washington（华盛顿大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems
The Chinese University of Hong Kong（香港中文大学）；AudioLab Hong Kong, Huawei Leibniz Research Center（香港AudioLab，华为Leibniz研究中心）；Nanyang Technological University（南洋理工大学）
NVIDIA（英伟达）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Text normalization (TN) and inverse text normalization (ITN) are essential preprocessing and postprocessing steps for text-to-speech synthesis and automatic speech recognition, respectively. Many methods have been proposed for either TN or ITN, ranging from weighted finite-state transducers to neural networks. Despite their impressive performance, these methods aim to tackle only one of the two tasks but not both. As a result, in a complete spoken dialog system, two separate models for TN and ITN need to be built. This heterogeneity increases the technical complexity of the system, which in tu…
Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Future Design Lab, Innovation Center of Yangtze River Delta, Zhejiang University（浙江大学长三角创新中心未来设计实验室）；University of Virginia（弗吉尼亚大学）
Spoken Language Understanding (SLU) is a key component of goal oriented dialogue systems that would parse user utterances into semantic frame representations. Traditionally SLU does not utilize the dialogue history beyond the previous system turn and contextual ambiguities are resolved by the downstream components. In this paper, we explore novel approaches for modeling dialogue context in a recurrent neural network (RNN) based language understanding system. We propose the Sequential Dialogue Encoder Network, that allows encoding context from the dialogue history in chronological order. We com…
In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommunication to demonstrate what a human-like interactive experience should be and how to achieve smooth turn-taking through three subtasks: user state detection, backchannel selection, and barge-in detection. Besides, we propose semi-supervised learning with multimodal data augmentation to leverage unlabeled data to increase model generalization. Experimental results on three sub-ta…
Tencent（腾讯）；Zhejiang University（浙江大学）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）
PersonaTAB: Predicting Personality Traits using Textual, Acoustic, and Behavioral Cues in Fully-Duplex Speech Dialogs
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanjing University, Suzhou（南京大学（苏州））；National University of Singapore（新加坡国立大学）；School of Data Science（数据科学学院）；School of Intelligence Science and Technology（智能科学与技术学院）；Department of ECE（电子与计算机工程系）
School of Computer Science and Engineering, Northeastern University, Shenyang, China（计算机科学与工程学院，东北大学，沈阳，中国）；NiuTrans Research（NiuTrans研究）；Kunming University of Science and Technology（昆明理工大学）
Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；Center for Language, Intelligence and Machines, Shenzhen Loop Area Institute, Shenzhen（深圳环智中语言、智能与机器中心）；School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）
