The ability to automatically detect stuttering events in speech could help speech pathologists track an individual's fluency over time or help improve speech recognition systems for people with atypical speech patterns. Despite increasing interest in this area, existing public datasets are too small to build generalizable dysfluency detection systems and lack sufficient annotations. In this work, we introduce Stuttering Events in Podcasts (SEP-28k), a dataset containing over 28k clips labeled with five event types including blocks, prolongations, sound repetitions, word repetitions, and interjections. Audio comes from public podcasts largely consisting of people who stutter interviewing other people who stutter. We benchmark a set of acoustic models on SEP-28k and the public FluencyBank dataset and highlight how simply increasing the amount of training data improves relative detection performance by 28\% and 24\% F1 on each. Annotations from over 32k clips across both datasets will be publicly released.
Findings of the 2024 Mandarin Stuttering Event Detection and Automatic Speech Recognition Challenge
Northwestern Polytechnical University(西北工业大学);StammerTalk;AIShell Inc(AIShell公司);Nankai University(南开大学);University of Science and Technology of China(中国科学技术大学);Wuhan University(武汉大学)
Leveraging LLM for Stuttering Speech: A Unified Architecture Bridging Recognition and Event Detection
Beijing Fosafer Information Technology Co., Ltd.(北京福赛弗信息技术有限公司);China Institute of Forensic Science, Ministry of Public Security(中国公安部 forensic 科学研究院);The Institute of Linguistics, Chinese Academy of Social Sciences(中国社会科学院语言研究所)
本文提出一种 LLM 驱动的 ASR-SED 多任务学习框架,通过动态交互机制联合优化口吃语音识别与事件检测,在 AS-70 数据集上将 ASR CER 降至 5.45% 并提升 SED F1 至 73.63%。
StutterNet: Stuttering Detection Using Time Delay Neural Network
Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);LORIA(洛林信息学及其应用实验室);Université Paul-Valéry Montpellier(蒙彼利埃保罗-瓦莱里大学);Praxiling(普拉西林实验室)
Dysfluencies Seldom Come Alone -- Detection as a Multi-Label Problem
Technische Hochschule Nürnberg Georg Simon Ohm(纽伦堡乔治·西蒙·欧姆应用科学大学);KST Institut GmbH(KST研究所有限公司);Intel Labs(英特尔实验室);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学)
AS-70: A Mandarin stuttered speech dataset for automatic speech recognition and stuttering event detection
StammerTalk;AIShell Inc(AIShell公司);AImpower;Nankai University(南开大学);University of Science and Technology of China(中国科学技术大学);Duke Kunshan University(昆山杜克大学)
本文构建首个公开且规模最大的普通话口吃语音数据集 AS-70,覆盖对话与命令朗读并提供逐字转写,通过基线实验和微调显著提升 Whisper、Hubert 等模型的口吃 ASR 与 SED 能力。
ASTER: Automatic Speech Recognition System Accessibility Testing for Stutterers
Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Meta AI;University of Southern California(南加州大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);The University of Tokyo(东京大学);University of Alberta(阿尔伯塔大学)
Large Language Models for Dysfluency Detection in Stuttered Speech
Technische Hochschule Nürnberg Georg Simon Ohm(纽伦堡乔治·西蒙·欧姆应用技术大学);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Intel Labs(英特尔实验室)
本文将多标签口吃不流利检测建模为语言建模问题,向 LLM 输入 ASR 假设候选与音频编码器声学表征并微调,在英语、德语数据集上取得有竞争力的结果。
A Stutter Seldom Comes Alone -- Cross-Corpus Stuttering Detection as a Multi-label Problem
Technische Hochschule Nürnberg Georg Simon Ohm(纽伦堡乔治·西蒙·欧姆应用技术大学);KST Institut GmbH(KST研究所有限公司);Intel Labs(英特尔实验室);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学)
Robust Stuttering Detection via Multi-task and Adversarial Learning
Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);LORIA(洛林计算机科学及其应用研究所);Université Paul-Valéry Montpellier(蒙彼利埃保罗-瓦莱里大学);Praxiling(Praxis语言学研究所)
Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning
Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);LORIA(洛林计算机科学与应用数学研究所);Université Paul-Valéry Montpellier(蒙彼利埃保罗-瓦莱里大学);Praxiling(普拉西林实验室)
Can large audio language models understand child stuttering speech? speech summarization, and source separation
Department of Computer Science, University of Florida, Gainesville, FL, USA(佛罗里达大学计算机科学系);Department of Computer Science, Lebanese American University(黎巴嫩美国大学计算机科学系)
Predicting Upcoming Stuttering Events from Three-Second Audio: Stratified Evaluation Reveals Severity-Selective Precursors, and the Model Deploys Fully On-Device
Virtual Speech Therapist: A Clinician-in-the-Loop AI Agent for Speech Therapy Planning
The Kashmir Hub for Artficial Intelligence(喀布尔人工智能中心);Microsoft / Vocametrix(微软 / Vocametrix);IAI, TCG CREST(IAI,TCG CREST);Université de Lorraine, CNRS, Inria, LORIA(洛林大学,CNRS,Inria,LORIA);Laboratoire Praxiling, UMR5267, CNRS et Université Paul-Valéry Montpellier 3(Praxiling实验室,UMR5267,CNRS及蒙彼利埃Paul-Valéry大学);Speechcare iStutter, Portuguese Catholic University(Speechcare iStutter,葡萄牙天主教大学);CHI – Chair of Health Informatics, TUM University Hospital(健康信息学系,TUM大学医院);GLAM – Group on Language, Audio, & Music, Imperial College London(语言、音频与音乐小组,伦敦帝国理工学院)