Recent advances in speech generation have enabled high-fidelity synthesis, yet systematic evaluation of models under long-context conditions remains largely underexplored. A comprehensive evaluation benchmark for long-form speech is indispensable for two reasons: 1) existing test scenarios are often confined to limited domains, creating a significant gap with the diverse downstream applications; 2) existing metrics overlook critical long-text factors such as consistency and coherence, failing to generalize reliably. To this end, we propose Swanbench-Speech, a comprehensive benchmark that decomposes long-form speech quality into specific, disentangled dimensions. SwanBench-Speech has three key properties. 1) Rich speech scenarios: Focusing on long-form speech generation and dialog generation, SwanBench-Speech covers acoustics, semantics, and expressiveness challenges, and consists of 1,101 samples spanning 17 common speech scenarios; 2) Comprehensive evaluation dimensions: Along the acoustics, semantics, and expressiveness axes, SwanBench-Speech defines an automated evaluation protocol with seven metrics to provide a comprehensive, accurate, and standardized assessment; 3) Valuable Insights: Through extensive experiments, we reveal that current models still struggle in highly expressive scenarios and exhibit a notable gap in consistency and hierarchy compared to real recordings.
Benchmarking Speech Systems for Frontline Health Conversations: The DISPLACE-M Challenge
Indian Institute of Science(印度科学学院);National Institute of Technology Karnataka(卡纳塔克国立技术学院);Josh Talks;TANUH AI Center of Excellence(TANUH 人工智能卓越中心);UPES(乌塔拉克罕大学与能源研究教育大学);Indian Institute of Technology Bombay(孟买印度理工学院)
SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant
Shanghai Jiao Tong University(上海交通大学);Ant Group(蚂蚁集团);Wuhan University(武汉大学);School of Artificial Intelligence(人工智能学院);School of Cyber Science and Engineering(网络科学与工程学院)
Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?
LTCI, Télécom Paris, Institut Polytechnique de Paris(巴黎理工学院巴黎电信学院LTCI);Hi! PARIS Engineering Team(Hi! PARIS工程团队);Capgemini(凯捷);Samsung AI Center(三星人工智能中心);University of Cambridge(剑桥大学);Mila-Quebec AI Institute(Mila-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Concordia University(康考迪亚大学)
Benchmarking Evaluation Metrics for Code-Switching Automatic Speech Recognition
New York University Abu Dhabi(纽约大学阿布扎比分校);University of Stuttgart(斯图加特大学);Johns Hopkins University(约翰斯·霍普金斯大学);Stanford University(斯坦福大学);Qatar Computing Research Institute(卡塔尔计算研究所);Microsoft Research India(微软印度研究院)
VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs
Department of Computer Science, University of Sheffield, United Kingdom(英国谢菲尔德大学计算机科学系);Signal Analysis and Interpretation Laboratory (SAIL), Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA 90089, USA(美国南加州大学电气与计算机工程系信号分析与解释实验室(SAIL))
MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models
Graduate School of Informatics, Nagoya University, Japan(名古屋大学信息科学研究生院,日本);National Institute of Information and Communications Technology (NICT), Japan(日本信息通信技术研究所);Information Technology Center, Nagoya University, Japan(名古屋大学信息技术中心,日本)
Lahore University of Management Sciences(拉瓦尔大学管理科学学院);University of Michigan - Ann Arbor(密歇根大学安娜堡分校);EMBL European Bioinformatics Institute(欧洲生物信息学研究所)
Benchmarking Foundation Speech and Language Models for Alzheimer's Disease and Related Dementia Detection from Spontaneous Speech
Georgia Institute of Technology(佐治亚理工学院);Shandong Mental Health Center(山东精神卫生中心);University of Texas Rio Grande Valley(德克萨斯大学里奥格兰德河谷分校);Kennesaw State University(肯尼索州立大学)
SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR
Department of Electrical Engineering KU Leuven Leuven, Belgium(电气工程系 卡尔·冯·特里尔大学 布鲁塞尔,比利时);Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, USA(语言技术研究所 哈佛大学 贝尔蒙特,美国)
Benchmarking Commercial Speech Recognition and Multimodal Large Language Models on Dysarthric Speech: Severity-Stratified Baselines and Architecture-Specific Prompting Effects
Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correction
Nara Institute of Science and Technology (NAIST)(奈良科学技术研究所(NAIST));National Institute of Information and Communications Technology (NICT)(信息通信研究机构(NICT));NVIDIA Research(英伟达研究院);Kyoto University(京都大学)
Benchmarking Automatic Speech Recognition Models for African Languages
Makerere University Centre for Artificial Intelligence(Makerere大学人工智能中心);Marconi Lab(Marconi实验室);Department of Information Systems(信息系统系);Department of Electrical Engineering(电气工程系);Department of Computer Science(计算机科学系)
A Comprehensive Survey with Critical Analysis for Deepfake Speech Detection
Austrian Institute of Technology(奥地利技术研究院);HCM University of Technology(胡志明市理工大学);University of Technology of Troyes(特鲁瓦技术大学);FPT University(FPT大学);BLOOM Social Analytics(BLOOM社会分析公司);Université de technologie de Compiègne(贡比涅技术大学)