PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects
Shenzhen International Graduate School, Tsinghua University(深圳国际研究生院,清华大学);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);JD AI Research(京东人工智能研究院)
Sicheng Yang, Shulan Ruan, Shiwei Wu, Yu Liu, Lu Fan, Zhi Li, You He
While End-to-End (E2E) Speech-Large Language Models (Speech-LLMs) are rapidly evolving, their evaluation methodologies remain limited to the era of simple transcription. Existing benchmarks suffer from three critical limitations: a pronounced bias towards high-resource languages, a focus on low-level recognition (ASR) rather than semantic reasoning, and a neglect of regional dialects. To bridge this gap, we introduce PolySpeech-100, a massive-scale benchmark designed to assess `native-level' speech comprehension across 110 linguistic variants. We employ a novel hybrid construction pipeline that augments gold-standard human recordings with instruction-driven synthetic speech, allowing us to cover 19 distinct Chinese dialects and over 80 low-resource languages. Extensive evaluation of 22 state-of-the-art models (including Gemini-3, GPT-Audio, and Qwen2.5-Omni) yields pivotal insights. First, we demonstrate that open-source E2E models outperform Cascade (ASR+LLM) systems on heavy dialects, proving that direct audio processing preserves critical paralinguistic cues and prosodic features (e.g., intonation, stress) that are often lost in standard transcription. Second, we reveal a significant performance gap: while commercial models maintain robustness, open-source models suffer catastrophic degradation on low-resource languages. Finally, counter-intuitively, we observe that under standard zero-shot settings, Chain-of-Thought prompting frequently degrades speech understanding performance for most evaluated models, revealing a potential modality alignment gap in current architectures. PolySpeech-100 establishes a rigorous standard for the next generation of inclusive, omni-capable Speech-LLMs. The data, demo, and code are publicly available at https://github.com/YoungSeng/PolySpeech-100.
Can LLM Agents Identify Spoken Dialects like a Linguist?
Department of Computer Science, University of Bonn(波恩大学计算机科学系);Research Center Deutscher Sprachatlas, Marburg University(马尔堡大学德语语言地图研究中心);Bonn-Aachen International Center for Information Technology, University of Bonn(波恩大学波恩-亚琛国际信息技术中心);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所)
DialectS2S: End-to-End Speech Dialogue Modeling for Low-Resource Chinese Dialects
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Wuhan AI Research(武汉人工智能研究院);GWM AI Lab(长城汽车人工智能实验室)
TMD-TTS: A Unified Tibetan Multi-Dialect Text-to-Speech Framework for Ü-Tsang, Amdo and Kham Speech Dataset Generation
School of Information Science and Technology, Tibet University, China(西藏大学信息科学与技术学院);Department of Ophthalmology, University of Texas Southwestern Medical Center, USA(德克萨斯西南医学中心眼科部门)
STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions
University of Applied Sciences and Arts Northwestern Switzerland(瑞士西北应用科学与艺术大学);Zurich University of Applied Sciences(苏黎世应用科学大学);University of Zurich(苏黎世大学)
SDS-200: A Swiss German Speech to Standard German Text Corpus
University of Applied Sciences and Arts Northwestern Switzerland(西北瑞士应用科学与艺术大学);Zurich University of Applied Sciences(苏黎世应用科学大学);SpinningBytes AG(SpinningBytes公司)
Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Dept. Computer Science and Information Engineering, National Taiwan Normal University, Taiwan(计算机科学与信息工程系,国立台湾师范大学,台湾);Institute of Computer Science, Academia Sinica, Taiwan(学术院计算机科学研究所,台湾);United Link Co., Ltd., Taiwan(联合链接公司,台湾)
From `Snippet-lects' to Doculects and Dialects: Leveraging Neural Representations of Speech for Placing Audio Signals in a Language Landscape
Langues et Civilisations à Tradition Orale (LACITO)(口头传统语言与文明研究所(LACITO));CNRS(法国国家科学研究中心);Université Sorbonne Nouvelle(索邦新大学);Institut National des Langues et Civilisations Orientales (INALCO)(法国国立东方语言文化学院(INALCO));Université de Paris Cité(巴黎西岱大学);Laboratoire de Linguistique Formelle (LLF)(形式语言学实验室(LLF))
Automatic Dialect Detection in Arabic Broadcast Speech
Qatar Computing Research Institute, HBKU(哈马德·本·哈利法大学卡塔尔计算研究所);MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)(麻省理工学院计算机科学与人工智能实验室);JHU Center for Language and Speech Processing (CLSP)(约翰斯·霍普金斯大学语言与语音处理中心);University of Edinburgh(爱丁堡大学)
A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Manipal Institute of Technology(马尼帕尔理工学院);Georgia Institute of Technology(佐治亚理工学院);SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence(沙特数据与人工智能局-阿卜杜拉国王科技大学数据科学与人工智能卓越中心);Karolinska Institutet(卡罗林斯卡学院)
Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition
QCRI, HBKU(卡塔尔计算研究所,哈马德·本·哈利法大学);Qatar University(卡塔尔大学);UDST(多哈科技大学);Algo AI;UCL(伦敦大学学院);University of Tripoli(的黎波里大学);AUC(开罗美国大学);KAUST(阿卜杜拉国王科技大学);CMU-Q(卡内基梅隆大学卡塔尔分校);KFUPM(法赫德国王石油与矿产大学);Alfaisal University(费萨尔大学);Damascus University(大马士革大学);Princeton University(普林斯顿大学);ENSIAS, Mohammed V University(穆罕默德五世大学国家计算机科学与系统分析学院);Sultan Qaboos University(苏丹卡布斯大学);DFKI(德国人工智能研究中心);University of Waterloo(滑铁卢大学);USTHB(阿尔及尔科技大学)
Overcoming Data Scarcity in Multi-Dialectal Arabic ASR via Whisper Fine-Tuning
Department of Internal Medicine, Division of Nephrology(内科学部,肾病科);Bernoulli Institute for Mathematics, Computer Science(伯努利数学、计算机科学研究所);Speech Technology Lab, Campus Fryslân(语音技术实验室,弗里西恩校区)