Evaluation of socially unsafe content in spoken dialogues remains text-centric, missing prosody and transcription failures. We present LALM-as-a-Judge, which includes an open benchmark of 24,000 multi-turn spoken dialogues with one localized unsafe turn, generated out of 8 socially unsafe categories and 5 severity levels. We evaluate 6 large audio-language models (LALMs) as judges, open and closed-source, in text-only, audio-only, and multimodal setups by their sensitivity, severity-order specificity, and turn-position bias for socially harmful content in the dialogue. Results show that audio contributes non-lexical evidence beyond transcript semantics and that multimodal gains are not universal but can be text-anchored, balanced, conservative, and interfering, which we link to the audio pathway bottlenecks and fusion limits. We position the benchmark as diagnostic and derive practitioner guidance for model, modality, and prompts choices.
CLASH: Counterfactual Auditing of Lexical and Prosodic Reliance in Spoken Sarcasm Detection
Imperial College London(伦敦帝国理工学院);New York University(纽约大学);Technical University of Munich(慕尼黑工业大学);Tencent Inc.(腾讯公司);Wuhan University(武汉大学);Nankai University(南开大学)
Can large audio language models understand child stuttering speech? speech summarization, and source separation
Department of Computer Science, University of Florida, Gainesville, FL, USA(佛罗里达大学计算机科学系);Department of Computer Science, Lebanese American University(黎巴嫩美国大学计算机科学系)
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Imperial College London(帝国理工学院);Technische Universität München(慕尼黑工业大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Shanghai Jiao Tong University(上海交通大学);Johns Hopkins University(约翰斯·霍普金斯大学)
ORCA: Open-ended Response Correctness Assessment for Audio Question Answering
Brno University of Technology(布尔诺科技大学);Universidad Autónoma de Madrid(马德里自治大学);University of Buenos Aires(布宜诺斯艾利斯大学);Tufts University(塔夫茨大学);University of Maryland(马里兰大学)
Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Models
School of Computer Science, University of Auckland(奥克兰大学计算机科学学院);Singapore University of Technology and Design(新加坡科技设计大学);School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院);School of Statistics and Data Science, Nankai University(南开大学统计与数据科学学院)
Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models
Center for Language and Speech Processing (CLSP)(语言与语音处理中心);Johns Hopkins University(约翰霍普金斯大学);Group on Language, Audio & Music (GLAM)(语言、音频与音乐小组);Imperial College London(伦敦帝国学院)
MSU-Bench: Towards Speaker-Centric Understanding in Conversational Multi-Speaker Scenarios
School of Software, Northwestern Polytechnical University(西北工业大学软件学院);School of Intelligent Science and Technology, Nanjing University(南京大学智能科学与技术学院);Shenzhen Loop Area Institute(深圳河套学院);Base Model, Li Auto(理想汽车基础模型)