Large Audio Language Models (LALMs) are rapidly advancing, but evaluating them remains challenging due to inefficient and non-standardized toolkits that limit fair comparison and systematic assessment. Existing evaluation frameworks exhibit three critical limitations: (1) slow and inefficient processing pipeline that bottlenecks large-scale studies, (2) inadequate multi-turn dialogue support, leaving fundamental questions about cross-turn context integration and performance dynamics over extended conversations in LALMs unanswered; and (3) the absence of unified and scalable evaluation framework capable of keeping pace with the rapid growth of both LALMs and audio benchmarks. To address these issues, we introduce AU-Harness, an efficient and comprehensive evaluation framework for LALMs. Our system achieves a speedup of up to 151% over existing evaluation toolkits through optimized batch processing and parallel execution, enabling large-scale evaluations previously considered impractical. We provide standardized prompting protocols and flexible configurations for fair model comparison across diverse scenarios. AU-Harness unlocks a range of in-depth analyses difficult to conduct without a unified foundation, including multi-turn dialogue dynamics, enabling the study of true audio reasoning capabilities in existing LALMs. AU-Harness provides both practical evaluation tools and insights into model limitations, advancing systematic LALM development.
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Electronic Science and Technology of China（电子科技大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；AstralForge AI Lab（AstralForge AI实验室）
University of Augsburg（奥格斯堡大学）；Imperial College London（帝国理工学院）
N-HANS is a Python toolkit for in-the-wild audio enhancement, including speech, music, and general audio denoising, separation, and selective noise or source suppression. The functionalities are realised based on two neural network models sharing the same architecture, but trained separately. The models are comprised of stacks of residual blocks, each conditioned on additional speech or environmental noise recordings for adapting to different unseen speakers or environments in real life. In addition to a Python API, a command line interface is provided to researchers and developers, both of wh…
The Chinese University of Hong Kong Shanghai AI Laboratory（香港中文大学上海人工智能实验室）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Innovation Institute（上海创新研究院）
Center for Research on Foundation Models (CRFM)（基础模型研究中心（CRFM））；Institute for Human-Centered Artificial Intelligence (HAI)（以人为本人工智能研究院（HAI））；Stanford University（斯坦福大学）
Hearing Health in Home Healthcare: Leveraging LLMs for Illness Scoring and ALMs for Vocal Biomarker Extraction
The Fu Foundation School of Engineering and Applied Science, Columbia University（哥伦比亚大学福基金会工程与应用科学学院）；School of Nursing, Columbia University（哥伦比亚大学护理学院）；Center for Home Care Policy & Research, VNS Health（VNS健康居家护理政策与研究中心）
The Foundations of Computational Management: A Systematic Approach to Task Automation for the Integration of Artificial Intelligence into Existing Workflows
Vivenxia Group LLC（Vivenxia集团有限责任公司）；Centre for Digital Therapeutics（数字治疗中心）；University of Southern California（南加州大学）
