Audio-visual understanding is a rapidly evolving field that seeks to integrate and interpret information from both auditory and visual modalities. Despite recent advances in multi-modal learning, existing benchmarks often suffer from strong visual bias -- when answers can be inferred from visual data alone -- and provide only aggregate scores that conflate multiple sources of error. This makes it difficult to determine whether models struggle with visual understanding, audio interpretation, or audio-visual alignment. In this work, we introduce DAVE: Diagnostic Audio Visual Evaluation, a novel benchmark dataset designed to systematically evaluate audio-visual models across controlled settings. DAVE alleviates existing limitations by (i) ensuring both modalities are necessary to answer correctly and (ii) decoupling evaluation into atomic subcategories. Our detailed analysis of state-of-the-art models reveals specific failure modes and provides targeted insights for improvement. By offering this standardized diagnostic framework, we aim to facilitate more robust development of audio-visual models.
Dataset: https://huggingface.co/datasets/gorjanradevski/dave
Code: https://github.com/gorjanradevski/dave
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Media Analytics and Computing Lab, Xiamen University(厦门大学媒体分析与计算实验室);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究所);School of Informatics, Xiamen University(厦门大学信息学院);Sichuan Agricultural University(四川农业大学);Department of Computer Science, University of Rochester(罗切斯特大学计算机科学系)
MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent
Shandong University(山东大学);University of Science and Technology of China(中国科学技术大学);Central Conservatory of Music(中央音乐学院);Kunlun Tech Co. Ltd.(昆仑万维科技股份有限公司);Shanghai Jiao Tong University(上海交通大学)
A Toolkit for Detecting Spurious Correlations in Speech Datasets
Instituto de Investigación en Ciencias de la Computación, UBA-CONICET(计算机科学研究所,UBA-CONICET);Departamento de Computación, Facultad de Ciencias Exactas y Naturales, UBA(计算系,科学与自然学院,UBA);Facultad de Medicina, Universidad de Chile(医学学院,智利大学);Centro de Neurociencias Cognitivas, Universidad de San Andrés(认知神经科学中心,安德雷斯大学)
Dense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration
Hefei University of Technology(合肥工业大学);University of Science and Technology of China(中国科学技术大学);Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Audio Frequency-Time Dual Domain Evaluation on Depression Diagnosis
Chongqing Engineering Research Center of Medical Electronics and Information Technology, Chongqing University of Posts and Telecommunications(重庆邮电大学医学电子与信息技术重庆市工程研究中心);Newport High School(纽波特高中);Millburn High School(米尔本高中);Basis Independent McLean(贝赛思独立学校麦克莱恩校区);Cupertino High School(库比蒂诺高中);The George Washington University(乔治华盛顿大学)
ESG-Net: Event-Aware Semantic Guided Network for Dense Audio-Visual Event Localization
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications(智能工程与自动化学院,北京邮电大学);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(人工智能学院,北京邮电大学)
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
Department of Electrical and Computer Engineering, Seoul National University(首尔国立大学电气与计算机工程系);Interdisciplinary Program in Artificial Intelligence, Seoul National University(首尔国立大学人工智能跨学科项目);Department of Artificial Intelligence, University of Seoul(首尔大学人工智能系)