AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）
Sören Becker, Johanna Vielhaben, Marcel Ackermann, Klaus-Robert Müller, Sebastian Lapuschkin, Wojciech Samek
Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation (LRP) to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study.
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）
Data-Free Knowledge Distillation (DFKD) has recently attracted growing attention in the academic community, especially with major breakthroughs in computer vision. Despite promising results, the technique has not been well applied to audio and signal processing. Due to the variable duration of audio signals, it has its own unique way of modeling. In this work, we propose feature-rich audio model inversion (FRAMI), a data-free knowledge distillation framework for general sound classification tasks. It first generates high-quality and feature-rich Mel-spectrograms through a feature-invariant con…
METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark
Institute of Artificial Intelligence (TeleAI), China Telecom（电信人工智能研究院）；Institute of Artificial Intelligence and Robotics(IAIR), Xi’an Jiaotong University（人工智能与机器人研究院）；Advanced Technique of Artificial Intelligence(ATAI), Chongqing University of Technology（人工智能先进技术研究院）
BenSParX: A Robust Explainable Machine Learning Framework for Parkinson's Disease Detection from Bengali Conversational Speech
Chittagong University of Engineering and Technology（吉大港工程技术大学）；East Delta University（东三角洲大学）；Charles Sturt University（查尔斯特大学）；Bangabandhu Sheikh Mujib Medical University（班加班杜·谢赫·穆吉布医科大学）
From Large-scale Audio Tagging to Real-Time Explainable Emergency Vehicle Sirens Detection
Department of Information Engineering, Computer Science and Mathematics (DISIM), University of L’Aquila（信息工程、计算机科学与数学系（DISIM），拉奎拉大学）；National Inter-University Consortium for Telecommunications (CNIT), University of L’Aquila（电信国家大学间联合体（CNIT），拉奎拉大学）
XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts
AudioLab, School of Physics, Engineering and Technology, University of York, United Kingdom（英国约克大学物理工程技术学院音频实验室）；Department of Computer Science, University of York, United Kingdom（英国约克大学计算机科学系）；SEED -- Electronic Arts (EA), Sweden（瑞典艺电SEED部门）；Electronic Arts (EA), United States（美国艺电）
University of Augsburg（奥格斯堡大学）；University of Surrey（萨里大学）；ByteDance AI Lab（字节跳动人工智能实验室）；University of Cambridge（剑桥大学）；Imperial College London（帝国理工学院）
Acoustic Scene Classification (ASC) aims to classify the environment in which the audio signals are recorded. Recently, Convolutional Neural Networks (CNNs) have been successfully applied to ASC. However, the data distributions of the audio signals recorded with multiple devices are different. There has been little research on the training of robust neural networks on acoustic scene datasets recorded with multiple devices, and on explaining the operation of the internal layers of the neural networks. In this article, we focus on training and explaining device-robust CNNs on multi-device acoust…
Unveiling Hidden Factors: Explainable AI for Feature Boosting in Speech Emotion Recognition
University of Québec (TÉLUQ)（魁北克大学（TÉLUQ））；Concordia University（康考迪亚大学）；Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程学院）；University of Québec at Montréal（蒙特利尔魁北克大学）；Centre for Research and Intervention on Suicide, Ethical Issues and End-of-Life Practices（自杀、伦理问题与临终实践研究与干预中心）
APEX: Audio Prototype EXplanations for Classification Tasks
Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland（华沙理工大学人工智能系）；Resemble AI, USA（Resemble AI公司）；IDEAS Research Institute, Poland（波兰IDEAS研究院）；Faculty of Mathematics and Computer Science, Jagiellonian University, Poland（雅盖隆大学数学与计算机科学系）；Doctoral School of Exact and Natural Sciences, Jagiellonian University, Poland（雅盖隆大学博士学院）
XAI-based Comparison of Input Representations for Audio Event Classification
Fraunhofer Heinrich-Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）
After constructing a deep neural network for urban sound classification, this work focuses on the sensitive application of assisting drivers suffering from hearing loss. As such, clear etiology justifying and interpreting model predictions comprise a strong requirement. To this end, we used two different representations of audio signals, i.e. Mel and constant-Q spectrograms, while the decisions made by the deep neural network are explained via layer-wise relevance propagation. At the same time, frequency content assigned with high relevance in both feature sets, indicates extremely discriminat…
Explainability of CNN Based Classification Models for Acoustic Signal
Department of Computer Science, Missouri State University（计算机科学系，密苏里州立大学）；Department of Biology, Missouri State University（生物学系，密苏里州立大学）；Biodiversity Research Institute（生物多样性研究学院）
Density Adaptive Attention-based Speech Network: Enhancing Feature Understanding for Mental Health Disorders
James Silberrad Brown Center for Artificial Intelligence（詹姆斯·西尔伯拉德·布朗人工智能中心）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）；Amazon GenAI（亚马逊生成式人工智能）
A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods
Department of Management, Economics and Industrial Engineering, Politecnico di Milano, Via Lambruschini 24/b, 20156, Milan, Italy（管理、经济与工业工程系，米兰理工学院）；Department of Materials and Mechanical Engineering, University of Turku, Vesilinnantie 5, Turku, 20014, Finland（材料与机械工程系，图尔库大学）；Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC V8P 5C2, Canada（电气与计算机工程系，维多利亚大学）
