We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We analyze the performance of the submissions to DCASE 2021 task 4 depending on time related modifications (time position of an event and length of clips) and we study the impact of non-target sound events and reverberation. We show that the localization in time of sound events is still a problem for SED systems. We also show that reverberation and non-target sound events are severely degrading the performance of the SED systems. In the latter case, sound separation seems like a promising solution.
We investigate an method for quantifying city characteristics based on impressions of a sound environment. The quantification of the city characteristics will be beneficial to government policy planning, tourism projects, etc. In this study, we try to predict two soundscape impressions, meaning pleasantness and eventfulness, using sound data collected by the cloud-sensing method. The collected sounds comprise meta information of recording location using Global Positioning System. Furthermore, the soundscape impressions and sound-source features are separately assigned to the cloud-sensing soun…
Simon Fraser University（西蒙菲莎大学）；Academia Sinica（中央研究院）；Dartmouth College（达特茅斯学院）
Whether literally or suggestively, the concept of soundscape is alluded in both modern and ancient music. In this study, we examine whether we can analyze and compare Western and Chinese classical music based on soundscape models. We addressed this question through a comparative study. Specifically, corpora of Western classical music excerpts (WCMED) and Chinese classical music excerpts (CCMED) were curated and annotated with emotional valence and arousal through a crowdsourcing experiment. We used a sound event detection (SED) and soundscape emotion recognition (SER) models with transfer lear…
Fraunhofer Institute for Digital Media Technology（弗劳恩霍夫数字媒体技术研究所）
Research on sound event detection (SED) in environmental settings has seen increased attention in recent years. The large amounts of (private) domestic or urban audio data needed raise significant logistical and privacy concerns. The inherently distributed nature of these tasks, make federated learning (FL) a promising approach to take advantage of largescale data while mitigating privacy issues. While FL has also seen increased attention recently, to the best of our knowledge there is no research towards FL for SED. To address this gap and foster further research in this field, we create and …
The impact of non-target events in synthetic soundscapes for sound event detection
Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Loria（洛林计算机科学实验室）；Department of Information Engineering, Universita Politecnica delle Marche（马尔凯理工大学信息工程系）
Misophonia Trigger Sound Detection on Synthetic Soundscapes Using a Hybrid Model with a Frozen Pre-Trained CNN and a Time-Series Module
Department of Computer Science, Nagoya Institute of Technology（名古屋技术大学计算机科学系）；International Research Center for Neurointelligence, The University of Tokyo（东京大学神经智能国际研究中心）
Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms
Music and Audio Research Laboratory, New York University（纽约大学音乐与音频研究实验室）；Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院）
Training a sound event detection algorithm on a heterogeneous dataset including both recorded and synthetic soundscapes that can have various labeling granularity is a non-trivial task that can lead to systems requiring several technical choices. These technical choices are often passed from one system to another without being questioned. We propose to perform a detailed analysis of DCASE 2020 task 4 sound event detection baseline with regards to several aspects such as the type of data used for training, the parameters of the mean-teacher or the transformations applied while generating the sy…
Self Training and Ensembling Frequency Dependent Networks with Coarse Prediction Pooling and Sound Event Bounding Boxes
Institute of Civil Military Technology Cooperation（军民技术合作研究所）；Defense Acquisition Program Administration（国防采计局）；Ministry of Trade, Industry and Energy（产业通商资源部）；Korea Research Institute of Ships and Ocean engineering（韩国船舶海洋工程研究院）；Ministry of Oceans and Fisheries（海洋水产部）
New York University（纽约大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Universidade Federal Fluminense（弗鲁米嫩塞联邦大学）；New Jersey Institute of Technology（新泽西理工学院）
Noise is one of the primary quality-of-life issues in urban environments. In addition to annoyance, noise negatively impacts public health and educational performance. While low-cost sensors can be deployed to monitor ambient noise levels at high temporal resolutions, the amount of data they produce and the complexity of these data pose significant analytical challenges. One way to address these challenges is through machine listening techniques, which are used to extract features in attempts to classify the source of noise and understand temporal patterns of a city's noise situation. However,…
This paper introduces a novel dataset for polyphonic sound event detection in urban sound monitoring use-cases. Based on isolated sounds taken from the FSD50k dataset, 20,000 polyphonic soundscapes are synthesized with sounds being randomly positioned in the stereo panorama using different loudness levels. The paper gives a detailed discussion of possible application scenarios, explains the dataset generation process in detail, and discusses current limitations of the proposed USM-SED dataset.
Time-frequency localization of bird calls in dense soundscapes
Acoustic Research Laboratory, National University of Singapore（新加坡国立大学声学研究实验室）；Tropical Marine Science Institute, National University of Singapore（新加坡国立大学热带海洋科学研究所）；School of Marine Science and Technology, Northwestern Polytechnical University（西北工业大学航海学院）
University of Chinese Academy of Sciences（中国科学院大学）
Recently, convolutional neural networks (CNNs) have been widely used in sound event detection (SED). However, traditional convolution is deficient in learning time-frequency domain representation of different sound events. To address this issue, we propose multi-dimensional frequency dynamic convolution (MFDConv), a new design that endows convolutional kernels with frequency-adaptive dynamic properties along multiple dimensions. MFDConv utilizes a novel multi-dimensional attention mechanism with a parallel strategy to learn complementary frequency-adaptive attentions, which substantially stren…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
The performances of Sound Event Detection (SED) systems are greatly limited by the difficulty in generating large strongly labeled dataset. In this work, we used two main approaches to overcome the lack of strongly labeled data. First, we applied heavy data augmentation on input features. Data augmentation methods used include not only conventional methods used in speech/audio domains but also our proposed method named FilterAugment. Second, we propose two methods to utilize weak predictions to enhance weakly supervised SED performance. As a result, we obtained the best PSDS1 of 0.4336 and bes…
University of Texas at Dallas（德克萨斯大学达拉斯分校）；NEC Laboratories America（NEC美国实验室）
Sound Event Early Detection (SEED) is an essential task in recognizing the acoustic environments and soundscapes. However, most of the existing methods focus on the offline sound event detection, which suffers from the over-confidence issue of early-stage event detection and usually yield unreliable results. To solve the problem, we propose a novel Polyphonic Evidential Neural Network (PENet) to model the evidential uncertainty of the class probability with Beta distribution. Specifically, we use a Beta distribution to model the distribution of class probabilities, and the evidential uncertain…
Soundscape Captioning using Sound Affective Quality Network and Large Language Model
Machine Learning Group, University of Oxford（牛津大学机器学习组）；AIRO-IDLab, Ghent University-Imec（根特大学-imec AIRO-IDLab）；CVSSP, University of Surrey（萨里大学计算机视觉与模式识别中心）；University College London（伦敦大学学院）；WAVES Group, Ghent University（根特大学WAVES组）
Generalization in birdsong classification: impact of transfer learning methods and dataset characteristics
Naturalis Biodiversity Center（自然生物多样性中心）；Xeno-Canto Foundation（异音基金会）；Landesbund fur Vogel- und Naturschutz（鸟类与自然保护州联合会）；Department of Cognitive Science and Artificial Intelligence, Tilburg University（蒂尔堡大学认知科学与人工智能系）
Automating Urban Soundscape Enhancements with AI: In-situ Assessment of Quality and Restorativeness in Traffic-Exposed Residential Areas
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；Center for Music Technology, Georgia Institute of Technology（佐治亚理工学院音乐技术中心）；Building & Research Institute, Housing & Development Board（建屋发展局建筑与研究院）；Department of Architectural Engineering, Chungnam National University（忠南大学建筑工程系）；UCL Institute for Environmental Design and Engineering, The Bartlett, University College London（伦敦大学学院巴特lett环境设计与工程学院）；Nanyang Business School, Nanyang Technological University（南洋理工大学南洋商学院）
Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detection
University of Southern California（南加州大学）；Universidad Nacional Autónoma de México（墨西哥国立自治大学）；Universidad Autónoma de Tamaulipas（塔毛利帕斯自治大学）；Queen Mary University of London（伦敦大学玛丽皇后学院）
New York University（纽约大学）；New Jersey Institute of Technology（新泽西理工学院）；Northwestern University（西北大学）
While the estimation of what sound sources are, when they occur, and from where they originate has been well-studied, the estimation of how loud these sound sources are has been often overlooked. Current solutions to this task, which we refer to as source-specific sound level estimation (SSSLE), suffer from challenges due to the impracticality of acquiring realistic data and a lack of robustness to realistic recording conditions. Recently proposed weakly supervised source separation offer a means of leveraging clip-level source annotations to train source separation models, which we augment wi…
Joint Prediction of Audio Event and Annoyance Rating in an Urban Soundscape by Hierarchical Graph Representation Learning
Ghent University（根特大学）；University of Leicester（莱斯特大学）；Shenzhen University（深圳大学）；University College London（伦敦大学学院）；Ghent University-Imec（根特大学-imec）；University of Surrey（萨里大学）
University of Chinese Academy of Sciences（中国科学院大学）；Xi'an Jiaotong Liverpool University（西交利物浦大学）；ByteDance（字节跳动）；University of Surrey（萨里大学）
Sound event localization and detection (SELD) is a joint task of sound event detection and direction-of-arrival estimation. In DCASE 2022 Task 3, types of data transform from computationally generated spatial recordings to recordings of real-sound scenes. Our system submitted to the DCASE 2022 Task 3 is based on our previous proposed Event-Independent Network V2 (EINV2) with a novel data augmentation method. Our method employs EINV2 with a track-wise output format, permutation-invariant training, and a soft parameter-sharing strategy, to detect different sound events of the same class but in d…
MTDA-HSED: Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）
Spike Encoding for Environmental Sound: A Comparative Benchmark
Audio and Neuromorphic Processing Group, Instituto Tecnologico de Informatica (ITI), Valencia, Spain（音频与神经形态处理组，技术信息研究所（ITI），西班牙瓦伦西亚）；Department of Computer Science, Universitat de Valencia, 46100 Burjassot, Spain（计算机科学系，瓦伦西亚大学，西班牙布尔贾索特）
Environmental Sound Deepfake Detection Challenge: An Overview
1 School of Electrical Engineering, KAIST, Daejeon, Republic of Korea 2 University of Melbourne, Australia, 3 Fortemedia Singapore, Singapore, 4 Xi’an University of Posts \& Telecommunications, Xi’an, China, 5 Xi'an Lianfeng Acoustic Technologies Co., Ltd., China
ESDD 2026: Environmental Sound Deepfake Detection Challenge Evaluation Plan
1 School of Electrical Engineering, KAIST, Daejeon, Republic of Korea 2 University of Melbourne, Australia 3 Fortemedia Singapore, Singapore 4 Xi'an University of Posts \& Telecommunications, Xi'an, China
