Few-shot learning is a type of classification through which predictions are made based on a limited number of samples for each class. This type of classification is sometimes referred to as a meta-learning problem, in which the model learns how to learn to identify rare cases. We seek to extract information from five exemplar vocalisations of mammals or birds and detect and classify these sounds in field recordings [2]. This task was provided in the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge of 2021. Rather than utilize deep learning, as is most commonly done, we formulated a novel solution using only machine learning methods. Various models were tested, and it was found that logistic regression outperformed both linear regression and template matching. However, all of these methods over-predicted the number of events in the field recordings.
Few-shot bioacoustic event detection is a task that detects the occurrence time of a novel sound given a few examples. Previous methods employ metric learning to build a latent space with the labeled part of different sound classes, also known as positive events. In this study, we propose a segment-level few-shot learning framework that utilizes both the positive and negative events during model optimization. Training with negative events, which are larger in volume than positive events, can increase the generalization ability of the model. In addition, we use transductive inference on the val…
Few-shot audio event detection is a task that detects the occurrence time of a novel sound class given a few examples. In this work, we propose a system based on segment-level metric learning for the DCASE 2022 challenge of few-shot bioacoustic event detection (task 5). We make better utilization of the negative data within each sound class to build the loss function, and use transductive inference to gain better adaptation on the evaluation set. For the input feature, we find the per-channel energy normalization concatenated with delta mel-frequency cepstral coefficients to be the most effect…
Adaptive Learning via a Negative Selection Strategy for Few-Shot Bioacoustic Event Detection
School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；Sanya Science and Education Innovation Park, Wuhan University of Technology（武汉理工大学三亚科教创新园）
Queen Mary University of London（伦敦玛丽女王大学）；Nantes Université（南特大学）；École Centrale Nantes（南特中央理工学院）；CNRS（法国国家科学研究中心）；University of Konstanz（康斯坦茨大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；BIOTOPIA Naturkundemuseum Bayern（巴伐利亚BIOTOPIA自然历史博物馆）；AGH University of Science and Technology（AGH科技大学）；Cornell University（康奈尔大学）；Tilburg University（蒂尔堡大学）；Naturalis Biodiversity Centre（Naturalis生物多样性中心）；University Ramon Llull（拉蒙·柳利大学）；Oxford University（牛津大学）；Syracuse University（雪城大学）；University of Salford（索尔福德大学）；University of Surrey（萨里大学）；Sonantic Limited（Sonantic有限公司）
Few-shot sound event detection is the task of detecting sound events, despite having only a few labelled examples of the class of interest. This framework is particularly useful in bioacoustics, where often there is a need to annotate very long recordings but the expert annotator time is limited. This paper presents an overview of the second edition of the few-shot bioacoustic sound event detection task included in the DCASE 2022 challenge. A detailed description of the task objectives, dataset, and baselines is presented, together with the main results obtained and characteristics of the subm…
Few-shot bioacoustic event detection at the DCASE 2023 challenge
Queen Mary University of London（伦敦玛丽女王大学）；Naturalis Biodiversity Centre（Naturalis生物多样性中心）；Ramon Llull University（拉蒙·柳利大学）；University of Konstanz（康斯坦茨大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Oxford University（牛津大学）；Syracuse University（雪城大学）；University of Surrey（萨里大学）；University of Salford（索尔福德大学）；Naturkundemuseum Bayern（巴伐利亚自然历史博物馆）；AGH University of Science and Technology（AGH科技大学）；Nantes Université（南特大学）；École Centrale Nantes（南特中央理工学院）；CNRS（法国国家科学研究中心）；Tilburg University（蒂尔堡大学）
animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics
Department for the Ecology of Animal Societies, Max Planck Institute of Animal Behavior（动物社会生态学部，动物行为马克斯·普朗克研究所）；Department of Biology, University of Konstanz（康斯坦茨大学生物系）；Centre for the Advanced Study of Collective Behaviour, University of Konstanz（集体行为高级研究所以及康斯坦茨大学）；Kalahari Research Centre, Van Zylsrus, Northern Cape, South Africa（卡拉哈里研究中心，南非北开普省范·齐尔鲁斯）；Royal Botanic Gardens Kew, Richmond, UK（皇家植物园肯特，英国里士满）；Zukunftskolleg, University of Konstanz（未来学院，康斯坦茨大学）；Department of Evolutionary Biology and Environmental Studies, University of Zurich（进化生物学与环境研究部，苏黎世大学）；Department of Cognitive Science and Artificial Intelligence, Tilburg University（认知科学与人工智能系，蒂尔堡大学）；Naturalis Biodiversity Center, Leiden, The Netherlands（自然史博物馆生物多样性研究中心，莱顿，荷兰）；Interdisciplinary Center for the Evolution of Language, University of Zurich（语言进化跨学科研究中心，苏黎世大学）；Department of Computer Science, San Diego State University（计算机科学系，圣地亚哥州立大学）
Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics
University College London（伦敦大学学院）；Google DeepMind（谷歌DeepMind）；Zoological Society of London（伦敦动物学会）；Conservation Metrics Inc.（保护指标公司）；University of New Hampshire（新罕布什尔大学）；K. Lisa Yang Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University（康奈尔大学鸟类学实验室K. Lisa Yang保护生物声学中心）；Department of Public and Ecosystem Health, Cornell University（康奈尔大学公共与生态系统健康系）；Sound Ocean Science（海洋声学科学）；University of Exeter（埃克塞特大学）；University of Bristol（布里斯托大学）；IPB University（IPB大学）；Google Research（谷歌研究院）
Learning to detect an animal sound from five examples
Queen Mary University of London（伦敦玛丽女王大学）；Nantes Universite, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004（南特大学，南特中央理工大学，法国国家科学研究中心，LS2N，UMR 6004）；University of Konstanz（康斯坦茨大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Universitat Ramon Llull（拉蒙·尤以大学）；BIOTOPIA Naturkundemuseum Bayern（巴伐利亚州自然博物馆BIOTOPIA）；AGH University of Science and Technology（AGH科技大学）；University of Salford（索尔福德大学）；University of Surrey（萨里大学）；Aarhus University（奥胡斯大学）；Syracuse University（雪城大学）；Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）
This work introduces the one-shot learning paradigm in the computational bioacoustics domain. Even though, most of the related literature assumes availability of data characterizing the entire class dictionary of the problem at hand, that is rarely true as a habitat's species composition is only known up to a certain extent. Thus, the problem needs to be addressed by methodologies able to cope with non-stationarity. To this end, we propose a framework able to detect changes in the class dictionary and incorporate new classes on the fly. We design an one-shot learning architecture composed of a…
Peking University（北京大学）；University of Surrey（萨里大学）
Although prototypical network (ProtoNet) has proved to be an effective method for few-shot sound event detection, two problems still exist. Firstly, the small-scaled support set is insufficient so that the class prototypes may not represent the class center accurately. Secondly, the feature extractor is task-agnostic (or class-agnostic): the feature extractor is trained with base-class data and directly applied to unseen-class data. To address these issues, we present a novel mutual learning framework with transductive learning, which aims at iteratively updating the class prototypes and featu…
Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Western Washington University（西华盛顿大学）；DocuSign（DocuSign公司）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
This report presents deep learning and data augmentation techniques used by a system entered into the Few-Shot Bioacoustic Event Detection for the DCASE2021 Challenge. The remit was to develop a few-shot learning system for animal (mammal and bird) vocalisations. Participants were tasked with developing a method that can extract information from five exemplar vocalisations, or shots, of mammals or birds and detect and classify sounds in field recordings. In the system described in this report, prototypical networks are used to learn a metric space, from which classification is performed by com…
We study few-shot acoustic event detection (AED) in this paper. Few-shot learning enables detection of new events with very limited labeled data. Compared to other research areas like computer vision, few-shot learning for audio recognition has been under-studied. We formulate few-shot AED problem and explore different ways of utilizing traditional supervised methods for this setting as well as a variety of meta-learning approaches, which are conventionally used to solve few-shot classification problem. Compared to supervised baselines, meta-learning models achieve superior performance, thus s…
Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation
College of Artificial Intelligence, China University of Petroleum（中国石油大学（北京）人工智能学院）；College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所）
South China University of Technology（华南理工大学）；Guangdong Polytechnic of Science and Technology（广东科学技术职业学院）
Although few-shot learning has attracted much attention from the fields of image and audio classification, few efforts have been made on few-shot speaker identification. In the task of few-shot learning, overfitting is a tough problem mainly due to the mismatch between training and testing conditions. In this paper, we propose a few-shot speaker identification method which can alleviate the overfitting problem. In the proposed method, the model of a depthwise separable convolutional network with channel attention is trained with a prototypical loss function. Experimental datasets are extracted…
Recently, few-shot voice cloning has achieved a significant improvement. However, most models for few-shot voice cloning are single-modal, and multi-modal few-shot voice cloning has been understudied. In this paper, we propose to use multi-modal learning to improve the few-shot voice cloning performance. Inspired by the recent works on unsupervised speech representation, the proposed multi-modal system is built by extending Tacotron2 with an unsupervised speech representation module. We evaluate our proposed system in two few-shot voice cloning scenarios, namely few-shot text-to-speech(TTS) an…
