arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.01253cs.SDcs.AIeess.ASq-bio.QMstat.AP
收藏

animal2vec 和 MeerKAT:一种自监督的 transformer 模型用于稀有事件原始音频输入以及一个大规模的生物声学参考数据集

animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics

Department for the Ecology of Animal Societies, Max Planck Institute of Animal Behavior(动物社会生态学部,动物行为马克斯·普朗克研究所);Department of Biology, University of Konstanz(康斯坦茨大学生物系);Centre for the Advanced Study of Collective Behaviour, University of Konstanz(集体行为高级研究所以及康斯坦茨大学);Kalahari Research Centre, Van Zylsrus, Northern Cape, South Africa(卡拉哈里研究中心,南非北开普省范·齐尔鲁斯);Royal Botanic Gardens Kew, Richmond, UK(皇家植物园肯特,英国里士满);Zukunftskolleg, University of Konstanz(未来学院,康斯坦茨大学);Department of Evolutionary Biology and Environmental Studies, University of Zurich(进化生物学与环境研究部,苏黎世大学);Department of Cognitive Science and Artificial Intelligence, Tilburg University(认知科学与人工智能系,蒂尔堡大学);Naturalis Biodiversity Center, Leiden, The Netherlands(自然史博物馆生物多样性研究中心,莱顿,荷兰);Interdisciplinary Center for the Evolution of Language, University of Zurich(语言进化跨学科研究中心,苏黎世大学);Department of Computer Science, San Diego State University(计算机科学系,圣地亚哥州立大学)
Julian C. Schäfer-Zimmermann, Vlad Demartsev, Baptiste Averly, Kiran Dhanjal-Adams, Mathieu Duteil, Gabriella Gall, Marius Faiß, Lily Johnson-Ulrich, Dan Stowel… 展开作者

Julian C. Schäfer-Zimmermann, Vlad Demartsev, Baptiste Averly, Kiran Dhanjal-Adams, Mathieu Duteil, Gabriella Gall, Marius Faiß, Lily Johnson-Ulrich, Dan Stowell, Marta B. Manser, Marie A. Roch, Ariana Strandburg-Peshkin

AI总结:

本文提出 animal2vec 模型和 MeerKAT 数据集,通过自监督学习处理稀有生物声音数据,提升生物声学研究中数据稀缺情况下的分析能力。

摘要:

生物声学研究对于理解动物行为、保护和生态至关重要,但分析大规模数据集时动物叫声稀有是一个巨大挑战。尽管深度学习技术日益普及,但将其应用于生物声学仍然困难。我们提出 animal2vec,一种可解释的大型 transformer 模型,并针对稀疏和不平衡的生物声学数据设计了自监督训练方案。该模型从未标记音频中学习,然后通过标记数据进一步优化。此外,我们引入并公开发布 MeerKAT:Meerkat Kalahari Audio Transcripts,一个包含毫秒级注释的 meerkat(Suricata suricatta)叫声数据集,目前是最大的非人类陆地哺乳动物标注数据集。我们的模型在 MeerKAT 和公开的 NIPS4Bplus 鸟鸣数据集上均优于现有方法。此外,animal2vec 即使在少量标注数据(少样本学习)下也能表现良好。animal2vec 和 MeerKAT 为生物声学研究提供了新的参考点,使科学家能够在少量真实信息的情况下分析大量数据。

英文摘要:

Bioacoustic research, vital for understanding animal behavior, conservation, and ecology, faces a monumental challenge: analyzing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this with animal2vec, an interpretable large transformer model, and a self-supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabeled audio and then refines its understanding with labeled data. Furthermore, we introduce and publicly release MeerKAT: Meerkat Kalahari Audio Transcripts, a dataset of meerkat (Suricata suricatta) vocalizations with millisecond-resolution annotations, the largest labeled dataset on non-human terrestrial mammals currently available. Our model outperforms existing methods on MeerKAT and the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labeled data (few-shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyze large amounts of data even with scarce ground truth information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑