arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.08895cs.SDcs.CLeess.AS
收藏

DECAR:用于学习通用音频表示的深度聚类

DECAR: Deep Clustering for learning general-purpose Audio Representations

IIT Madras(印度理工学院马德拉斯分校)

Sreyan Ghosh, Sandesh V Katta, Ashish Seth, S. Umesh

摘要:

我们介绍DECAR,一种用于学习通用音频表示的自监督预训练方法。我们的系统基于聚类:它利用离线聚类步骤来提供目标标签,这些标签充当用于解决预测任务的伪标签。我们在计算机视觉的自监督学习的最新进展之上开发,并设计了一个轻量级,易于使用的自监督预训练方案。我们在大规模Audioset数据集的平衡子集上预训练DECAR嵌入,并将这些表示转移到9个下游分类任务,包括语音,音乐,动物声音和声学场景。此外,我们进行消融研究,确定关键的设计选择,并公开我们所有的代码和预训练模型。

英文摘要:

We introduce DECAR, a self-supervised pre-training approach for learning general-purpose audio representations. Our system is based on clustering: it utilizes an offline clustering step to provide target labels that act as pseudo-labels for solving a prediction task. We develop on top of recent advances in self-supervised learning for computer vision and design a lightweight, easy-to-use self-supervised pre-training scheme. We pre-train DECAR embeddings on a balanced subset of the large-scale Audioset dataset and transfer those representations to 9 downstream classification tasks, including speech, music, animal sounds, and acoustic scenes. Furthermore, we conduct ablation studies identifying key design choices and also make all our code and pre-trained models publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑