arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.01879cs.SDeess.AS
收藏

低复杂度音频嵌入提取器

Low-Complexity Audio Embedding Extractors

Florian Schmid, Khaled Koutini, Gerhard Widmer

AI总结:

本研究通过知识蒸馏训练高效MobileNets作为通用音频嵌入提取器,降低计算成本,证明低复杂度模型能生成有竞争力的嵌入,适用于边缘设备上的多任务音频分析。

摘要:

使用深度学习模型解决说话人识别、音乐分类或语义音频事件标记等任务通常需要计算密集的网络。通用音频嵌入(GPAEs)是音频信号的密集表示,允许轻量级的浅层分类器处理各种音频任务。其思路是,一个单一的复杂特征提取器提取密集的GPAEs,而浅层MLP可以产生特定任务的预测。如果提取的密集表示足够通用,使得简单的下游分类器能够泛化到音频领域的各种任务,那么一次昂贵的正向传播就足以并行解决多个任务。在这项工作中,我们尝试降低GPAE提取器的成本,使其适用于资源受限的设备。我们使用在AudioSet上训练的、通过从Transformer集成进行知识蒸馏的高效MobileNets作为高效的GPAE提取器。我们探索如何从模型中获取高质量的GPAEs,研究模型复杂度与提取的GPAEs质量之间的关系,并得出结论:低复杂度模型可以生成具有竞争力的GPAEs,为在边缘设备上分析音频流以应对多种音频分类和识别任务铺平了道路。

英文摘要:

Solving tasks such as speaker recognition, music classification, or semantic audio event tagging with deep learning models typically requires computationally demanding networks. General-purpose audio embeddings (GPAEs) are dense representations of audio signals that allow lightweight, shallow classifiers to tackle various audio tasks. The idea is that a single complex feature extractor would extract dense GPAEs, while shallow MLPs can produce task-specific predictions. If the extracted dense representations are general enough to allow the simple downstream classifiers to generalize to a variety of tasks in the audio domain, a single costly forward pass suffices to solve multiple tasks in parallel. In this work, we try to reduce the cost of GPAE extractors to make them suitable for resource-constrained devices. We use efficient MobileNets trained on AudioSet using Knowledge Distillation from a Transformer ensemble as efficient GPAE extractors. We explore how to obtain high-quality GPAEs from the model, study how model complexity relates to the quality of extracted GPAEs, and conclude that low-complexity models can generate competitive GPAEs, paving the way for analyzing audio streams on edge devices w.r.t. multiple audio classification and recognition tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑