基于可微排列的音频表示的自监督学习
Self-Supervised Learning of Audio Representations from Permutations with Differentiable Ranking
摘要：
使用所谓的“借口”任务的自我监督预训练最近在各种各样的模式中表现出令人印象深刻的性能。在这项工作中，我们从排列中推进自监督学习，通过预训练模型来重新排序音频信号的频谱图的混洗部分，以提高下游分类性能。我们做出了两个主要贡献。首先，我们克服了将置换反转集成到端到端训练方案中的主要挑战，使用可微排名的最新进展。在此之前，通过将重新排序任务转换为分类，从根本上减少了可以利用的排列空间，从而回避了这一点。我们的实验验证了从所有可能的排列中学习比使用有限的固定集合提高了预训练表示的质量。其次，我们表明，颠倒排列是以无监督方式学习音频表示的一项有意义的借口任务。特别是，我们通过在时间-频率空间中重新排序谱图补丁来改进乐器分类和音符的音高估计。
英文摘要：
Self-supervised pre-training using so-called "pretext" tasks has recently shown impressive performance across a wide range of modalities. In this work, we advance self-supervised learning from permutations, by pre-training a model to reorder shuffled parts of the spectrogram of an audio signal, to improve downstream classification performance. We make two main contributions. First, we overcome the main challenges of integrating permutation inversions into an end-to-end training scheme, using recent advances in differentiable ranking. This was heretofore sidestepped by casting the reordering task as classification, fundamentally reducing the space of permutations that can be exploited. Our experiments validate that learning from all possible permutations improves the quality of the pre-trained representations over using a limited, fixed set. Second, we show that inverting permutations is a meaningful pretext task for learning audio representations in an unsupervised fashion. In particular, we improve instrument classification and pitch estimation of musical notes by reordering spectrogram patches in the time-frequency space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
