双射信号-符号转换的跨模态变分推断
Cross-modal variational inference for bijective signal-symbol translation
摘要：
从信号中提取符号信息是一个活跃的研究领域，使得许多应用，特别是在音乐信息检索领域。这个复杂的任务，也涉及到其他主题，如音高提取或乐器识别，是一个苛刻的主题，产生了许多方法，主要是基于先进的信号处理算法。然而，这些技术通常是非通用的，允许提取信号的明确的物理属性（音高，倍频程），但不允许任意的词汇表或更一般的注释。最重要的是，这些技术是片面的，这意味着它们可以从音频信号中提取符号数据，但不能执行相反的过程并生成符号到信号。在本文中，我们提出了一个双射的信号/符号转换的方法，把这个问题转化为一个密度估计任务的信号和符号域，考虑相关的随机变量。我们用两个不同的变分自动编码器来估计这个联合分布，每个域一个，其内部表示被迫与加性约束相匹配，允许两个模型分别学习和生成，同时允许信号到符号和符号到信号的推断。在这篇文章中，我们测试我们的模型的音高，八度和动态符号，其中包括音乐转录和标签约束的音频生成的基本步骤。除了它的多功能性之外，这个系统在训练和生成过程中相当轻便，同时允许我们在文章末尾概述的几个有趣的创造性用途。
英文摘要：
Extraction of symbolic information from signals is an active field of research enabling numerous applications especially in the Musical Information Retrieval domain. This complex task, that is also related to other topics such as pitch extraction or instrument recognition, is a demanding subject that gave birth to numerous approaches, mostly based on advanced signal processing-based algorithms. However, these techniques are often non-generic, allowing the extraction of definite physical properties of the signal (pitch, octave), but not allowing arbitrary vocabularies or more general annotations. On top of that, these techniques are one-sided, meaning that they can extract symbolic data from an audio signal, but cannot perform the reverse process and make symbol-to-signal generation. In this paper, we propose an bijective approach for signal/symbol translation by turning this problem into a density estimation task over signal and symbolic domains, considered both as related random variables. We estimate this joint distribution with two different variational auto-encoders, one for each domain, whose inner representations are forced to match with an additive constraint, allowing both models to learn and generate separately while allowing signal-to-symbol and symbol-to-signal inference. In this article, we test our models on pitch, octave and dynamics symbols, which comprise a fundamental step towards music transcription and label-constrained audio generation. In addition to its versatility, this system is rather light during training and generation while allowing several interesting creative uses that we outline at the end of the article.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
