音频编解码器潜在空间中音乐带宽扩展的几何特性研究
On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs
浏览论文内容
中文总结 AI 辅助
本文分析了多种先进方法与简单算术变换在神经编解码器潜在空间中用于音乐带宽扩展的性能，发现简单向量加法可媲美大型扩散模型，建议将其设为研究基准。
中文摘要 AI 辅助
近期音频修复越来越依赖大规模条件潜在生成建模，包括扩散模型、薛定谔桥（Schrödinger Bridges）以及流匹配（Flow Matching）变体，来逆转带宽限制或噪声等退化问题。本文针对多种最先进方法与简单算术变换在多个神经编解码器潜在空间中用于音乐带宽扩展的性能展开分析。研究表明，在参考集上估计干净潜在质心与退化潜在质心之间的单个传输向量，并将其添加到退化潜变量中，可产生与大型扩散模型相当的修复性能。这首先说明，部分神经编解码器的潜在空间呈现与音频带宽对齐的结构；其次，在这类情况下，复杂条件模型相较于简单向量加法可能仅能提供有限增益。本文认为，这些发现为未来研究揭示了一条有趣途径：模型可利用潜在空间结构，以实现更高的训练效率、参数效率及整体更好的性能。此外，本文建议将这种简单算术变换作为音乐带宽扩展研究的基准，因为它有助于评估可学习参数对修复性能的贡献。
英文摘要
Recent audio restoration increasingly relies on large-scale conditional latent generative modeling, including diffusion, Schrodinger Bridges, and Flow Matching variants, to invert degradations such as bandwidth limitation or noise. We present an analysis of the performance of various state-of-the-art methods compared to simple arithmetic transformations in the latent spaces of multiple neural codecs for musical bandwidth extension. We show that estimating a single transport vector between the clean and degraded latent centroids on a reference set, and adding it to degraded latents, can yield restoration performance competitive with large diffusion models. This suggests, first, that some neural codec latent spaces exhibit structure aligned with audio bandwidth; and second, that in such cases complex conditional models may offer only limited gains over a simple vector addition. We argue that these findings reveal an interesting avenue for future research whereby models could take advantage of the latent space structure in order to offer greater training and parameter efficiency, and overall better performance. Additionally, we propose to consider this simple arithmetic transformation as a baseline for music bandwidth extension research, as it allows an assessment of the contribution of learnable parameters towards restoration performance.
