PEACE：DSP效果代码与音频的联合嵌入
PEACE: Joint Embeddings of DSP Effects Code and Audio
浏览论文内容
中文总结 AI 辅助
PEACE首次联合嵌入音频效果代码与音频，利用Faust代码编码器，在音频到代码检索中恢复链拓扑，优于预训练模型，奠定DSP代码音乐检索基础。
中文摘要 AI 辅助
本文介绍了PEACE，这是首个音频效果代码与输出音频的联合嵌入。基于SLAP的多模态目标，我们将一个AFx-Rep音频编码器与两个用于Faust（一种用于音频信号处理的函数式语言）的代码编码器配对。首先，我们评估了一个在Faust源代码上微调的T5 transformer。其次，我们评估了一个在Faust编译器中间表示上的消息传递图神经网络，该网络同时捕获拓扑结构和UI参数。我们在音频到代码检索任务上进行评估，其中在推理时遮蔽UI参数会生成仅编码效果链拓扑的嵌入。当参数可见时，两个代码编码器在混合长度链的检索上持平，但在单效果画廊上存在权衡。当参数完全遮蔽时，PEACE恢复有序链拓扑的能力远超随机水平，且没有监督方法的局限性。PEACE在分布外混响检索基准上优于预训练模型，并能改进冻结的仅音频表示。其对拓扑和参数的双重理解，为搜索、生成和基于DSP代码进行条件控制的音乐信息检索系统奠定了基础。
英文摘要
This paper introduces PEACE, the first joint embedding of audio effect code and output audio. Building on SLAP's multimodal objective, we pair an AFx-Rep audio encoder with two code encoders for Faust, a functional language for audio signal processing. First, we evaluate a fine-tuned T5 transformer over Faust source code. Second, we evaluate a message-passing graph neural network over an intermediate representation of the Faust compiler, capturing both topology and UI parameters. We evaluate on audio-to-code retrieval, where masking UI parameters at inference yields embeddings that encode effect chain topology alone. When parameters are visible, the two code encoders tie on retrieval of mixed-length chains but have tradeoffs on single-effect galleries. With parameters fully masked, PEACE recovers ordered chain topology far above chance without the limitations of supervised methods. PEACE outperforms pretrained models on an out-of-distribution reverb retrieval benchmark and can improve frozen audio-only representations. Its dual understanding of topology and parameters lays the groundwork for music information retrieval systems that search, generate, and condition on DSP code.
