arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.22005cs.SD
收藏

InstructFX2FX: 用于迭代音频效果优化的多轮文本到预设演示

InstructFX2FX: A Multi-Turn Text-to-Effect System for Sequential Audio Effect Refinement

Center for New Music and Audio Technologies (CNMAT)(新音乐与音频技术中心)

Song-Ze Yu, Milan Liessens Dujardin, Yuxuan Cai, Wantong Zhang, Brian Cruz, Jeremy Wagner, Carmine-Emanuele Cella

AI总结:

提出InstructFX2FX,一种多轮文本引导的音频效果优化演示,结合语言模型和CLAP引导优化,实现迭代效果调整,相比纯LLM重提示基线将平均MMD降低约24%。

摘要:

我们提出InstructFX2FX,一个多轮、文本引导的音频效果优化的交互式演示。现有的文本到预设系统大多是单次映射,将一个文本描述符映射到一个预设。而真实的音频工程是顺序的:工程师通过连续指令(如“让它更温暖”或“太刺耳了,柔化它”)来优化现有的效果链。这提出了一个单次系统无法解决的有状态问题:给定当前效果参数和新指令,更新声音同时保留先前指令已实现的效果。InstructFX2FX采用一种混合架构来解决这个问题,该架构在语言模型和CLAP引导优化之间分工。LLM作为高级规划器,选择效果、排序信号链并提出初始参数,其动机是近期证据表明LLM初始化优于从随机起点开始的CLAP优化;然后CLAP引导优化在感知上细化这些参数,提供比每轮重新提示LLM更可控的更新,因为后者往往会扰动用户未要求更改的设置。在演示中,参与者通过连续的自然语言指令驱动干录音,并检查不断演变的效果链、中间优化检查点和会话状态。在SocialFX派生的描述符转换上,相对于仅LLM重提示基线,CLAP引导细化在10个定向描述符对中的9个上降低了目标导向的DSP特征MMD,平均MMD降低了约24%(从0.45降至0.34)。

英文摘要:

We present InstructFX2FX, a system for sequential audio effect refinement through multi-turn natural-language instructions. Existing text-to-effect systems are largely single-shot, mapping one textual descriptor to one preset. Real audio engineering is instead sequential: engineers refine an existing effect chain through successive instructions. This poses a stateful problem that single-shot systems do not address: given the current effect parameters state and a new instruction, update the sound while preserving what earlier instructions already achieved. InstructFX2FX addresses this with a hybrid architecture that divides labor between a language model and CLAP-guided optimization. The LLM serves as a high-level planner that selects effects and proposes the initial parameter state, motivated by recent evidence that LLMs can outperform CLAP-based optimization for single-turn text-to-effect mapping; CLAP-guided optimization then refines the existing parameter state, providing a more stable and robust refinement mechanism than LLM reprompting. In the demo, attendees drive a dry recording through successive natural-language instructions: after each turn, they choose how strongly the effect is applied, then issue the next instruction based on what still differs from the sound they intend. In a preliminary evaluation on SocialFX-derived descriptor pairs, CLAP-guided refinement achieves lower DSP-feature MMD than an LLM+LLM initialize-then-reprompt baseline on 9 of 10 pairs. Trajectory analysis further shows that, for differentiable effects, optimization tends to gradually move the audio toward the new target while retaining the effects of the previous instruction, highlighting the potential for gradual refinement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑