arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.11910cs.SDcs.LG
收藏

TADA! 通过激活引导调整音频扩散模型

TADA! Tuning Audio Diffusion Models through Activation Steering

Warsaw University of Technology(华沙技术大学);IDEAS Research Institute(IDEAS研究院)

Łukasz Staniszewski, Katarzyna Zaleska, Mateusz Modrzejewski, Kamil Deja

AI总结:

本文通过激活引导技术揭示音频扩散模型中的语义瓶颈,并展示了局部激活引导在音频概念调节中的新状态-of-the-art性能。

摘要:

音频扩散模型能够从文本生成高质量的音乐,但实现对特定音乐属性的精细控制仍然具有挑战性,因为其内部机制对高级概念的表示尚不明确。在本文中,我们利用激活修补技术证明,最近的音频扩散架构存在语义瓶颈,其中一小部分连续的注意力层控制不同的音乐概念,例如特定乐器、人声或音乐类型的存在。在此基础上,我们系统地评估了广泛的应用引导方法,比较了激活引导与提示级、乐谱空间和权重空间干预,分析了引导机制与干预位置之间的相互作用。我们的新基准,通过广泛的用户研究支持,证明了局部激活引导在音频概念调节中建立了新的状态-of-the-art性能。

英文摘要:

Audio diffusion models can synthesize high-fidelity music from text, yet achieving fine-grained control over specific musical attributes remains challenging, as their internal mechanisms for representing high-level concepts are poorly understood. In this work, we use activation patching to demonstrate that recent audio diffusion architectures exhibit a semantic bottleneck, where a small, shared subset of consecutive attention layers controls distinct musical concepts, such as the presence of specific instruments, vocals, or genres. Building on this, we systematically evaluate a broad spectrum of steering paradigms, comparing activation steering against prompt-level, score-space, and weight-space interventions, analyzing the interaction between the steering mechanism and the intervention site. Our new benchmark, supported by an extensive user study, demonstrates that localized activation steering establishes a new state-of-the-art in audio concept modulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑