arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.10046cs.SDcs.AI
收藏

潜流内部:音频分离基础模型中注意力动力学的因果解读

Inside the Latent Flow: Causal Deciphering of Attention Dynamics in Audio Separation Foundation Models

Jilin University(吉林大学);Hunan University(湖南大学);University of Electronic Science and Technology of China(电子科学与技术大学)

Yuxuan Chen, Haoyuan Yu, Peize He

AI总结:

本文通过因果干预协议揭示流匹配Transformer在音频分离中的双路径注意力机制,并提出无训练加速方法LSAC,在保持质量的同时减少约25%自注意力计算。

摘要:

流匹配变压器实现了强大的音频分离,但其注意力动力学是不透明的。我们将已建立的因果干预原则适应为SAM Audio的确定性推理时探测协议。正交探测揭示了一种双路径文本条件机制:加法注入控制语义身份,而交叉注意力细化声学结构。我们观察到异步逐层收敛:稳定层早期构建时间支架,而快速层在采样过程中继续解决伪影。该模型还减弱时间分割线索以维持连续流稳定性。利用这些见解,我们提出了层选择性注意力缓存(LSAC),一种无训练加速方法,在稳定层中缓存注意力。在各种声学复杂度下,LSAC将自注意力计算减少约25%,质量损失可忽略,并且与朴素步长减少相比,质量保持率高达6.7倍。

英文摘要:

Flow-matching transformers achieve strong audio separation, yet their attention dynamics are opaque. We adapt established causal-intervention principles into a deterministic, inference-time probing protocol for SAM Audio. Orthogonal probing uncovers a dual-pathway text-conditioning mechanism: additive injections control semantic identity, while cross-attention refines acoustic structure. We observe an asynchronous layerwise convergence: stable layers build temporal scaffolds early, whereas fast layers continue resolving artifacts during sampling. The model also attenuates temporal segmentation cues to maintain continuous-flow stability. Using these insights, we propose Layer-Selective Attention Caching (LSAC), a training-free acceleration method that caches attention in stable layers. Across acoustic complexities, LSAC cuts self-attention computation by about ~25% with negligible quality loss and yields up to 6.7x higher quality retention than naive step reduction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑