arXiv 2605.082502026-05-07cs.CVcs.AI

为何DiT编辑器会漂移？VAE潜在空间中的插件式低频对齐

Why Do DiT Editors Drift? Plug-and-Play Low Frequency Alignment in VAE Latent Space

Tsinghua University（清华大学）；Carnegie Mellon University（卡内基梅隆大学）；Peking University（北京大学）；CASIA；University of Electronic Science and Technology of China（电子科技大学）；Guangming Laboratory（光明实验室）

Xiaoce Wang, Sifan Zhou, Kaifei Wang, Leli Xu, Xuerui Qiu, Tao He, Ming Li

备注：: 9 pages main paper, 12 figures, 25 pages in total

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文从潜在空间频率角度研究DiT编辑器漂移问题，提出无需训练的VAE-LFA方法，通过低频对齐抑制语义漂移并保持高频细节，适用于白盒和黑盒DiT编辑器。

摘要：

最近扩散变换器（DiTs）的进展使单轮图像编辑能力显著提升。然而，多轮编辑常导致渐进式语义漂移和质量退化。本文从潜在空间频率角度研究该问题，将编辑过程分解为VAE和DiT两个功能组件。通过系统分析VAE潜在空间，发现DiT引入主导的低频漂移，随着编辑轮次增加而累积，而VAE贡献相对稳定的重建偏差。基于此，提出VAE-LFA（低频对齐），一种无需训练、即插即用的方法，在VAE潜在空间中进行对齐。VAE-LFA通过低通滤波分解编辑轮次间的潜在差异，并将低频统计对齐到先前轮次的指数移动平均，有效抑制累积语义漂移并保留高频细节。该方法无需重新训练、真实先验或扩散参数访问，适用于白盒和黑盒DiT编辑器。对于白盒模型，VAE-LFA无缝集成到编辑流程中，通过消除冗余的VAE轮次；对于黑盒模型，通过现成的VAE进行轮次间潜在对齐。大量实验表明，VAE-LFA在多样化的多轮编辑场景中提升了语义一致性和视觉保真度，包括受控和真实场景中的图像。

英文摘要：

Recent advances in diffusion transformers (DiTs) have enabled promising single-turn image editing capabilities. However, multi-turn editing often leads to progressive semantic drift and quality degradation.In this work, we study this problem from a latent-space frequency perspective by decomposing the editing process into two functional components: VAE and DiT. Through systematic analysis in the VAE latent space, we uncover that the DiT introduces dominant low-frequency drift that accumulates as semantic misalignment across editing rounds, while the VAE contributes comparatively stable reconstruction bias.Based on this insight, we propose VAE-LFA (Low Frequency Alignment), a training-free, plug-and-play method that performs alignment in VAE latent space. VAE-LFA decomposes latent discrepancies across editing rounds via low-pass filtering, and aligns low-frequency statistics to an exponential moving average of previous rounds, effectively suppressing accumulated semantic drift while preserving high-frequency details.Our method requires no retraining, ground-truth priors, or access to diffusion parameters, making it applicable to both white-box and black-box DiT editors. For white-box models, VAE-LFA is seamlessly integrated into the editing pipeline by eliminating redundant VAE round trips; for black-box models, it operates via an off-the-shelf VAE to perform inter-round latent alignment.Extensive experiments demonstrate that VAE-LFA improves semantic consistency and visual fidelity across diverse multi-turn editing scenarios, including both controlled and in-the-wild images.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。