合并哪些Token？用于高效图像生成的扩散动力学
Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation
AI总结：
本研究提出DaTMe框架，基于扩散动力学构建重要性图，自适应合并Token，在PixArt-a和FLUX实验中保持图像视觉保真度，实现高效图像生成。
摘要：
Token合并已成为一种有效的无训练策略，通过减少冗余Token计算来加速扩散模型，同时最小化生成质量下降。一个关键挑战是确定哪些Token可以安全合并，哪些应保留用于后续计算。现有方法通过保留与提示对齐的前景区域来解决这一问题，从而保护语义相关结构。然而，这种文本相关性标准忽略了背景对象和提示中未明确提及的局部细节，导致结构连贯性下降。在本研究中，我们提出了Dynamics-aware Token Merging（DaTMe），这是一种无训练框架，将Token合并基于底层扩散动力学。我们的核心见解是，Token重要性应反映其在去噪过程中是否仍未解决，以及是否携带结构信息。为此，DaTMe从Tweedie去噪估计中构建了一个动力学感知重要性图。该图结合了时间信号（测量预测干净图像的步间变化）和结构信号（捕获局部高频内容，如边界和精细细节）。使用该图，DaTMe在给定压缩率下自适应地将Token分配到合并角色：重要Token作为锚点保留，而时间稳定且结构均匀的Token被选为安全合并候选。在PixArt-a和FLUX上的实验表明，DaTMe在整个图像中保持了视觉保真度，凸显了扩散动力学作为高效生成中Token级合并决策的有效性。我们的代码可在此https URL获取。
英文摘要：
Token merging has emerged as an effective training-free strategy for accelerating diffusion models by reducing redundant token computation while minimizing generation degradation. A key challenge is determining which tokens can be safely merged and which should be preserved for subsequent computation. Existing method addresses this by preserving prompt-aligned foreground regions, thereby protecting semantically relevant structures. However, this text-relevance criterion overlooks background objects, local details that are not explicitly mentioned in the prompt, leading to degraded structural coherence. In this work, we propose Dynamics-aware Token Merging (DaTMe), a training-free framework that grounds token merging in the underlying diffusion dynamics. Our key insight is that token importance should reflect whether a token remains unresolved during denoising and whether it carries structural information. To this end, DaTMe constructs a dynamics- aware importance map from the Tweedie denoised estimate. The map combines a temporal signal, measuring inter-step changes in the predicted clean image, with a structural signal, capturing local high-frequency content such as boundaries and fine details. Using this map, DaTMe adaptively assigns tokens into merging roles under a given compression ratio: important tokens are preserved as anchors, while tokens that are both temporally stabilized and structurally homogeneous are selected as safe merge candidates. Experiments on PixArt-a and FLUX demonstrate that DaTMe maintains visual fidelity across the entire image, highlighting diffusion dynamics as an effective token-level merging decision in efficient generation. Our code is available at https://github.com/SeungJuCha/DaTMe.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
