发表机构
Graz University of Technology; Swiss Data Science Center; ETH Zurich; Complexity Science Hub(格拉茨工业大学; 瑞士数据科学中心; 苏黎世联邦理工学院; 复杂性科学中心)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
提出黄金路径假说,证明扩散缓存中提示无关的调度可媲美提示特定调度,通过误差分解和少量示例搜索出跨提示数据集迁移的端到端调度,兼顾重建保真度与感知相似性。
AI 中文摘要
扩散缓存通过在选择去噪步骤中用缓存或预测的特征替代Transformer计算来加速生成。我们提出了黄金路径假说(GPH):在固定推理条件下,与提示无关的缓存调度可以在各提示上实现与最佳提示特定调度相当的最终输出质量。我们跨十种缓存方法、四种图像和视频模型以及三种缓存比率研究了GPH。提示自适应方法反复选择少量调度,在新提示上重用它们最频繁的调度与提示特定选择的质量非常接近。对四个示例上140万个调度的详尽评估进一步确定了在未见提示上仍具竞争力的提示无关调度。为解释这种迁移,我们分析了去噪轨迹和缓存误差的累积。潜在状态轨迹在数据集和种子间表现出相似结构,而精确的误差分解表明,早期误差的累积效应比局部近似误差更能预测最终潜在状态误差。这促使我们使用最终输出质量搜索端到端调度。仅用一小部分示例,所得黄金路径即可跨提示和数据集迁移,并可针对所需质量目标(包括重建保真度或感知相似性)进行调整。
英文摘要
Diffusion caching accelerates generation by replacing transformer computation with cached or predicted features at selected denoising steps. We introduce the Golden Path Hypothesis (GPH): under fixed inference conditions, prompt-independent cache schedules can achieve final-output quality comparable to the best prompt-specific schedules across prompts. We investigate the GPH across ten caching methods, four image and video models, and three cache ratios. Prompt-adaptive methods repeatedly select a small number of schedules, and reusing their most frequent schedules on new prompts closely matches the quality of prompt-specific choices. Exhaustive evaluation of 1.4 million schedules on four examples further identifies prompt-independent schedules that remain competitive on unseen prompts. To explain this transfer, we analyze denoising trajectories and the accumulation of caching errors. Latent-state trajectories exhibit similar structures across datasets and seeds, while an exact error decomposition shows that accumulated effects of earlier errors predict final latent-state error better than local approximation errors. This motivates searching for end-to-end schedules using final-output quality. With only a small set of examples, the resulting golden paths transfer across prompts and datasets, and can be tuned to the desired quality objective, including reconstruction fidelity or perceptual similarity.