原型引导扩散:无需外部记忆的视觉条件生成
Prototype-Guided Diffusion: Visual Conditioning without External Memory
AI总结:
原型引导扩散模型通过对比学习生成紧凑视觉原型,实现无需外部记忆的高效图像生成。
摘要:
扩散模型在图像生成方面达到最先进的水平,但因迭代去噪过程而计算成本高。潜在空间模型如Stable Diffusion减少了开销却丢失了细节,而检索增强方法提高了效率但依赖大型记忆库、静态相似性模型和刚性基础设施。我们引入了原型扩散模型(PDM),将原型学习嵌入扩散过程,提供适应性强且无记忆的条件生成。PDM通过对比学习从干净特征中学习紧凑的视觉原型,然后在去噪过程中将噪声表示与语义相关模式对齐。实验表明,PDM在保持高质量生成的同时降低了计算和存储成本,为基于检索的条件生成提供了可扩展的替代方案。
英文摘要:
Diffusion models achieve state-of-the-art image generation but remain computationally costly due to iterative denoising. Latent-space models like Stable Diffusion reduce overhead yet lose fine detail, while retrieval-augmented methods improve efficiency but rely on large memory banks, static similarity models, and rigid infrastructures. We introduce the Prototype Diffusion Model (PDM), which embeds prototype learning into the diffusion process to provide adaptive, memory-free conditioning. Instead of retrieving references, PDM learns compact visual prototypes from clean features via contrastive learning, then aligns noisy representations with semantically relevant patterns during denoising. Experiments demonstrate that PDM sustains high generation quality while lowering computational and storage costs, offering a scalable alternative to retrieval-based conditioning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。