arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.16627cs.CV
收藏

SDXS:带图像条件的实时单步潜扩散模型

SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions

Xiaomi Inc.(小米公司)

Yuda Song, Zehao Sun, Xuanwu Yin

AI总结:

针对扩散模型计算需求高、采样延迟大的问题,提出结合模型小型化与单步训练的SDXS系列模型,大幅提升推理速度,且支持图像条件控制与高效图像转换。

摘要:

扩散模型的最新进展使其跻身图像生成领域前沿。尽管性能优异,扩散模型并非没有缺陷:它们架构复杂且计算需求极高,迭代采样过程导致延迟显著。为缓解这些局限,我们提出一种包含模型小型化与采样步数减少的双重方案,旨在大幅降低模型延迟。我们的方法利用知识蒸馏精简U-Net和图像解码器架构,并提出一种创新的单步扩散模型(DM)训练技术,该技术采用特征匹配与分数蒸馏。我们推出SDXS-512和SDXS-1024两个模型,在单块GPU上分别实现约100 FPS(比SD v1.5快30倍)和30 FPS(比SDXL快60倍)的推理速度。此外,我们的训练方法在图像条件控制方面展现出良好应用前景,可助力实现高效的图像到图像转换。

英文摘要:

Recent advancements in diffusion models have positioned them at the forefront of image generation. Despite their superior performance, diffusion models are not without drawbacks; they are characterized by complex architectures and substantial computational demands, resulting in significant latency due to their iterative sampling process. To mitigate these limitations, we introduce a dual approach involving model miniaturization and a reduction in sampling steps, aimed at significantly decreasing model latency. Our methodology leverages knowledge distillation to streamline the U-Net and image decoder architectures, and introduces an innovative one-step DM training technique that utilizes feature matching and score distillation. We present two models, SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS (30x faster than SD v1.5) and 30 FPS (60x faster than SDXL) on a single GPU, respectively. Moreover, our training approach offers promising applications in image-conditioned control, facilitating efficient image-to-image translation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑