arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.23902cs.CV
收藏

PiD: 基于像素扩散的快速高分辨率潜解码

PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

NVIDIA

Yifan Lu, Qi Wu, Jay Zhangjie Wu, Zian Wang, Huan Ling, Sanja Fidler, Xuanchi Ren

AI总结:

提出像素扩散解码器PiD,将潜解码重构为条件像素扩散,统一解码与上采样,实现低延迟高分辨率图像生成。

摘要:

大多数实用的高分辨率文本到图像系统,包括潜扩散和自回归模型,在紧凑的潜空间中生成,然后由解码器将生成的潜变量映射回像素。然而,潜到像素解码器是面向重建的,优化目标是反转编码器而非合成更多细节,并且在百万像素尺度上成本越来越高。这一缺陷呼唤更具表现力和高效的解码范式。受近期可扩展像素空间扩散进展的启发,我们引入了PiD,一种像素扩散解码器,将潜解码重新表述为条件像素扩散,将解码和上采样统一到一个生成模块中。通过直接在高分辨率像素空间中去噪,PiD以低延迟合成$4 imes$甚至$8 imes$上采样的图像。对于潜条件,一个轻量级的sigma感知适配器将噪声污染的潜变量注入像素扩散主干,使PiD能够解码部分去噪的潜变量并提前终止潜扩散过程。为进一步提高效率,我们使用DMD2蒸馏模型,将推理步骤减少到仅4步。PiD适用于传统的VAE潜变量和近期基于RAE的模型中使用的语义潜变量(例如SigLIP、DINOv2)。PiD在消费级RTX 5090上,以13 GB峰值内存,在不到1秒内将$512 imes 512$图像的潜变量解码为$2048 imes 2048$像素,在GB200 GPU上最快可达210毫秒,比级联扩散超分辨率流水线快约$6 imes$,且视觉保真度更高。

英文摘要:

Most practical high-resolution text-to-image systems, including latent diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the generated latents back to pixels. Yet the latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details, and becomes increasingly costly at megapixel scale. This drawback calls for a more expressive and efficient decoding paradigm. Motivated by recent progress in scalable pixel-space diffusion, we introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion, unifying decoding and upsampling into one generative module. By denoising directly in high-resolution pixel space, PiD synthesizes $4\times$ and even $8\times$ upscaled images with low latency. For latent conditioning, a lightweight sigma-aware adapter injects noise-corrupted latents into the pixel diffusion backbone, enabling PiD to decode partially denoised latents and terminate the latent diffusion process early. To further improve efficiency, we distill the model using DMD2, reducing inference to just 4 steps. PiD applies to both conventional VAE latents and semantic latents (e.g., SigLIP, DINOv2) used in recent RAE-based models. PiD decodes latents of $512 \times 512$ images into $2048 \times 2048$ pixels in under 1 second with 13 GB peak memory on a consumer RTX 5090, and as fast as 210 ms on a GB200 GPU, about $6\times$ faster than cascaded diffusion-based super-resolution pipelines with better visual fidelity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑