PixRestore：基于像素扩散Transformer的统一图像修复模型
PixRestore: Unified Image Restoration via Pixel Diffusion Transformer
- The Hong Kong Polytechnic University（香港理工大学）
- OPPO Research Institute（OPPO研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出无VAE的像素空间DiT模型PixRestore，通过流匹配与DINO特征可靠性预测实现UIR，仅50M参数且单步推理，在效率与修复性能上优于同类模型。
AI中文摘要：
统一图像修复（UIR）旨在使用单一模型从具有不同退化类型的低质量（LQ）图像中恢复高质量（HQ）内容。近期多数方法因大型预训练文本到图像（T2I）潜在扩散模型的强大容量与生成先验而采用该类模型，但潜在T2I模型中的变分自编码器（VAE）可能丢失对修复敏感的细节，且开放式合成先验会引入内容不一致的伪影。本文提出PixRestore，一种无VAE的像素空间扩散Transformer（DiT）用于UIR，其扩散骨干完全从头训练，不依赖T2I预训练。PixRestore直接在分块像素上执行流匹配，在保持token序列易处理的同时保留细粒度细节。为适配不同退化，PixRestore利用LQ-HQ的DINO特征相似性学习预测层特征的可靠性：更可靠层的特征被融合为密集条件输入，可靠性较低的层则接收更强的HQ特征监督以促进退化去除。我们在包含多样场景与退化的大规模语料库上训练PixRestore，并进一步使用基于DINO的对抗目标将其微调为单步生成器以实现高效推理。在公开基准与真实测试集上的实验表明，仅约50M参数且单步推理的PixRestore，在竞争UIR模型中实现了最佳的整体保真度、感知质量与退化鲁棒性，同时效率远高于其他模型；更大的PixRestore变体可进一步提升性能，证明了像素空间设计的可扩展性。代码与整理后的基准可在指定URL获取。
英文摘要：
Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors. However, the variational autoencoder (VAE) in latent T2I models may discard restoration-sensitive details, while the open-ended synthesis prior can introduce content-inconsistent artifacts. We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining. PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable. To adapt to different degradations, PixRestore learns to predict the reliability of layer features using LQ--HQ DINO feature similarity. Features from more reliable layers are fused as dense conditioning, while less reliable layers receive stronger HQ-feature supervision to encourage degradation removal. We train PixRestore on a large-scale corpus of diverse scenes and degradations, and further finetune it into a one-step generator using DINO-based adversarial objectives for efficient inference. Experiments on public benchmarks and real-world test sets show that, with only about 50M parameters and single-step inference, PixRestore achieves the best overall fidelity, perceptual quality, and robustness to degradations among competing UIR models while being far more efficient. Larger PixRestore variants can further boost performance, demonstrating the scalability of our pixel-space design. Code and the curated benchmark can be found at https://github.com/csslc/PixRestore.
