EAM: 利用扩散变换器提升盲超分辨率
EAM: Enhancing Anything with Diffusion Transformers for Blind Super-Resolution
AI总结：
本文提出EAM模型，利用DiT架构提升盲超分辨率，引入Ψ-DiT块和渐进性掩码图像建模策略，实现更优的图像恢复效果。
摘要：
利用预训练的文本到图像（T2I）扩散模型指导盲超分辨率（BSR）已成为该领域的主要方法。尽管T2I模型传统上依赖U-Net架构，但最近的进展表明，扩散变换器（DiT）在该领域实现了显著更高的性能。在本文中，我们介绍了增强任何模型（EAM），一种新的BSR方法，利用DiT并优于之前的U-Net基于方法。我们引入了一个新的块，Ψ-DiT，该块有效引导DiT增强图像恢复。该块采用低分辨率潜在作为可分离的流注入控制，形成三流架构，有效利用预训练DiT中嵌入的先验知识。为了充分利用T2I模型的先验指导能力并增强其在BSR中的泛化能力，我们引入了渐进性掩码图像建模策略，这也减少了训练成本。此外，我们提出了一种主体感知的提示生成策略，该策略在上下文学习框架中使用鲁棒的多模态模型。该策略自动识别关键图像区域，提供详细的描述，并优化T2I扩散先验的利用。我们的实验表明，EAM在多个数据集上实现了最先进的结果，优于现有方法在定量指标和视觉质量方面。
英文摘要：
Utilizing pre-trained Text-to-Image (T2I) diffusion models to guide Blind Super-Resolution (BSR) has become a predominant approach in the field. While T2I models have traditionally relied on U-Net architectures, recent advancements have demonstrated that Diffusion Transformers (DiT) achieve significantly higher performance in this domain. In this work, we introduce Enhancing Anything Model (EAM), a novel BSR method that leverages DiT and outperforms previous U-Net-based approaches. We introduce a novel block, $Ψ$-DiT, which effectively guides the DiT to enhance image restoration. This block employs a low-resolution latent as a separable flow injection control, forming a triple-flow architecture that effectively leverages the prior knowledge embedded in the pre-trained DiT. To fully exploit the prior guidance capabilities of T2I models and enhance their generalization in BSR, we introduce a progressive Masked Image Modeling strategy, which also reduces training costs. Additionally, we propose a subject-aware prompt generation strategy that employs a robust multi-modal model in an in-context learning framework. This strategy automatically identifies key image areas, provides detailed descriptions, and optimizes the utilization of T2I diffusion priors. Our experiments demonstrate that EAM achieves state-of-the-art results across multiple datasets, outperforming existing methods in both quantitative metrics and visual quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
