arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02710cs.CV

MiM-DiT:基于扩散变换器的MoE在MoE中实现全功能图像修复

MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration

Nanjing University of Science and Technology(南京理工大学);Nankai University(南开大学);Harbin Institute of Technology(哈尔滨工业大学)

Lingshun Kong, Jiawei Zhang, Zhengpeng Duan, Xiaohe Wu, Yueqi Yang, Xiaotao Wang, Dongqing Zou, Lei Lei, Jinshan Pan

AI总结:

MiM-DiT通过双层MoE架构与预训练扩散模型,实现对多种图像退化类型的统一修复,提升复杂真实场景下的修复效果。

摘要:

全功能图像修复具有挑战性,因为不同的退化类型,如雾霾、模糊、噪声和低光照,对修复策略提出了多样化的要求,使单一模型难以有效处理。在本文中,我们提出了一种统一的图像修复框架,该框架集成了双层混合专家(MoE)架构与预训练的扩散模型。该框架在两个层面运行:Inter-MoE层自适应地结合专家组以处理主要的退化类型,而Intra-MoE层进一步选择专门的子专家以解决每种类型内的细微变化。这种设计使模型能够在多样化的退化类别之间实现粗粒度适应,同时对特定类内的细微变化进行精细调节,确保在处理复杂真实世界退化时具有高专业化。广泛的实验表明,所提出的方法在多个图像修复任务中优于最先进的方法。

英文摘要:

All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑