一种高效混合异构图像复原模型
An Efficient and Mixed Heterogeneous Model for Image Restoration
AI总结：
本文提出RestorMixer，一种基于CNN、Mamba和注意力机制混合架构融合的三阶段编码器-解码器图像复原模型，在多个复原任务上实现领先性能并保持高推理效率。
摘要：
图像复原（IR）作为一项基础的多媒体数据处理任务，对下游视觉应用具有重要影响。近年来，研究者们致力于开发能够处理多种退化类型的通用图像复原模型，以降低模型开发的成本和复杂度。当前主流方法基于三种架构范式：CNN、Transformer和Mamba。CNN擅长高效推理，而Transformer和Mamba擅长捕捉长距离依赖和建模全局上下文。尽管每种架构在专门的单任务场景中已展现出成功，但在有效整合异构架构以共同应对多样化的图像复原挑战方面所做的努力仍然有限。为弥补这一空白，我们提出了RestorMixer，一种基于混合架构融合的高效通用图像复原模型。RestorMixer采用三阶段编码器-解码器结构，每个阶段根据输入的分辨率和特征特性进行定制。在初始高分辨率阶段，采用基于CNN的模块快速提取浅层局部特征。在后续阶段，我们整合了精细的多向扫描Mamba模块与基于多尺度窗口的自注意力机制。这种层次化且自适应的设计使模型能够利用CNN在局部特征提取方面的优势、Mamba在全局上下文建模方面的优势以及注意力机制在动态特征细化方面的优势。大量实验结果表明，RestorMixer在多个图像复原任务上取得了领先性能，同时保持了高推理效率。官方代码可访问https://github.com/ClimBin/RestorMixer。
英文摘要：
Image restoration~(IR), as a fundamental multimedia data processing task, has a significant impact on downstream visual applications. In recent years, researchers have focused on developing general-purpose IR models capable of handling diverse degradation types, thereby reducing the cost and complexity of model development. Current mainstream approaches are based on three architectural paradigms: CNNs, Transformers, and Mambas. CNNs excel in efficient inference, whereas Transformers and Mamba excel at capturing long-range dependencies and modeling global contexts. While each architecture has demonstrated success in specialized, single-task settings, limited efforts have been made to effectively integrate heterogeneous architectures to jointly address diverse IR challenges. To bridge this gap, we propose RestorMixer, an efficient and general-purpose IR model based on mixed-architecture fusion. RestorMixer adopts a three-stage encoder-decoder structure, where each stage is tailored to the resolution and feature characteristics of the input. In the initial high-resolution stage, CNN-based blocks are employed to rapidly extract shallow local features. In the subsequent stages, we integrate a refined multi-directional scanning Mamba module with a multi-scale window-based self-attention mechanism. This hierarchical and adaptive design enables the model to leverage the strengths of CNNs in local feature extraction, Mamba in global context modeling, and attention mechanisms in dynamic feature refinement. Extensive experimental results demonstrate that RestorMixer achieves leading performance across multiple IR tasks while maintaining high inference efficiency. The official code can be accessed at https://github.com/ClimBin/RestorMixer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
