用于统一联合去马赛克与去噪的结构引导
Structural Guidance for Unified Joint Demosaicing and Denoising
浏览论文内容
中文总结 AI 辅助
该研究提出含SwinIR恢复分支、结构推理分支与轻量适配器的结构引导统一恢复框架，在多CFA模式和噪声水平实验中较现有方法实现一致性能提升，增强了相机图像恢复的鲁棒性。
中文摘要 AI 辅助
联合去马赛克与去噪是相机图像信号处理中的基础步骤，但仍具挑战性，因为不同的拜耳类彩色滤光片阵列（CFA）和传感器噪声会共同破坏颜色采样与图像内容。现有的统一恢复网络虽明确建模CFA几何结构，但仍主要由像素级监督驱动，易在边缘、重复纹理和摩尔纹等局部证据不可靠的区域出现结构退化。我们将该局限性部分归因于除像素级重建监督外缺乏显式结构引导。基于此观察，我们提出一种结构引导的统一恢复框架，将预训练的结构知识注入感知CFA的图像恢复任务。我们的模型接收由原始马赛克、CFA掩码和噪声水平图组成的五通道统一观测；其中SwinIR恢复分支在CFA条件调制下重建像素细节，并行的结构推理分支从稀疏伪RGB观测中提取互补结构线索。为弥合稀疏噪声传感器数据与结构编码器自然图像预训练域之间的显著域差距，我们引入轻量可训练适配器，再通过残差融合结构特征与恢复特征。共享解码器共同预测恢复后的RGB图像和辅助干净马赛克，提供图像域与传感器域的双重监督。在多种CFA模式和噪声水平下开展的大量实验表明，该方法较现有最先进的统一方法及CFA特定方法均取得一致提升，说明适配后的结构先验可增强相机图像恢复的鲁棒性。源代码与数据集见补充材料。
英文摘要
Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and dataset are provided in the supplementary material.
发表机构
- Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
机构由 AI 辅助整理，请以论文原文为准。
