发表机构
Harbin Institute of Technology, Shenzhen; Shanghai Jiao Tong University; Nanjing University (Suzhou)（哈尔滨工业大学（深圳）; 上海交通大学; 南京大学（苏州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出SegDem框架，通过实例分割与跨任务解码器表示迁移，结合DINOv2共享表示空间，在单/四拜耳去马赛克任务上，于多类数据集实现不同架构下的性能稳定提升。
AI 中文摘要
图像去马赛克是从覆盖有彩色滤光片阵列（CFA）的传感器产生的不完整彩色测量值中重建全彩色图像的过程。现有大多数方法将去马赛克建模为像素级重建问题，主要依赖局部纹理、跨通道相关性和低级图像统计信息。本文的核心见解是，重建与视觉理解可视为共享场景结构的互补视角：两者均基于同一底层物理世界，因此从图像中推断出的结构与物理信息应在两个任务间保持一致。我们通过实例分割实现这一思路，提出SegDem——一种用于去马赛克的跨任务解码器表示迁移框架。SegDem首先通过实例感知结构预训练学习区域和边界感知表示，随后将解码器迁移至RAW条件下的重建。分割条件和去马赛克条件的特征进一步被锚定到共享的冻结DINOv2表示空间，以保留跨任务的结构组织。我们为统一的单拜耳（Single-Bayer）和四拜耳（Quad-Bayer）去马赛克分别实例化了基于卷积、Transformer和状态空间的骨干网络。在合成数据集、外部数据集及挑战性数据集上开展的大量实验表明，该方法在不同架构和CFA布局下均实现了稳定提升。
英文摘要
Image demosaicing reconstructs a full-color image from incomplete color measurements produced by a sensor covered with a color filter array (CFA). Most existing methods formulate demosaicing as pixel-level reconstruction and mainly rely on local textures, cross-channel correlations, and low-level image statistics. Our core insight is that reconstruction and visual understanding can be viewed as complementary views of shared scene structure: both are grounded in the same underlying physical world, and therefore the structural and physical information inferred from an image should remain consistent across the two tasks. We instantiate this idea with instance segmentation and propose \emph{SegDem}, a cross-task decoder representation transfer framework for demosaicing. SegDem first learns region- and boundary-aware representations through instance-aware structural pretraining and then transfers the decoder to RAW-conditioned reconstruction. Segmentation- and demosaicing-conditioned features are further anchored to a shared frozen DINOv2 representation space to preserve structural organization across tasks. We instantiate SegDem with convolutional, Transformer-based, and state-space backbones for unified Single- and Quad-Bayer demosaicing. Extensive experiments on synthetic, external, and challenging datasets demonstrate consistent improvements across different architectures and CFA layouts.
Comments20 pagess
