用于鲁棒自监督泊松逆问题的冻结CLIP先验

中文摘要 AI 辅助

成像逆问题的自监督学习在光子受限场景中愈发重要，该场景下获取干净的真实值不现实，且重建需在数据集与采集偏移下保持稳定。泊松噪声会放大这一挑战，其与信号相关的统计特性会与采样算子（如CFA马赛克）相互作用。与此同时，在网络规模数据上训练的基础视觉编码器能提供与内容相关、对失真不变的表示，可跨域良好泛化，为构建无需昂贵微调、能在训练分布外迁移的先验提供了可行路径。本文提出一种受ADMM启发的展开即插即用求解器，用于泊松逆问题，该求解器将闭式数据一致性更新与参数高效的先验解耦。该先验实现为轻量解码器，基于冻结CLIP RN50密集多尺度特征运行，用更少可训练参数适配基础表示。对于自监督，该方法通过虚拟采集将GR2R测量域重损坏与等变成像正则化器相结合。在泊松CFA去马赛克与去模糊任务上的实验表明，该方法具有竞争力的质量、偏移下的改进鲁棒性，且自监督性能接近监督训练。

英文摘要

Self-supervised learning for imaging inverse problems is increasingly important in photon-limited settings, where acquiring clean ground truth is impractical and reconstruction must remain stable under dataset and acquisition shifts. This challenge is amplified under Poisson noise, whose signal-dependent statistics interact with sampling operators (e.g., CFA mosaicing). Meanwhile, foundation vision encoders trained at web scale offer distortion-invariant, content-related representations that generalize well across domains, suggesting a promising route to build priors that transfer beyond the training distribution without expensive fine-tuning. This paper proposes an ADMM-inspired unrolled plug-and-play solver for Poisson inverse problems that decouples a closed-form data-consistency update from a parameter-efficient prior. The prior is implemented as a lightweight decoder operating on frozen CLIP RN50 dense multi-scale features, adapting foundation representations with less trainable parameters. For self-supervision, the method integrates GR2R measurement-domain re-corruption with an Equivariant Imaging regularizer via virtual acquisitions. Experiments on Poisson CFA demosaicing and deblurring show competitive quality, improved robustness under shifts, and self-supervised performance approaching supervised training.

发表机构

• Universidad Industrial de Santander（桑坦德工业大学）

机构由 AI 辅助整理，请以论文原文为准。