arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.00761cs.CV
收藏

学习相机ISP管线的退化无关表示

Learning Degradation-Independent Representations for Camera ISP Pipelines

McMaster University(麦克马斯特大学)

Yanhui Guo, Fangzhou Luo, Xiaolin Wu

AI总结:

针对ISP退化复杂未知导致DNN泛化差的问题,提出学习退化无关表示(DiR)的方法,通过优化自监督基线表示,提升域泛化能力,在盲恢复、目标检测和实例分割等下游任务中超越现有方法。

摘要:

图像信号处理(ISP)管线在数码相机中扮演着基础性角色,它将原始拜耳传感器数据转换为RGB图像。然而,由于传感器噪声、去马赛克噪声、压缩伪影以及可能由错误的ISP超参数设置(如ISO和伽马值)引起的不利影响所导致的复合退化,ISP生成的图像通常会遭受不完美之处。从广义上讲,这些ISP缺陷可以被视为退化。ISP退化的高度复杂机制,其中一些甚至是未知的,对深度神经网络(DNN)在图像恢复中的泛化能力及其对下游任务的适应性构成了巨大挑战。为了解决这些问题,我们提出了一种新颖的DNN方法,通过优化自监督学习得到的基线表示来学习退化无关表示(DiR)。所提出的DiR学习技术具有显著的域泛化能力,因此,在我们的实验中验证,它在各种下游任务中(包括盲图像恢复、目标检测和实例分割)均优于最先进的方法。

英文摘要:

Image signal processing (ISP) pipeline plays a fundamental role in digital cameras, which converts raw Bayer sensor data to RGB images. However, ISP-generated images usually suffer from imperfections due to the compounded degradations that stem from sensor noises, demosaicing noises, compression artifacts, and possibly adverse effects of erroneous ISP hyperparameter settings such as ISO and gamma values. In a general sense, these ISP imperfections can be considered as degradations. The highly complex mechanisms of ISP degradations, some of which are even unknown, pose great challenges to the generalization capability of deep neural networks (DNN) for image restoration and to their adaptability to downstream tasks. To tackle the issues, we propose a novel DNN approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and consequently, it outperforms state-of-the-art methods across various downstream tasks, including blind image restoration, object detection, and instance segmentation, as verified in our experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑