MetaISP -- 利用全局场景结构实现精确的多设备色彩渲染
MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition
AI总结：
提出MetaISP模型，利用轻量级深度学习和跨协方差注意力机制学习全局场景语义，实现不同设备间色彩与局部对比度特征的精确转换，并利用元数据估计场景光源。
摘要：
图像信号处理器（ISP）是历史上发展起来的遗留软件系统，用于从嘈杂的原始传感器测量值中重建彩色图像。每个智能手机制造商都开发了各自的ISP，并采用其特有的启发式方法来改善色彩渲染，例如肤色和其他视觉上重要的颜色。最近，人们对用深度学习流水线取代历史发展而来的ISP系统以匹配DSLR的图像质量产生了兴趣，这改善了图像中的结构特征。然而，这些工作忽略了基于语义场景分析的卓越色彩处理，而这正是区分手机ISP与DSLR的关键。在此，我们提出MetaISP，这是一个旨在学习如何在不同设备的色彩和局部对比度特征之间进行转换的单一模型。MetaISP将设备A的RAW图像作为输入，并将其转换为继承了设备A、B和C外观特征的RGB图像。我们通过采用一种轻量级深度学习技术实现了这一结果，该技术根据目标设备条件化其输出外观。在这种方法中，我们利用受跨协方差启发的新型注意力机制来学习全局场景语义。此外，我们使用通常随RAW图像附带的元数据，并在场景光源不可用时对其进行估计。
英文摘要：
Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. Each smartphone manufacturer has developed its ISPs with its own characteristic heuristics for improving the color rendition, for example, skin tones and other visually essential colors. The recent interest in replacing the historically grown ISP systems with deep-learned pipelines to match DSLR's image quality improves structural features in the image. However, these works ignore the superior color processing based on semantic scene analysis that distinguishes mobile phone ISPs from DSLRs. Here, we present MetaISP, a single model designed to learn how to translate between the color and local contrast characteristics of different devices. MetaISP takes the RAW image from device A as input and translates it to RGB images that inherit the appearance characteristics of devices A, B, and C. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest. In this approach, we leverage novel attention mechanisms inspired by cross-covariance to learn global scene semantics. Additionally, we use the metadata that typically accompanies RAW images and estimate scene illuminants when they are unavailable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
