用于无监督背光图像增强的迭代提示学习
Iterative Prompt Learning for Unsupervised Backlit Image Enhancement
AI总结：
本文提出无监督背光图像增强方法 CLIP-LIT，利用 CLIP 先验和迭代提示学习交替优化提示与增强网络，在无成对数据条件下取得优于现有方法的视觉质量与泛化效果。
摘要：
我们提出了一种新颖的无监督背光图像增强方法，简称为 CLIP-LIT，该方法探索了 Contrastive Language-Image Pre-Training（CLIP，对比语言-图像预训练）在像素级图像增强方面的潜力。我们表明，开放世界的 CLIP 先验不仅有助于区分背光图像和光照良好的图像，而且有助于感知具有不同亮度的异质区域，从而促进增强网络的优化。与高层视觉任务和图像处理任务不同，由于难以找到准确的提示，将 CLIP 直接应用于增强任务并非易事。为了解决这一问题，我们设计了一个提示学习框架，该框架首先通过在 CLIP 潜在空间中约束提示（负样本/正样本）与相应图像（背光图像/光照良好图像）之间的文本-图像相似性，来学习初始提示对。随后，我们基于增强结果与初始提示对之间的文本-图像相似性来训练增强网络。为进一步提高初始提示对的准确性，我们通过排序学习迭代微调提示学习框架，以缩小背光图像、增强结果和光照良好图像之间的分布差距，从而提升增强性能。我们的方法在更新提示学习框架和增强网络之间交替进行，直到获得视觉上令人满意的结果。大量实验表明，在不需要任何成对数据的情况下，我们的方法在视觉质量和泛化能力方面优于最先进的方法。
英文摘要：
We propose a novel unsupervised backlit image enhancement method, abbreviated as CLIP-LIT, by exploring the potential of Contrastive Language-Image Pre-Training (CLIP) for pixel-level image enhancement. We show that the open-world CLIP prior not only aids in distinguishing between backlit and well-lit images, but also in perceiving heterogeneous regions with different luminance, facilitating the optimization of the enhancement network. Unlike high-level and image manipulation tasks, directly applying CLIP to enhancement tasks is non-trivial, owing to the difficulty in finding accurate prompts. To solve this issue, we devise a prompt learning framework that first learns an initial prompt pair by constraining the text-image similarity between the prompt (negative/positive sample) and the corresponding image (backlit image/well-lit image) in the CLIP latent space. Then, we train the enhancement network based on the text-image similarity between the enhanced result and the initial prompt pair. To further improve the accuracy of the initial prompt pair, we iteratively fine-tune the prompt learning framework to reduce the distribution gaps between the backlit images, enhanced results, and well-lit images via rank learning, boosting the enhancement performance. Our method alternates between updating the prompt learning framework and enhancement network until visually pleasing results are achieved. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in terms of visual quality and generalization ability, without requiring any paired data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
