CLIPtone：基于文本的图像色调调整的无监督学习
CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment
AI总结：
CLIPtone 提出一种基于无监督学习的文本图像色调调整方法，通过超网络根据文本描述自适应调制预训练模型参数，并利用 CLIP 评估图文一致性，从而降低数据成本并支持训练中未见过的文本描述。
摘要：
近年来，图像色调调整（或增强）方法主要采用监督学习来学习以人为中心的感知评估。然而，这些方法受到监督学习固有挑战的制约。首先，对专家精心处理或修饰过的图像的需求增加了数据采集成本。此外，它们对目标风格的覆盖范围仅限于从训练数据中推断出的风格变体。为了克服上述挑战，我们提出了一种基于无监督学习的文本图像色调调整方法 CLIPtone，该方法扩展现有的图像增强方法以适应自然语言描述。具体来说，我们设计了一个超网络，根据文本描述自适应地调制骨干模型的预训练参数。为了在没有真实图像的情况下评估调整后的图像是否与文本描述一致，我们利用 CLIP，该模型在大量语言-图像对上进行了训练，因此包含了人类感知的知识。我们的方法的主要优势有三个方面：（i）最小的数据收集成本，（ii）支持一系列调整，以及（iii）能够处理训练中未见过的新的文本描述。我们方法的有效性通过包括用户研究在内的综合实验得到了证明。
英文摘要：
Recent image tone adjustment (or enhancement) approaches have predominantly adopted supervised learning for learning human-centric perceptual assessment. However, these approaches are constrained by intrinsic challenges of supervised learning. Primarily, the requirement for expertly-curated or retouched images escalates the data acquisition expenses. Moreover, their coverage of target style is confined to stylistic variants inferred from the training data. To surmount the above challenges, we propose an unsupervised learning-based approach for text-based image tone adjustment method, CLIPtone, that extends an existing image enhancement method to accommodate natural language descriptions. Specifically, we design a hyper-network to adaptively modulate the pretrained parameters of the backbone model based on text description. To assess whether the adjusted image aligns with the text description without ground truth image, we utilize CLIP, which is trained on a vast set of language-image pairs and thus encompasses knowledge of human perception. The major advantages of our approach are three fold: (i) minimal data collection expenses, (ii) support for a range of adjustments, and (iii) the ability to handle novel text descriptions unseen in training. Our approach's efficacy is demonstrated through comprehensive experiments, including a user study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
