Text2Relight：基于文本引导的创意人像重打光
Text2Relight: Creative Portrait Relighting with Text Guidance
AI总结：
提出一种基于文本引导的人像重打光流水线，通过大型语言模型生成多样化文本提示并利用光场系统合成大规模配对数据，训练扩散模型实现前景与背景的创意光照编辑。
摘要：
我们提出了一种光照感知的图像编辑流水线，给定一张人像图像和文本提示，即可执行单图像重打光。我们的模型修改前景和背景的光照与颜色，使其与提供的文本描述对齐。文本在创意上的无界性使我们能够用任何感官特征（包括温度、情感、气味、时间等）来描述场景的光照。然而，由于缺乏提供大量文本与重打光配对的可扩展数据的数据集，建模这种无界文本与光照之间的映射极具挑战性，因此当前的文本驱动图像编辑模型无法泛化到特定光照的用例中。我们通过引入一种新颖的数据合成流水线来解决这一问题：首先，利用大型语言模型（例如ChatGPT），在精心设计的层级下自动生成描述各种光照场景的多样且富有创意的文本提示。文本引导的图像生成模型创建与文本最匹配的光照图像。作为光照图像的条件，我们使用单人像图像或从光场系统捕获的一组OLAT（One-Light-at-a-Time）图像，对前景和背景执行基于图像的重打光。特别是对于背景重打光，我们将光照图像表示为一组点光源，并将它们转移到其他背景图像上。生成式扩散模型通过辅助任务增强（例如人像去光和光照定位）学习合成的大规模数据，以关联潜在文本和光照分布，从而实现文本引导的人像重打光。
英文摘要：
We present a lighting-aware image editing pipeline that, given a portrait image and a text prompt, performs single image relighting. Our model modifies the lighting and color of both the foreground and background to align with the provided text description. The unbounded nature in creativeness of a text allows us to describe the lighting of a scene with any sensory features including temperature, emotion, smell, time, and so on. However, the modeling of such mapping between the unbounded text and lighting is extremely challenging due to the lack of dataset where there exists no scalable data that provides large pairs of text and relighting, and therefore, current text-driven image editing models does not generalize to lighting-specific use cases. We overcome this problem by introducing a novel data synthesis pipeline: First, diverse and creative text prompts that describe the scenes with various lighting are automatically generated under a crafted hierarchy using a large language model (*e.g.,* ChatGPT). A text-guided image generation model creates a lighting image that best matches the text. As a condition of the lighting images, we perform image-based relighting for both foreground and background using a single portrait image or a set of OLAT (One-Light-at-A-Time) images captured from lightstage system. Particularly for the background relighting, we represent the lighting image as a set of point lights and transfer them to other background images. A generative diffusion model learns the synthesized large-scale data with auxiliary task augmentation (*e.g.,* portrait delighting and light positioning) to correlate the latent text and lighting distribution for text-guided portrait relighting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
