LGTM: 通过初始噪声操控实现无训练的光引导文本到图像扩散模型
LGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation
AI总结：
本文提出LGTM模型，通过操控扩散过程的初始噪声实现文本提示和用户指定光照方向的图像生成，无需微调或修改预训练模型，提升了光照控制的灵活性和适应性。
摘要：
扩散模型在条件文本到图像生成中表现出高质量性能，特别是在结构线索如边缘、布局和深度方面。然而，光照条件却受到有限关注，难以在生成过程中控制。现有方法通过两阶段流程在生成后重新照明，效率低下。此外，它们依赖于大规模数据集和大量计算，限制了其对新模型和任务的适应性。为此，我们提出了一种新的无训练光引导文本到图像扩散模型LGTM，通过操控扩散过程的初始潜在噪声，引导图像生成。通过潜在空间的通道分析，我们发现选择性操控潜在通道可以在不微调或修改预训练模型的情况下实现细粒度光照控制。大量实验表明，我们的方法在光照一致性方面优于基于提示的基线，同时保持图像质量和文本对齐。这种方法为动态、用户引导的光照控制提供了新可能。此外，它能无缝集成到ControlNet等模型中，展示了在多样化场景中的适应性。
英文摘要：
Diffusion models have demonstrated high-quality performance in conditional text-to-image generation, particularly with structural cues such as edges, layouts, and depth. However, lighting conditions have received limited attention and remain difficult to control within the generative process. Existing methods handle lighting through a two-stage pipeline that relights images after generation, which is inefficient. Moreover, they rely on fine-tuning with large datasets and heavy computation, limiting their adaptability to new models and tasks. To address this, we propose a novel Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation (LGTM), which manipulates the initial latent noise of the diffusion process to guide image generation with text prompts and user-specified light directions. Through a channel-wise analysis of the latent space, we find that selectively manipulating latent channels enables fine-grained lighting control without fine-tuning or modifying the pre-trained model. Extensive experiments show that our method surpasses prompt-based baselines in lighting consistency, while preserving image quality and text alignment. This approach introduces new possibilities for dynamic, user-guided light control. Furthermore, it integrates seamlessly with models like ControlNet, demonstrating adaptability across diverse scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
