IntrinsicWeather: 内在空间中可控的天气编辑
IntrinsicWeather: Controllable Weather Editing in Intrinsic Space
AI总结：
本文提出IntrinsicWeather框架，通过内在空间编辑实现可控天气生成，利用内在映射提升大场景控制能力，并在合成与真实数据集上验证其优越性。
摘要：
我们提出了IntrinsicWeather，一种基于扩散优先的框架，用于内在空间中的可控天气编辑。我们的框架包含两个组件：逆向渲染器估计材料属性、场景几何和照明作为内在映射，以及正向渲染器利用这些几何和材料映射以及描述特定天气条件的文本提示生成最终图像。内在映射相比传统像素空间编辑方法提升了可控性。我们提出了一种内在映射感知的注意力机制，以提高大室外场景中的空间对应和分解质量。对于正向渲染，我们利用CLIP空间插值天气提示以实现精细的天气控制。我们还引入了一个合成和真实世界的数据集，包含38k和18k张在不同天气条件下拍摄的图像，每张图像都有内在映射注释。IntrinsicWeather在最先进的像素空间编辑方法、天气恢复方法和渲染方法上表现更优，展示了在自动驾驶等下游任务中的潜力，增强了恶劣天气场景下的检测和分割鲁棒性。
英文摘要：
We present IntrinsicWeather, a diffusion-based framework for controllable weather editing in intrinsic space. Our framework includes two components based on diffusion priors: an inverse renderer that estimates material properties, scene geometry, and lighting as intrinsic maps from an input image, and a forward renderer that utilizes these geometry and material maps along with a text prompt that describes specific weather conditions to generate a final image. The intrinsic maps enhance controllability compared to traditional pixel-space editing approaches. We propose an intrinsic map-aware attention mechanism that improves spatial correspondence and decomposition quality in large outdoor scenes. For forward rendering, we leverage CLIP-space interpolation of weather prompts to achieve fine-grained weather control. We also introduce a synthetic and a real-world dataset, containing 38k and 18k images under various weather conditions, each with intrinsic map annotations. IntrinsicWeather outperforms state-of-the-art pixel-space editing approaches, weather restoration methods, and rendering-based methods, showing promise for downstream tasks such as autonomous driving, enhancing the robustness of detection and segmentation in challenging weather scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
