RealWeather：基于驾驶世界模型的逼真且场景忠实的天气转换
RealWeather: Realistic and Scene-Faithful Weather Translation with Driving World Models
浏览论文内容
中文总结 AI 辅助
RealWeather是一种驾驶世界模型，通过渐进式逼真度引导和场景忠实度强化学习优化实现逼真且场景忠实的天气转换，在视觉逼真度、结构保留等方面优于现有方法，支持长尾天气场景生成与零样本泛化。
中文摘要 AI 辅助
逼真的天气转换对开发和评估自动驾驶系统具有重要价值，但大规模收集同一场景在不同天气条件下的配对视频并不现实。因此现有方法依赖合成数据、3D天气编辑或几何条件生成，往往在天气逼真度或场景忠实度上有所妥协。我们提出RealWeather，一种用于实现逼真且场景忠实的天气转换的驾驶世界模型。我们的核心思路是直接从真实世界视频中学习真实的天气动态。具体而言，RealWeather采用渐进式逼真度引导（Progressive Realism Bootstrapping），这是一种迭代数据优化策略。在辅助伪清晰生成（Pseudo-Clear Generation）流水线的帮助下，训练最初从伪风格条件视频开始；随着训练推进，这些输入会逐步被模型自身生成的越来越逼真的视频所取代。该策略弥合了伪到真的域差距，使模型能够无缝适应真实世界输入分布，并自然支持清晰-恶劣天气的双向转换。此外，为严格执行结构完整性并抑制幻觉，我们引入场景忠实度强化学习优化（Scene-Fidelity RL Optimization），这是一种奖励驱动的策略优化策略，明确惩罚对安全关键驾驶元素的改变。大量实验表明，RealWeather在视觉逼真度和结构保留方面显著优于现有方法，同时支持稳健的长尾天气场景生成和强大的零样本分布外泛化。
英文摘要
Realistic weather translation is valuable for developing and evaluating autonomous driving systems, yet collecting paired videos of the same scenes under different weather conditions at scale is impractical. Existing methods therefore rely on synthetic data, 3D weather editing, or geometry-conditioned generation, often compromising weather realism or scene fidelity. We propose RealWeather, a driving world model for both realistic and scene-faithful weather translation. Our key idea is to learn authentic weather dynamics directly from real-world videos. Specifically, RealWeather employs Progressive Realism Bootstrapping, an iterative data-refinement strategy. Assisted by an auxiliary Pseudo-Clear Generation pipeline, training initially starts with pseudo-style conditioning videos. As training proceeds, these inputs are progressively replaced with increasingly realistic videos generated by the model itself. This strategy bridges the pseudo-to-real domain gap, allowing the model to adapt seamlessly to real-world input distributions and naturally support bidirectional clear adverse translation. Furthermore, to strictly enforce structural integrity and suppress hallucinations, we introduce Scene-Fidelity RL Optimization, a reward-driven policy optimization strategy that explicitly penalizes alterations to safety-critical driving elements. Extensive experiments demonstrate that RealWeather significantly outperforms existing methods in visual realism and structural preservation, while enabling robust long-tail weather scenario generation and strong zero-shot out-of-distribution generalization. Our video demos can be found at https://hust-umi.github.io/RealWeather/.
