arXiv 2504.051352025-04-07cs.CV

DA2Diff：探索面向一体化天气复原的退化感知自适应扩散先验

DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration

School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；Collaborative Innovation Center of Novel Software Technology and Industrialization（新型软件技术与产业化协同创新中心）；College of Computer Science and Technology, Anhui University of Technology（安徽工业大学计算机科学与技术学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）

Jiamei Xiong, Xuefeng Yan, Yongzhen Wang, Wei Zhao, Xiao-Ping Zhang, Mingqiang Wei

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DA2Diff，利用CLIP退化感知提示和动态专家选择调制器构建自适应扩散先验，实现多种天气退化的一体化图像复原，并在定量与定性评估中优于现有方法。

摘要：

在恶劣天气条件下的图像复原是许多基于视觉的应用中的关键任务。最近，在统一模型中处理多种天气退化的一体化框架已展现出潜力。然而，不同天气条件下退化模式的多样性，以及真实世界退化复杂多变的特性，给多天气去除带来了重大挑战。为应对这些挑战，我们提出了一种创新的扩散范式，该范式具有面向一体化天气复原的退化感知自适应先验，称为DA2Diff。这是一项新的探索，应用CLIP来感知退化感知属性，以实现更好的多天气复原。具体而言，我们部署一组可学习提示，通过在CLIP空间中的提示-图像相似性约束来捕获退化感知表示。通过将雪/雾/雨图像与雪/雾/雨提示对齐，每个提示对不同天气退化特征做出贡献。学习到的提示随后通过设计的天气特定提示引导模块集成到扩散模型中，使得复原多种天气类型成为可能。为了进一步提高对复杂天气退化的适应性，我们提出了一种动态专家选择调制器，该调制器采用动态天气感知路由器，为每个天气失真图像灵活分派不同数量的复原专家，从而使扩散模型能够自适应地复原多样化的退化。实验结果在定量和定性评估中证实了DA2Diff相对于最先进方法的优异性能。源代码将在录用后公开。

英文摘要：

Image restoration under adverse weather conditions is a critical task for many vision-based applications. Recent all-in-one frameworks that handle multiple weather degradations within a unified model have shown potential. However, the diversity of degradation patterns across different weather conditions, as well as the complex and varied nature of real-world degradations, pose significant challenges for multiple weather removal. To address these challenges, we propose an innovative diffusion paradigm with degradation-aware adaptive priors for all-in-one weather restoration, termed DA2Diff. It is a new exploration that applies CLIP to perceive degradation-aware properties for better multi-weather restoration. Specifically, we deploy a set of learnable prompts to capture degradation-aware representations by the prompt-image similarity constraints in the CLIP space. By aligning the snowy/hazy/rainy images with snow/haze/rain prompts, each prompt contributes to different weather degradation characteristics. The learned prompts are then integrated into the diffusion model via the designed weather specific prompt guidance module, making it possible to restore multiple weather types. To further improve the adaptiveness to complex weather degradations, we propose a dynamic expert selection modulator that employs a dynamic weather-aware router to flexibly dispatch varying numbers of restoration experts for each weather-distorted image, allowing the diffusion model to restore diverse degradations adaptively. Experimental results substantiate the favorable performance of DA2Diff over state-of-the-arts in quantitative and qualitative evaluation. Source code will be available after acceptance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。