因材施教：基于提示词池与Depth-Anything约束的恶劣天气图像恢复
Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint
- Xiamen University（厦门大学）
- The Hong Kong University of Science Technology (Guangzhou)（香港科技大学（广州））
- The Hong Kong University of Science Technology（香港科技大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对恶劣天气退化的复杂组合，提出T3-DiffWeather框架，利用提示词池自适应构建天气提示词，并结合Depth-Anything特征约束与对比损失优化扩散过程，在多数据集上达到SOTA且计算效率更优。
AI中文摘要：
近期恶劣天气图像恢复技术取得进展，但现实世界中不可预测且多样的天气退化组合带来了巨大挑战。以往方法难以动态处理复杂的退化组合并精确进行背景重建，导致性能和泛化受限。受提示词学习与“因材施教”理念启发，我们提出新框架T3-DiffWeather。具体而言，我们采用提示词池，使网络能自主组合子提示词构建天气提示词，利用必要属性自适应应对未知天气输入。此外，从场景建模角度，我们引入受Depth-Anything特征约束的通用提示词，为扩散过程提供特定场景条件。进一步地，通过引入对比提示词损失，利用相互推拒策略确保两类提示词具有独特表征。实验表明，本方法在多个合成与真实数据集上达到SOTA性能，且在计算效率上显著优于现有扩散技术。
英文摘要：
Recent advancements in adverse weather restoration have shown potential, yet the unpredictable and varied combinations of weather degradations in the real world pose significant challenges. Previous methods typically struggle with dynamically handling intricate degradation combinations and carrying on background reconstruction precisely, leading to performance and generalization limitations. Drawing inspiration from prompt learning and the "Teaching Tailored to Talent" concept, we introduce a novel pipeline, T3-DiffWeather. Specifically, we employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts, harnessing the necessary attributes to adaptively tackle unforeseen weather input. Moreover, from a scene modeling perspective, we incorporate general prompts constrained by Depth-Anything feature to provide the scene-specific condition for the diffusion process. Furthermore, by incorporating contrastive prompt loss, we ensures distinctive representations for both types of prompts by a mutual pushing strategy. Experimental results demonstrate that our method achieves state-of-the-art performance across various synthetic and real-world datasets, markedly outperforming existing diffusion techniques in terms of computational efficiency.
