MWFormer：基于退化感知Transformer的多天气图像恢复
MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers
AI总结：
针对现有方法仅能处理单一天气退化的不足，提出MWFormer多天气Transformer，通过超网络和特征调制实现统一架构处理多种天气退化，在基准上性能优于现有方法且可控性更强。
摘要：
恢复恶劣天气条件下拍摄的图像是许多计算机视觉应用的基础任务。然而，大多数现有天气恢复方法仅能处理特定类型的退化，这在雨雪或雨雾等现实场景中往往不足。为解决此类情况，我们提出多天气Transformer（简称MWFormer），这是一种整体视觉Transformer，旨在通过单一统一架构解决多种天气诱导的退化问题。MWFormer使用超网络和特征级线性调制块，利用同一组学习参数恢复受多种天气类型退化的图像。我们首先采用对比学习训练辅助网络，提取内容无关、失真感知的特征嵌入，以有效表示预测的天气类型（可能存在多种）。在这些天气感知预测的引导下，图像恢复Transformer自适应调制其参数，以应对多种可能的天气，进行局部和全局特征处理。此外，MWFormer在应用期间支持一种新颖的调优方式，无需重新训练即可针对单一类型天气恢复或混合天气恢复，提供比现有方法更强的可控性。我们在多天气恢复基准上的实验结果表明，MWFormer相比现有最先进方法实现了显著的性能提升，且无需大量计算成本。此外，我们证明使用超网络的方法可集成到各种网络架构中，进一步提升其性能。代码可在https://github.com/taco-group/MWFormer获取。
英文摘要：
Restoring images captured under adverse weather conditions is a fundamental task for many computer vision applications. However, most existing weather restoration approaches are only capable of handling a specific type of degradation, which is often insufficient in real-world scenarios, such as rainy-snowy or rainy-hazy weather. Towards being able to address these situations, we propose a multi-weather Transformer, or MWFormer for short, which is a holistic vision Transformer that aims to solve multiple weather-induced degradations using a single, unified architecture. MWFormer uses hyper-networks and feature-wise linear modulation blocks to restore images degraded by various weather types using the same set of learned parameters. We first employ contrastive learning to train an auxiliary network that extracts content-independent, distortion-aware feature embeddings that efficiently represent predicted weather types, of which more than one may occur. Guided by these weather-informed predictions, the image restoration Transformer adaptively modulates its parameters to conduct both local and global feature processing, in response to multiple possible weather. Moreover, MWFormer allows for a novel way of tuning, during application, to either a single type of weather restoration or to hybrid weather restoration without any retraining, offering greater controllability than existing methods. Our experimental results on multi-weather restoration benchmarks show that MWFormer achieves significant performance improvements compared to existing state-of-the-art methods, without requiring much computational cost. Moreover, we demonstrate that our methodology of using hyper-networks can be integrated into various network architectures to further boost their performance. The code is available at: https://github.com/taco-group/MWFormer
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
