面向恶劣天气去除的退化感知跨模态补偿提示学习
Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal
浏览论文内容
中文总结 AI 辅助
该研究针对恶劣天气导致图像退化影响视觉系统可靠性的问题，提出 DCMPC-Net 模型，通过跨模态提示补偿实现鲁棒的恶劣天气图像修复，性能优于现有最先进方法。
中文摘要 AI 辅助
恶劣天气会引发多样且复杂的图像退化，严重损害计算机视觉系统的可靠性。现有的一体化修复模型试图在统一框架内解决多种退化类型，但往往缺乏对退化特征的显式空间与语义建模，限制了其对不同天气状况的适应性。为解决这一局限，本文提出 Degradation-Aware Cross-Modal Prompt Compensation Network（DCMPC-Net，退化感知跨模态提示补偿网络），该网络利用预训练视觉-语言模型的跨模态退化线索，在统一骨干网络中对修复特征进行条件约束。具体而言，DCMPC-Net 主要由跨模态提示生成器（CMPG）、提示引导注意力对齐模块（PGAAM）和双特征补偿模块（DFCM）构成。CMPG 将文本嵌入与视觉特征相融合，生成编码退化相关语义与上下文线索的退化感知提示；这些提示通过 PGAAM 注入解码器，PGAAM 可自适应地将语义信息与退化区域对齐，以实现感知上下文的修复。为进一步提升结构保真度，本文引入 DFCM，该模块可将退化伪影与场景结构解耦，从而优化精细纹理与细节内容的重建。通过将跨模态语义指导与空间对齐、结构增强相融合，DCMPC-Net 可在不同天气状况下实现鲁棒且感知一致的图像修复。大量实验表明，DCMPC-Net 在特定任务与一体化设置中均优于现有最先进方法，具备更出色的准确率与视觉保真度。
英文摘要
Adverse weather causes diverse and complex image degradations, severely compromising the reliability of computer vision systems. Existing all-in-one restoration models attempt to address multiple degradation types within a unified framework, but often lack explicit spatial and semantic modeling of degradation characteristics, limiting their adaptability to diverse weather conditions. To address this limitation, we propose a Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net) that leverages cross-modal degradation cues from a pretrained vision-language model to condition restoration features within a unified backbone. Specifically, our DCMPC-Net mainly consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM). The CMPG integrates textual embeddings with visual features to produce degradation-aware prompts that encode degradation-related semantic and contextual cues. These prompts are injected into the decoder via a PGAAM, which adaptively aligns semantic information with degraded regions to facilitate context-aware restoration. To further enhance structural fidelity, DFCM is introduced that disentangles degradation artifacts from scene structures, thereby improving the reconstruction of fine textures and detailed content. By integrating cross-modal semantic guidance with spatial alignment and structural enhancement, DCMPC-Net achieves robust and perceptually consistent restoration across diverse weather conditions. Extensive experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings, achieving superior accuracy and visual fidelity.
发表机构
- School of Software Engineering, Dalian University（大连大学软件工程学院）
- School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）
- The Hong Kong Polytechnic University（香港理工大学）
- University of California, San Francisco（加利福尼亚大学旧金山分校）
- School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
机构由 AI 辅助整理，请以论文原文为准。
