DAP-LED：利用CLIP学习退化感知先验，用于联合低光增强与去模糊
DAP-LED: Learning Degradation-Aware Priors with CLIP for Joint Low-light Enhancement and Deblurring
AI总结：
针对夜间低光和运动模糊问题，提出DAP-LED联合学习框架，利用CLIP自适应感知退化程度，通过交叉融合模块和Transformer块优化，实现低光增强与去模糊，并在暗光下游任务中取得最先进性能。
摘要：
自动驾驶车辆和机器人在夜间常常因光照不足以及RGB相机长时间曝光引起的运动模糊而难以实现可靠的视觉感知。现有方法通过顺序连接现成的预训练低光增强和去模糊模型来解决这一挑战。然而，这些方法往往在过度曝光区域产生明显的伪影（例如颜色失真），或者使得学习暗区域的运动线索变得几乎不可能。在本文中，我们有趣地发现视觉语言模型，例如对比语言-图像预训练（CLIP），能够全面感知夜间多样的退化程度。鉴于此，我们提出了一种新颖的基于Transformer的联合学习框架，名为DAP-LED，该框架能够联合实现低光增强和去模糊，从而有益于下游任务，如暗光下的深度估计、分割和检测。其关键洞察在于利用CLIP自适应地从夜间图像中学习退化程度。这巧妙地使得学习丰富的语义信息和视觉表示以优化联合任务成为可能。为实现这一目标，我们首先引入一个CLIP引导的交叉融合模块，从图像嵌入中获取多尺度的分块退化热图。然后，通过设计的CLIP增强Transformer块融合热图，以保留有用的退化信息用于有效的模型优化。实验结果表明，与现有方法相比，我们的DAP-LED在暗光条件下达到了最先进的性能。同时，增强结果被证明对三个下游任务有效。有关演示和更多结果，请查看项目页面：\url{https://vlislab22.github.io/dap-led/}。
英文摘要：
Autonomous vehicles and robots often struggle with reliable visual perception at night due to the low illumination and motion blur caused by the long exposure time of RGB cameras. Existing methods address this challenge by sequentially connecting the off-the-shelf pretrained low-light enhancement and deblurring models. Unfortunately, these methods often lead to noticeable artifacts (\eg, color distortions) in the over-exposed regions or make it hardly possible to learn the motion cues of the dark regions. In this paper, we interestingly find vision-language models, \eg, Contrastive Language-Image Pretraining (CLIP), can comprehensively perceive diverse degradation levels at night. In light of this, we propose a novel transformer-based joint learning framework, named DAP-LED, which can jointly achieve low-light enhancement and deblurring, benefiting downstream tasks, such as depth estimation, segmentation, and detection in the dark. The key insight is to leverage CLIP to adaptively learn the degradation levels from images at night. This subtly enables learning rich semantic information and visual representation for optimization of the joint tasks. To achieve this, we first introduce a CLIP-guided cross-fusion module to obtain multi-scale patch-wise degradation heatmaps from the image embeddings. Then, the heatmaps are fused via the designed CLIP-enhanced transformer blocks to retain useful degradation information for effective model optimization. Experimental results show that, compared to existing methods, our DAP-LED achieves state-of-the-art performance in the dark. Meanwhile, the enhanced results are demonstrated to be effective for three downstream tasks. For demo and more results, please check the project page: \url{https://vlislab22.github.io/dap-led/}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
