arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.09831cs.CV
收藏

深入探究基于扩散模型的对比学习用于鲁棒深度估计

Digging into contrastive learning for robust depth estimation with diffusion models

Beijing Jiaotong University(北京交通大学);Nanyang Technological University(南洋理工大学);Beihang University(北京航空航天大学)

Jiyuan Wang, Chunyu Lin, Lang Nie, Kang Liao, Shuwei Shao, Yao Zhao

AI总结:

提出D4RD方法,通过定制对比学习与知识蒸馏构建多级“三位一体”对比方案,并改进基线扩散模型,显著提升了复杂环境下深度估计的鲁棒性。

摘要:

近期,基于扩散模型的深度估计方法因其优雅的去噪模式和出色的性能而引起了广泛关注。然而,在现实场景中普遍存在的恶劣条件(如雨天、雪天等)下,它们通常不可靠。在本文中,我们提出了一种名为D4RD的新型鲁棒深度估计方法,其采用专为扩散模型定制的对比学习模式,以缓解复杂环境中的性能退化问题。具体而言,我们将知识蒸馏的优势整合到对比学习中,构建了“三位一体”对比方案。该方案利用前向扩散过程的采样噪声作为自然参考,引导不同场景中的预测噪声趋向更稳定和精确的最优解。此外,我们将噪声级的三位一体扩展至更通用的特征级和图像级,建立多级对比,将鲁棒感知的负担分散到整个网络中。在处理复杂场景之前,我们通过三项简单但有效的改进增强了基线扩散模型的稳定性,这促进了收敛并消除了深度异常值。大量实验表明,D4RD在合成损坏数据集和真实世界天气条件上均超越了现有的最先进解决方案。源代码和数据可在https://github.com/wangjiyuan9/D4RD获取。

英文摘要:

Recently, diffusion-based depth estimation methods have drawn widespread attention due to their elegant denoising patterns and promising performance. However, they are typically unreliable under adverse conditions prevalent in real-world scenarios, such as rainy, snowy, etc. In this paper, we propose a novel robust depth estimation method called D4RD, featuring a custom contrastive learning mode tailored for diffusion models to mitigate performance degradation in complex environments. Concretely, we integrate the strength of knowledge distillation into contrastive learning, building the `trinity' contrastive scheme. This scheme utilizes the sampled noise of the forward diffusion process as a natural reference, guiding the predicted noise in diverse scenes toward a more stable and precise optimum. Moreover, we extend noise-level trinity to encompass more generic feature and image levels, establishing a multi-level contrast to distribute the burden of robust perception across the overall network. Before addressing complex scenarios, we enhance the stability of the baseline diffusion model with three straightforward yet effective improvements, which facilitate convergence and remove depth outliers. Extensive experiments demonstrate that D4RD surpasses existing state-of-the-art solutions on synthetic corruption datasets and real-world weather conditions. Source code and data are available at \url{https://github.com/wangjiyuan9/D4RD}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑