arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02020cs.CV

在恶劣成像条件下适应Depth Anything以应对事件

Adapting Depth Anything to Adverse Imaging Conditions with Events

National Key Lab of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(multispectral information intelligent processing technology 国家重点实验室,人工智能与自动化学院,华中科技大学);School of Computing, National University of Singapore(computing 学院,新加坡国立大学);School of Computer Science and Engineering, Sun Yat-Sen University(computer science and engineering 学院,中山大学)

Shihan Peng, Yuyang Xiong, Hanyu Zhou, Zhiwei Shi, Haoyue Liu, Gang Chen, Luxin Yan, Yi Chang

AI总结:

本文提出ADAE框架,通过熵感知空间融合和运动引导时间校正,提升Depth Anything在恶劣成像条件下的深度估计性能。

摘要:

在动态和恶劣光照条件下实现鲁棒的深度估计对于机器人系统至关重要。目前,深度基础模型,如Depth Anything,在理想场景中取得了巨大成功,但在恶劣成像条件下,如极端光照和运动模糊,仍面临挑战。这些退化会破坏帧相机的视觉信号,削弱帧基深度在空间和时间维度上的判别特征。通常,现有方法会结合事件相机,利用其高动态范围和时间分辨率,以补偿被破坏的帧特征。然而,此类专门融合模型主要从头开始在领域特定数据集上训练,因此无法继承基础模型固有的开放世界知识和鲁棒泛化能力。在本工作中,我们提出了ADAE,一种用于退化场景中Depth Anything的事件引导空间时间融合框架。我们的设计基于两个关键洞察:1)熵感知空间融合。我们使用信息熵策略自适应地融合帧基和事件基特征,以指示光照引起的退化。2)运动引导的时间校正。我们利用事件基运动线索来校正模糊区域中的模糊特征。在我们的统一框架下,这两个组件相互补充,并共同增强Depth Anything在恶劣成像条件下的性能。进行了大量实验以验证所提方法的优越性。我们的代码将在接受后发布。

英文摘要:

Robust depth estimation under dynamic and adverse lighting conditions is essential for robotic systems. Currently, depth foundation models, such as Depth Anything, achieve great success in ideal scenes but remain challenging under adverse imaging conditions such as extreme illumination and motion blur. These degradations corrupt the visual signals of frame cameras, weakening the discriminative features of frame-based depths across the spatial and temporal dimensions. Typically, existing approaches incorporate event cameras to leverage their high dynamic range and temporal resolution, aiming to compensate for corrupted frame features. However, such specialized fusion models are predominantly trained from scratch on domain-specific datasets, thereby failing to inherit the open-world knowledge and robust generalization inherent to foundation models. In this work, we propose ADAE, an event-guided spatiotemporal fusion framework for Depth Anything in degraded scenes. Our design is guided by two key insights: 1) Entropy-Aware Spatial Fusion. We adaptively merge frame-based and event-based features using an information entropy strategy to indicate illumination-induced degradation. 2) Motion-Guided Temporal Correction. We resort to the event-based motion cue to recalibrate ambiguous features in blurred regions. Under our unified framework, the two components are complementary to each other and jointly enhance Depth Anything under adverse imaging conditions. Extensive experiments have been performed to verify the superiority of the proposed method. Our code will be released upon acceptance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑