arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.16850cs.CV
收藏

使用视觉基础模型和交叉注意力机制的鲁棒场景变化检测

Robust Scene Change Detection Using Visual Foundation Models and Cross-Attention Mechanisms

Australian Institute for Machine Learning (AIML)(澳大利亚机器学习研究所);University of Adelaide(阿德莱德大学)

Chun-Jung Lin, Sourav Garg, Tat-Jun Chin, Feras Dayoub

AI总结:

提出基于DINOv2和全图交叉注意力的场景变化检测方法,通过冻结主干保留特征通用性并处理视角变化,在VL-CMU-CD和PSCD基准上显著提升F1分数,展现卓越泛化能力。

摘要:

我们提出了一种新的场景变化检测方法,该方法利用视觉基础模型DINOv2的鲁棒特征提取能力,并集成全图交叉注意力以解决光照变化、季节变化和视角差异等关键挑战。为了有效地学习图像对之间用于变化检测任务的对应关系和错对应关系,我们建议a)“冻结”主干网络以保留密集基础特征的通用性,以及b)采用“全图”交叉注意力以更好地处理图像对之间的视角变化。我们在两个基准数据集VL-CMU-CD和PSCD及其视角变化版本上评估了我们的方法。我们的实验表明,F1分数有显著提高,特别是在涉及图像对之间几何变化的场景中。结果表明,我们的方法比现有的最先进方法具有更好的泛化能力,对光度变化和几何变化具有鲁棒性,并且在微调以适应新环境时具有更好的整体泛化能力。详细的消融研究进一步验证了我们架构中每个组件的贡献。我们的源代码可在https://github.com/ChadLin9596/Robust-Scene-Change-Detection获取。

英文摘要:

We present a novel method for scene change detection that leverages the robust feature extraction capabilities of a visual foundational model, DINOv2, and integrates full-image cross-attention to address key challenges such as varying lighting, seasonal variations, and viewpoint differences. In order to effectively learn correspondences and mis-correspondences between an image pair for the change detection task, we propose to a) ``freeze'' the backbone in order to retain the generality of dense foundation features, and b) employ ``full-image'' cross-attention to better tackle the viewpoint variations between the image pair. We evaluate our approach on two benchmark datasets, VL-CMU-CD and PSCD, along with their viewpoint-varied versions. Our experiments demonstrate significant improvements in F1-score, particularly in scenarios involving geometric changes between image pairs. The results indicate our method's superior generalization capabilities over existing state-of-the-art approaches, showing robustness against photometric and geometric variations as well as better overall generalization when fine-tuned to adapt to new environments. Detailed ablation studies further validate the contributions of each component in our architecture. Our source code is available at: https://github.com/ChadLin9596/Robust-Scene-Change-Detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑