XPoint：一种基于视觉-状态空间的自监督架构用于多光谱图像配准
XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration
AI总结：
XPoint是一种基于视觉-状态空间的自监督架构，通过模块化和自监督方法实现多光谱图像配准的高效适应与性能提升。
摘要：
准确的多光谱图像匹配面临显著挑战，原因包括不同光谱模态间的非线性强度变化、极端视角变化以及标注数据稀缺。目前最先进的方法通常专门针对单一光谱差异，如可见光-红外，由于依赖昂贵的监督信息，如深度图或相机姿态，难以适应其他模态。为了解决跨模态快速适应的需求，我们引入XPoint，一种自监督、模块化的图像匹配框架，旨在实现对对齐多光谱数据集的自适应训练和微调，允许用户根据特定任务自定义关键组件。XPoint通过模块化和自监督的方法，允许调整如基础检测器等元素，该检测器生成对视角和光谱变化不变的伪地面真实关键点。该框架集成了一个在分割任务上预训练的VMamba编码器，用于稳健的特征提取，并包含三个联合解码器头：两个专门用于兴趣点和描述符提取；一个任务特定的射影变换回归头对几何约束施加约束，以在图像配准等任务中实现更优性能。这种灵活的架构使XPoint能够快速适应广泛的模态，通过在光学-热数据上训练并在视觉-近红外、视觉-红外、视觉-长波红外和视觉-合成孔径雷达等设置上微调来证明。实验结果表明，XPoint在五个不同的多光谱数据集上，在特征匹配和图像配准任务中始终优于或匹配最先进的方法。我们的源代码可在https://github.com/canyagmur/XPoint上获得。
英文摘要：
Accurate multispectral image matching presents significant challenges due to non-linear intensity variations across spectral modalities, extreme viewpoint changes, and the scarcity of labeled datasets. Current state-of-the-art methods are typically specialized for a single spectral difference, such as visibleinfrared, and struggle to adapt to other modalities due to their reliance on expensive supervision, such as depth maps or camera poses. To address the need for rapid adaptation across modalities, we introduce XPoint, a self-supervised, modular image-matching framework designed for adaptive training and fine-tuning on aligned multispectral datasets, allowing users to customize key components based on their specific tasks. XPoint employs modularity and self-supervision to allow for the adjustment of elements such as the base detector, which generates pseudoground truth keypoints invariant to viewpoint and spectrum variations. The framework integrates a VMamba encoder, pretrained on segmentation tasks, for robust feature extraction, and includes three joint decoder heads: two are dedicated to interest point and descriptor extraction; and a task-specific homography regression head imposes geometric constraints for superior performance in tasks like image registration. This flexible architecture enables quick adaptation to a wide range of modalities, demonstrated by training on Optical-Thermal data and fine-tuning on settings such as visual-near infrared, visual-infrared, visual-longwave infrared, and visual-synthetic aperture radar. Experimental results show that XPoint consistently outperforms or matches state-ofthe-art methods in feature matching and image registration tasks across five distinct multispectral datasets. Our source code is available at https://github.com/canyagmur/XPoint.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
