通过双阶段蒸馏实现无约束事件-图像特征匹配的无标签目标域适应
Label-Free Target-Domain Adaptation for Unconstrained Event-Image Feature Matching via Dual-Stage Distillation
浏览论文内容
中文总结 AI 辅助
研究针对无约束事件-图像特征匹配问题，提出双阶段训练范式，先进行无标签蒸馏预训练，再引入极线引导自蒸馏框架，在新的跨模态评估基准上实验，取得领先的姿态估计性能。
中文摘要 AI 辅助
在事件和图像数据之间建立像素级对应关系是多传感器系统的一项基本任务。然而，现有的跨模态匹配方法很大程度上受到对匹配标签或严格对齐硬件的依赖限制，无法应用于无标签和无约束的现实场景。为此，提出一种新颖的两阶段训练范式。首先利用大规模数据进行无标签蒸馏预训练，通过基于分布和对比损失升级优化目标以学习高度通用的表示。其次引入极线引导的自蒸馏框架，利用一致性验证分离稳健匹配并结合极线先验导出的几何置信度，使模型能在目标域无监督自进化。还引入基于TUM-VIE的严格跨模态评估基准。实验表明该方法在MVSEC和TUM-VIE姿态估计任务上取得了领先性能。
英文摘要
Building pixel-level correspondence between event and image data is a fundamental task for multi-sensor systems. However, existing cross-modal matching methods are largely restricted by their reliance on either matching labels or strictly aligned hardware, which limits them to unlabeled and unconstrained real-world scenarios where neither matching ground truth nor prior sensor relationships are available. To address this, we propose a novel two-stage training paradigm. First, we leverage large-scale data to perform label-agnostic distillation pretraining, upgrading optimization objectives with distribution-based and contrastive losses to learn highly generalizable representations. Second, to tackle unlabeled and unconstrained downstream data, we introduce an epipolar-guided self-distillation framework. By utilizing consistency verification to isolate robust matches and incorporating geometric confidence derived from an external epipolar prior, our model can effectively self-evolve directly on target domains without any supervision. Furthermore, we introduce a rigorous cross-modal evaluation benchmark based on TUM-VIE, featuring physically separated cameras with distinct intrinsic parameters and resolutions. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance on both MVSEC and TUM-VIE pose estimation tasks. The source code and benchmark will be made publicly available at https://github.com/ZhonghuaYi/nexus2-official.
发表机构
- Zhejiang University（浙江大学）
- Hunan University（湖南大学）
- National University of Defense Technology（国防科技大学）
- Ant Group（蚂蚁集团）
机构由 AI 辅助整理，请以论文原文为准。
