XVO：通过跨模态自训练实现通用视觉里程计
XVO: Generalized Visual Odometry via Cross-Modal Self-Training
AI总结：
本研究提出半监督跨模态自训练方法XVO，利用多模态辅助任务和无约束行车视频训练通用单目VO模型，无需相机参数与微调即可在多基准数据集实现SOTA性能与跨场景迁移。
摘要：
我们提出XVO，这是一种用于训练通用单目视觉里程计（Visual Odometry，VO）模型的半监督学习方法，可在不同数据集和场景下实现鲁棒的即开即用运行。标准单目VO方法通常仅在单个数据集内研究已知校准参数的情况，与之不同，XVO能够高效地从视觉场景语义中恢复具有真实世界尺度的相对位姿，即无需依赖任何已知的相机参数。我们通过利用YouTube上大量无约束、异构的行车记录仪视频进行自训练，来优化运动估计模型。我们的核心贡献有两方面：第一，我们通过实验证明了半监督训练对于学习通用直接VO回归网络的优势；第二，我们验证了多模态监督（包括分割、光流、深度和音频辅助预测任务）能够为VO任务构建通用化表征。具体而言，我们发现音频预测任务可显著增强半监督学习过程，同时缓解伪标签噪声问题，在高动态、域外视频数据中效果尤为明显。我们提出的教师网络即便未使用多帧优化、也不具备相机参数先验知识，仍在常用的KITTI基准上取得了当前最优性能。结合所提出的半监督步骤，XVO在无需微调的情况下，在KITTI、nuScenes和Argoverse数据集的多种场景下展现出即开即用的知识迁移能力。
英文摘要：
We propose XVO, a semi-supervised learning method for training generalized monocular Visual Odometry (VO) models with robust off-the-self operation across diverse datasets and settings. In contrast to standard monocular VO approaches which often study a known calibration within a single dataset, XVO efficiently learns to recover relative pose with real-world scale from visual scene semantics, i.e., without relying on any known camera parameters. We optimize the motion estimation model via self-training from large amounts of unconstrained and heterogeneous dash camera videos available on YouTube. Our key contribution is twofold. First, we empirically demonstrate the benefits of semi-supervised training for learning a general-purpose direct VO regression network. Second, we demonstrate multi-modal supervision, including segmentation, flow, depth, and audio auxiliary prediction tasks, to facilitate generalized representations for the VO task. Specifically, we find audio prediction task to significantly enhance the semi-supervised learning process while alleviating noisy pseudo-labels, particularly in highly dynamic and out-of-domain video data. Our proposed teacher network achieves state-of-the-art performance on the commonly used KITTI benchmark despite no multi-frame optimization or knowledge of camera parameters. Combined with the proposed semi-supervised step, XVO demonstrates off-the-shelf knowledge transfer across diverse conditions on KITTI, nuScenes, and Argoverse without fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
