将基础单目深度估计器扩展到鱼眼相机：校准标记
Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
AI总结：
本文提出一种方法，通过校准标记将训练于透视图像的基础单目深度估计器扩展至鱼眼图像，无需重新训练即可对齐潜在嵌入分布，提升深度估计精度。
摘要：
我们提出了一种方法，将训练于透视图像的基础单目深度估计器（FMDEs）扩展至鱼眼图像。尽管训练于数千万张图像，FMDEs 对相机校准（内参、畸变）参数的变化导致的协变量偏移敏感，导致深度估计错误。我们的方法对齐了编码鱼眼图像的潜在嵌入分布与透视图像的分布，使FMDEs能用于鱼眼相机而无需重新训练或微调。为此，我们引入一组校准标记作为轻量级适应机制，调节潜在嵌入以实现对齐。通过利用FMDEs已具表现力的潜在空间，我们提出调节其嵌入可以避免传统重新校准或映射到图像空间标准参考框架时引入的负影响。我们的方法是自监督的，不需要鱼眼图像，而是利用公开的大规模透视图像数据集。这通过将透视图像重新校准到鱼眼图像，并在训练中强制其估计的一致性来实现。我们在多个FMDEs上评估了我们的方法，在室内外场景中，使用单组标记始终优于现有最先进方法。代码可在：https://github.com/JungHeeKim29/calibration-token获取。
英文摘要：
We propose a method to extend foundational monocular depth estimators (FMDEs), trained on perspective images, to fisheye images. Despite being trained on tens of millions of images, FMDEs are susceptible to the covariate shift introduced by changes in camera calibration (intrinsic, distortion) parameters, leading to erroneous depth estimates. Our method aligns the distribution of latent embeddings encoding fisheye images to those of perspective images, enabling the reuse of FMDEs for fisheye cameras without retraining or finetuning. To this end, we introduce a set of Calibration Tokens as a light-weight adaptation mechanism that modulates the latent embeddings for alignment. By exploiting the already expressive latent space of FMDEs, we posit that modulating their embeddings avoids the negative impact of artifacts and loss introduced in conventional recalibration or map projection to a canonical reference frame in the image space. Our method is self-supervised and does not require fisheye images but leverages publicly available large-scale perspective image datasets. This is done by recalibrating perspective images to fisheye images, and enforcing consistency between their estimates during training. We evaluate our approach with several FMDEs, on both indoors and outdoors, where we consistently improve over state-of-the-art methods using a single set of tokens for both. Code available at: https://github.com/JungHeeKim29/calibration-token; https://github.com/Suchisrit/CalibrationTokens.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
