EndoDAC：面向任意内窥镜相机自监督深度估计的基础模型高效适配方法
EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera
AI总结：
针对基础模型直接用于内窥镜深度估计效果不佳的问题，提出高效自监督框架EndoDAC，通过DV-LoRA、卷积颈模块及相机内参自估计策略，仅用单目手术视频和少量参数训练即取得更优性能。
摘要：
深度估计在内窥镜手术的导航、表面重建、增强现实可视化等各类任务中发挥着关键作用。尽管基础模型在包括深度估计在内的视觉任务中已取得显著成果，但将其直接应用于医学领域往往效果欠佳，这凸显了开发高效适配方法、让这类模型适用于内窥镜深度估计的必要性。我们提出了内窥镜任意相机深度估计框架（Endoscopic Depth Any Camera，EndoDAC），这是一种将基础模型适配到内窥镜场景的高效自监督深度估计框架。具体而言，我们设计了基于动态向量的低秩适配（Dynamic Vector-Based Low-Rank Adaptation，DV-LoRA），并引入卷积颈（Convolutional Neck）模块，仅用极少的可训练参数就让基础模型适配手术领域。考虑到相机信息并非总能获取，我们还提出了一种自监督适配策略，通过位姿编码器估计相机内参。我们的框架仅需使用来自任意相机的单目手术视频即可训练，训练成本极低。实验表明，即便训练轮次更少且无需已知相机内参真值，我们的方法仍能取得更优性能。代码已开源至https://github.com/BeileiCui/EndoDAC。
英文摘要：
Depth estimation plays a crucial role in various tasks within endoscopic surgery, including navigation, surface reconstruction, and augmented reality visualization. Despite the significant achievements of foundation models in vision tasks, including depth estimation, their direct application to the medical domain often results in suboptimal performance. This highlights the need for efficient adaptation methods to adapt these models to endoscopic depth estimation. We propose Endoscopic Depth Any Camera (EndoDAC) which is an efficient self-supervised depth estimation framework that adapts foundation models to endoscopic scenes. Specifically, we develop the Dynamic Vector-Based Low-Rank Adaptation (DV-LoRA) and employ Convolutional Neck blocks to tailor the foundational model to the surgical domain, utilizing remarkably few trainable parameters. Given that camera information is not always accessible, we also introduce a self-supervised adaptation strategy that estimates camera intrinsics using the pose encoder. Our framework is capable of being trained solely on monocular surgical videos from any camera, ensuring minimal training costs. Experiments demonstrate that our approach obtains superior performance even with fewer training epochs and unaware of the ground truth camera intrinsics. Code is available at https://github.com/BeileiCui/EndoDAC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
