轻量级提示引导的CLIP适应用于单目深度估计
Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
AI总结：
本文提出MoA-DepthCLIP框架，通过轻量级MoA模块和选择性微调提升单目深度估计的精度与效率，优于基线模型。
摘要：
利用视觉-语言模型（VLMs）如CLIP的丰富语义特征进行单目深度估计任务具有前景，但通常需要大量微调或缺乏几何精度。我们提出了一种参数高效的框架，名为MoA-DepthCLIP，通过最小监督适应预训练的CLIP表示进行单目深度估计。我们的方法将轻量级Mixture-of-Adapters（MoA）模块整合到预训练的视觉Transformer（ViT-B/32）骨干网络中，并结合最终层的选择性微调。这种设计实现了空间感知的适应，通过全局语义上下文向量和融合深度分箱分类与直接回归的混合预测架构进行引导。为提高结构精度，我们采用复合损失函数以强制几何约束。在NYU Depth V2基准测试中，MoA-DepthCLIP实现了有竞争力的结果，显著优于DepthCLIP基线，将δ₁准确率从0.390提升到0.745，并将RMSE从1.176降低到0.520。这些结果是在需要大量可训练参数的情况下取得的，证明了轻量级、提示引导的MoA在将VLM知识转移到细粒度单目深度估计任务中的有效性。
英文摘要：
Leveraging the rich semantic features of vision-language models (VLMs) like CLIP for monocular depth estimation tasks is a promising direction, yet often requires extensive fine-tuning or lacks geometric precision. We present a parameter-efficient framework, named MoA-DepthCLIP, that adapts pretrained CLIP representations for monocular depth estimation with minimal supervision. Our method integrates a lightweight Mixture-of-Adapters (MoA) module into the pretrained Vision Transformer (ViT-B/32) backbone combined with selective fine-tuning of the final layers. This design enables spatially-aware adaptation, guided by a global semantic context vector and a hybrid prediction architecture that synergizes depth bin classification with direct regression. To enhance structural accuracy, we employ a composite loss function that enforces geometric constraints. On the NYU Depth V2 benchmark, MoA-DepthCLIP achieves competitive results, significantly outperforming the DepthCLIP baseline by improving the $δ_1$ accuracy from 0.390 to 0.745 and reducing the RMSE from 1.176 to 0.520. These results are achieved while requiring substantially few trainable parameters, demonstrating that lightweight, prompt-guided MoA is a highly effective strategy for transferring VLM knowledge to fine-grained monocular depth estimation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
