DepthAgent: 通过样本级专家选择实现更好的通用深度估计
DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selection
AI总结：
提出DepthAgent，一种视觉语言智能体，通过多轮工具调用和强化学习微调，自适应地为不同相机设置（透视、鱼眼、全景）选择或融合多个深度估计专家模型，显著提升通用深度估计性能。
摘要：
单目度量深度估计通过大规模训练和通用相机建模取得了显著进展，但在不同相机设置（如透视、鱼眼和全景图像）下的鲁棒部署仍然具有挑战性。现有方法通常依赖单一深度估计器，忽略了不同模型编码不同的相机假设并在不同输入域下表现最佳。本文中，我们展示了深度专家在样本级上具有强互补性：模型偏好与相机几何高度相关，多模型融合在单个专家不可靠的困难样本上带来最大收益。受这些观察启发，我们提出了 extbf{\ours}，一种用于自适应单目深度估计的视觉语言智能体。DepthAgent将现有深度模型视为冻结工具，学习分析场景和相机线索，通过多轮工具调用调用合适的专家，并为每个输入选择或融合它们的预测。为了优化这种离散决策以实现密集几何质量，我们设计了一种多奖励强化学习微调方案，共同鼓励有效的工具执行、相机/场景分析、专家选择质量和推理效率。在透视、鱼眼和全景基准上的大量实验表明，\ours一致优于单个专家、固定模型融合和不同选择策略，在困难样本上取得了显著改进，突显了专家选择和融合的关键作用。代码和模型将在发表后发布。
英文摘要：
Monocular metric depth estimation has achieved strong progress with large-scale training and universal-camera modeling, yet robust deployment across diverse camera settings, such as perspective, fisheye, and panoramic images, remains challenging. Existing methods typically rely on a single depth estimator, overlooking that different models encode different camera assumptions and perform best under different input domains. In this paper, we show that depth experts exhibit strong sample-wise complementarity: model preference is highly correlated with camera geometry, and multi-model fusion brings the largest gains on difficult samples where individual experts are unreliable. Motivated by these observations, we propose \textbf{\ours}, a vision-language agent for adaptive monocular depth estimation. DepthAgent treats existing depth models as frozen tools and learns to analyze scene and camera cues, invoke suitable experts through multi-turn tool utilization, and select or fuse their predictions for each input. To optimize such discrete decision-making toward dense geometric quality, we design a multi-reward reinforcement fine-tuning scheme that jointly encourages valid tool execution, camera/scene analysis, expert-selection quality, and inference efficiency. Extensive experiments across perspective, fisheye, and panoramic benchmarks show that \ours consistently outperforms individual experts, fixed model fusion, and different selection strategies, with strong improvements on challenging samples, highlighting the critical role of expert selection and fusion. The code and model will be released upon publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
