一场景,两深度:探究单目基础模型中的几何歧义性
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models
AI总结:
本文提出稀疏双层序数基准MD-3k,用于测量单目深度基础模型的深度层偏好和多层空间关系准确性,发现不同模型对同一分层几何结构有不同解析,且拉普拉斯视觉提示可改变冻结模型的输出层。
摘要:
一个忠实的三维世界表示应考虑分层几何结构,其中单条相机射线可能包含多个可见且几何有效的表面。然而,单目深度估计将此结构简化为每个像素一个标量深度。透明场景使这种歧义性可测量:同一条射线可以穿过前景玻璃并观察到背景,将监督目标转变为标注、数据和训练的一种约定,而非场景固有的真实。学习到的预测器将其深度层偏好暴露出来。我们引入了MultiDepth-3k(MD-3k),一个稀疏双层序数基准,用于测量深度层偏好和多层空间关系准确性(ML-SRA)。在MD-3k上,领先的深度基础模型在标准RGB输入下表现出多样的层偏好,表明相同的分层几何结构在不同模型中可被不同解析。我们进一步发现,拉普拉斯视觉提示(LVP),一种无需训练的光谱输入变换,可以显著改变某些冻结模型的报告层。最强的RGB/LVP组合DAv2-L达到了75.5%的ML-SRA。这些结果表明,深度基础模型可能表达了互补的几何假设,而标准RGB推理未将其表达出来。我们邀请社区通过歧义感知的视角重新思考深度监督和评估,将多个有效的三维解释视为待测量、保留和表达的几何结构。
英文摘要:
A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces. Monocular depth estimation, however, reduces this structure to one scalar depth per pixel. Transparent scenes make this ambiguity measurable: the same ray can pass through foreground glass and observe the background, turning the supervised target into a convention of annotation, data, and training rather than a scene-intrinsic truth. A learned predictor exposes this convention as its depth-layer preference. We introduce MultiDepth-3k (MD-3k), a sparse two-layer ordinal benchmark for measuring depth-layer preference and multi-layer spatial relationship accuracy (ML-SRA). On MD-3k, leading depth foundation models exhibit diverse layer preferences under standard RGB input, showing that the same layered geometry can be resolved differently across models. We further find that Laplacian Visual Prompting (LVP), a training-free spectral input transformation, can substantially change the reported layer for certain frozen models. The strongest RGB/LVP pair, DAv2-L, reaches 75.5% ML-SRA. These results suggest that depth foundation models may express complementary geometric hypotheses that standard RGB inference leaves unexpressed. We invite the community to rethink depth supervision and evaluation through an ambiguity-aware lens, where multiple valid 3D interpretations are treated as geometric structure to be measured, preserved, and expressed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。