发表机构
Zhejiang University; Zhejiang University of Technology; Shanghai Jiaotong University（浙江大学; 浙江工业大学; 上海交通大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Pumpire是用于评估带/不带深度先验的图像、视频级3D基础模型点对距离估计能力的统一基准，含6400帧的pumpire-6k数据集，经29种基线实验分析，填补了物理尺度感知评估的空白。
AI 中文摘要
我们提出了Pumpire，这是一个统一基准，用于评估图像级和视频级3D基础模型（无论是否带有深度先验）的度量点对距离估计能力。与以往通常分别评估深度和相机内参、或使用几何相似性指标评估点云（无法直接反映模型的点对距离估计能力）的方法不同，Pumpire直接从重建的几何结构评估点对距离。为此，我们收集了大规模多样化数据集pumpire-6k，包含100个真实场景，每个场景都标注了物理测量的点对距离，且包含64帧，总共有6400帧。基于该数据集，我们建立了全面的评估协议，涵盖图像级和视频级3D基础模型，可直接评估点对距离误差并实现跨设置比较。我们对29种代表性3D基础模型的基线配置进行了大量实验，并对结果进行了全面分析。通过提供该基准，我们旨在解决先前评估协议大多忽略的、重建3D空间中感知和估计物理尺度的更基础能力。项目页面可通过此https URL获取。
英文摘要
We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation models, with or without depth priors. In contrast to previous approaches that normally evaluate depth and camera intrinsics separately or evaluate point-clouds with geometric similarity metrics, which cannot directly reflect models' point-to-point distance estimation capability, Pumpire directly assesses point-to-point distances from the reconstructed geometry. To this end, we collect a large-scale and diverse dataset (pumpire-6k) comprising 100 real-world scenes, each annotated with physically measured point-pair distances and containing 64 frames, for a total of 6,400 frames. Building on this dataset, we establish a holistic evaluation protocol that covers both image- and video-level 3D foundation models and enables direct assessment of point-pair distance errors and cross-setting comparison. We conduct extensive experiments across 29 baseline configurations of representative 3D foundation models and provide a comprehensive analysis of the results. By offering this benchmark, we target the more fundamental ability to perceive and estimate physical scale in the reconstructed 3D space, which prior evaluation protocols have largely overlooked. The project page can be found at https://pumpire.github.io/
CommentsProject page: https://pumpire.github.io/
