VisTa3D：用于从视觉、触觉和三维点云重建薄物体的数据集与基准
VisTa3D: A Dataset and Benchmark for Thin Object Reconstruction from Vision, Tactile, and 3D Point Clouds
浏览论文内容
中文总结 AI 辅助
VisTa3D是首个含视觉、触觉等数据的薄物体重建数据集，作者在其上基准测试现有模型发现其保真度低，还提出视觉-深度-触觉基线模型以探究触觉数据对薄物体重建的辅助作用。
中文摘要 AI 辅助
当前最先进的三维重建模型，无论基于视觉、深度数据还是两者结合，在薄物体上的性能往往不佳，部分原因是这类物体在RGB图像和三维点云中占据的空间很小。为测试这些模型的误差程度，我们收集了首个包含同步RGB图像、深度图和触觉响应图的薄物体数据集，其中每一帧都关联了惯性测量、相机位姿与标定，以及通过激光扫描薄物体获得的真实深度图和分割图。我们假设触觉数据可辅助薄物体重建，因为其响应图能提供局部形状与形变信息。我们的数据集名为VisTa3D，包含387个场景，覆盖17种环境中的70个薄物体。我们在VisTa3D上对现有三维重建模型进行基准测试，发现这些模型在薄物体上的保真度确实较低。为验证触觉数据是否有帮助，我们引入了首个视觉-深度-触觉三维重建模型作为基线。代码和数据可在this https URL获取。
英文摘要
State-of-the-art 3D reconstruction models, whether from visual, range, or both, tend to underperform on thin objects. This is partially due to the small amount of space such objects occupy in RGB images and in 3D point clouds. To test the extent of their errors, we collected the first thin object dataset comprising of synchronized RGB images, depth maps, and tactile response maps, where each frame is associated with inertial measurements, camera pose and calibration, and groundtruth depth and segmentation maps obtained from laser scanning of thin objects. We hypothesize that tactile data can aid in the reconstruction of thin objects as their response maps provide local shape and deformation information. Our dataset, termed VisTa3D, comprises of 387 scenes covering 70 thin objects over 17 environments. We benchmarked current 3D reconstruction models on VisTa3D and found that, indeed, they exhibit low fidelity on thin objects. To test if tactile data can help, we introduce the first visual-range-tactile 3D reconstruction model as a baseline. Code and data: https://huggingface.co/datasets/shaniaguo/VisTa3D.
发表机构
- Yale University（耶鲁大学）
- Yale Vision Laboratory（耶鲁视觉实验室）
- GRAB Lab（GRAB实验室）
机构由 AI 辅助整理，请以论文原文为准。
