VTLoc：基于学习的视觉点云中触觉接触定位

中文摘要 AI 辅助

视觉和触觉是机器人感知与操作的重要互补模态。视觉提供全局物体信息，触觉提供接触点精确局部信息。融合二者进行接触定位面临挑战，因触觉数据与视觉几何需精确空间对齐。为此提出VTLoc框架，利用3D点云作为视觉输入从触觉读数定位接触点。它含几何多模态对齐模块和迭代定位更新器。在100个真实物体新基准上评估，VTLoc减少局部到全局对应模糊性，改善单触接触定位。

英文摘要

Vision and touch are complementary modalities essential for robotic perception and manipulation. While vision provides global object context, touch offers precise local information at contact points. Integrating these modalities for contact localization, i.e., predicting the location of touch on an object's surface, poses significant challenges due to the need for accurate spatial alignment between tactile data and visual geometry. To address this challenge, we propose VTLoc, a novel visual-tactile framework that localizes contact points from tactile readings using a 3D point cloud as visual input. VTLoc introduces two key components: a geometric multi-modal alignment module, which reconstructs a pseudo-point cloud from fused visual-tactile features and aligns it with the visual point cloud to enforce spatial consistencies across modalities; and an iterative localizing updater, which iteratively refines the predicted contact location using fused visual-tactile features. Evaluated on a new benchmark of 100 real-world objects, VTLoc improves single-touch contact localization by reducing local-to-global correspondence ambiguity.

发表机构

• Department of Engineering, King’s College London（伦敦国王学院工程系）

机构由 AI 辅助整理，请以论文原文为准。