语义接触场用于类别级可泛化的触觉工具操作
Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation
AI总结：
SCFields通过融合视觉语义与密集接触估计，实现类别级可泛化的触觉工具操作，显著优于视觉和原始触觉基线。
摘要：
通用工具操作需要同时具备语义规划和精确的物理控制。现代通用机器人策略，如视觉-语言-动作（VLA）模型，通常缺乏接触丰富工具操作所需的高保真的物理基础。相反，现有的接触感知策略利用触觉或力觉传感通常都是实例特定的，无法在多样化的工具几何形状上进行泛化。弥合这一差距需要从多样化数据中学习统一的接触表示，但仍然存在根本性的障碍：现实世界的触觉数据在大规模上是不可行的，而直接的零样本仿真到现实转移由于非线性变形的软传感器复杂动力学而具有挑战性。为此，我们提出了语义接触场（SCFields），一种融合视觉语义与密集接触估计的统一3D表示。我们通过两阶段的仿真到现实接触学习流程实现这一目标：首先，在大规模仿真数据集上预训练以学习通用接触物理；其次，在少量真实数据上进行微调，通过几何启发法和力优化伪标记，以对齐传感器特性。这使物理泛化能够应用于未见过的工具。我们利用SCFields作为密集观测输入，为扩散策略启用鲁棒的接触丰富工具操作任务。在刮擦、蜡笔绘画和剥离实验中，展示了鲁棒的类别级泛化能力，显著优于仅视觉和原始触觉基线。
英文摘要：
Generalizing tool manipulation requires both semantic planning and precise physical control. Modern generalist robot policies, such as Vision-Language-Action (VLA) models, often lack the high-fidelity physical grounding required for contact-rich tool manipulation. Conversely, existing contact-aware policies that leverage tactile or haptic sensing are typically instance-specific and fail to generalize across diverse tool geometries. Bridging this gap requires learning unified contact representations from diverse data, yet a fundamental barrier remains: diverse real-world tactile data are prohibitive at scale, while direct zero-shot sim-to-real transfer is challenging due to the complex dynamics of nonlinear deformation of soft sensors. To address this, we propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates. We enable this via a two-stage Sim-to-Real Contact Learning Pipeline: first, we pre-train on a large simulation data set to learn general contact physics; second, we fine-tune on a small set of real data, pseudo-labeled via geometric heuristics and force optimization, to align sensor characteristics. This allows physical generalization to unseen tools. We leverage SCFields as the dense observation input for a diffusion policy to enable robust execution of contact-rich tool manipulation tasks. Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization, significantly outperforming vision-only and raw-tactile baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
