基于触觉交互感知的物体中心表示学习用于机器人操作
Learning with Object-centric Representations of Tactile Interactive Perception for Robot Manipulation
浏览论文内容
中文总结 AI 辅助
提出一种物体中心情境感知操作框架，通过触觉探索学习任务无关表示，利用标记学习器和对比对齐捕获物体属性，在视觉模糊任务中将成功率从41%提升至92%（已见物体）和从19%提升至67%（未见物体）。
中文摘要 AI 辅助
难以直接从视觉中推断的隐式物体属性，如材质、容器内容或柔软度，可以通过触觉感知和探索性交互来揭示。然而，由于触觉信号是瞬态且稀疏的，提取信息丰富的触觉事件并将其有效整合到机器人操作中仍然是一个挑战。在这项工作中，我们提出了一种物体中心的情境感知操作框架，通过触觉探索学习任务无关的物体表示。一个标记学习器自主地从长时程探索中选择具有代表性的触觉片段，而与描述性文本嵌入的对比对齐使得潜在空间能够捕获多种物理物体属性。这些学习到的表示随后被用作语义上下文，以指导物体中心操作策略，并根据物体属性调整策略。实验表明，学习到的表示在已见和未见物体上的属性估计准确率分别达到93%和84%。在涉及视觉模糊物体的三项任务（即多物体重排、倒水和开箱）上的评估中，所提出的框架提高了目标选择和基于属性的操作适应性，将所有评估试验汇总的任务成功率从基线策略（无物体上下文条件）的41%提高到已见物体上的92%，从19%提高到未见物体上的67%。视频和其他结果可在以下网址获取：https URL。
英文摘要
Implicit object properties that are difficult to directly infer from vision, such as material, container contents, or softness, can be revealed through tactile sensing and exploratory interactions. However, because tactile signals are transient and sparse, extracting informative tactile events and effectively incorporating them into robotic manipulation remains a challenge. In this work, we present an object-centric context-aware manipulation framework that learns task-agnostic object representations through tactile exploration. A token learner autonomously selects representative tactile segments from long-horizon exploration, while contrastive alignment with descriptive text embeddings enables a latent space that captures multiple physical object properties. These learned representations are then used as semantic context to guide object-centric manipulation policies and adapt strategies based on object properties. Experiments show that the learned representations achieve 93% and 84% property estimation accuracy on seen and unseen objects. Evaluated on three tasks involving visually ambiguous objects, i.e. multi-object rearrangement, pouring, and box opening, the proposed framework improves both target selection and property-dependent manipulation adaptation, raising task success, aggregated over all evaluation trials, from 41% to 92% on seen objects and from 19% to 67% on unseen objects over baseline policies without object-context conditioning. Videos and additional results are available at https://xinyiyxyx.github.io/tactile-object-centric/.
