ReTouch：利用在线优化的触觉预测实现接触丰富的灵巧操作
ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction
浏览论文内容
中文总结 AI 辅助
本研究提出视觉-语言-动作模型ReTouch，通过在线优化触觉预测实现接触丰富的灵巧操作，在真实机器人实验中，其平均成功率较最强基线提升18.4至23.8个百分点。
中文摘要 AI 辅助
融合触觉信号已被证明对接触丰富的操作有效，使机器人能够感知接触状态并适应快速变化的物理交互。然而，将触觉反馈有效集成到灵巧操作中仍有待深入探索。本研究提出ReTouch，这是一种视觉-语言-动作模型（VLA），通过利用执行时反馈不断在线优化的触觉预测来支持接触丰富的灵巧操作。ReTouch在触觉表示和闭环动作生成方面有两项主要创新：其一，其触觉补丁编码器将触觉观测表示为结构化的触觉补丁特征，保留手指身份和局部接触结构，为细粒度灵巧控制提供接触线索；其二，其高频动作模块联合预测未来触觉状态和动作块，并在执行过程中利用传入的触觉反馈优化两者。这种闭环优化使触觉预测与不断变化的物理交互保持一致，实现响应式动作校正，提高对接触变化和执行错误的鲁棒性。我们还引入XHT-Dataset，包含在XHand-UR7e平台上收集的7项接触丰富任务的900个现实世界演示，并通过闭环真实机器人实验评估ReTouch。在标准和具有挑战性的条件下，ReTouch的平均成功率分别超过最强基线18.4和23.8个百分点，证明了其有效性和鲁棒性。
英文摘要
Fusing tactile signals has proven effective for contact-rich manipulation, enabling robots to perceive contact states and adapt to rapidly changing physical interactions. Yet effectively integrating tactile feedback into dexterous manipulation remains underexplored. In this work, we introduce ReTouch, a vision-language-action model (VLA) that supports contact-rich dexterous manipulation through tactile predictions continually refined online using execution-time feedback. ReTouch builds on two main innovations for tactile representation and closed-loop action generation. First, its Tactile-Patch Encoder represents tactile observations as structured tactile patch features that preserve finger identity and local contact structure, providing contact cues for fine-grained dexterous control. Second, its high-frequency action module jointly predicts future tactile states and action chunks and refines both using incoming tactile feedback during execution. This closed-loop refinement keeps tactile predictions aligned with evolving physical interactions, enabling responsive action correction and improving robustness to contact changes and execution errors. We further introduce XHT-Dataset, comprising 900 real-world demonstrations across seven contact-rich tasks collected on an XHand--UR7e platform, and evaluate ReTouch through closed-loop real-robot experiments. ReTouch surpasses the strongest baseline by 18.4 and 23.8 percentage points in average success rate under standard and challenging conditions, respectively, demonstrating its effectiveness and robustness.
