ViTacFormer：学习跨模态表示以实现视觉-触觉灵巧操作
ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation
AI总结：
本文提出ViTacFormer，通过跨注意力编码器融合高分辨率视觉与触觉，并结合自回归触觉预测头提升精度与鲁棒性，实现多指手的模仿学习，成功完成高精度灵巧操作任务。
摘要：
灵巧操作是机器人系统实现类人物理交互的核心能力。尽管基于视觉的方法发展迅速，但触觉传感在细粒度控制中仍至关重要，特别是在无结构或视觉遮挡环境中。我们提出了ViTacFormer，一种结合跨注意力编码器融合高分辨率视觉和触觉，并采用自回归触觉预测头预测未来接触信号的表示学习方法。基于此架构，我们设计了由易到难的课程，逐步优化视觉-触觉潜在空间，提升准确性和鲁棒性。所学的跨模态表示驱动多指手的模仿学习，实现精确和适应性的操作。在一系列具有挑战性的现实世界基准测试中，我们的方法在成功率方面比先前的最先进系统高出约50%。据我们所知，这也是首次自主完成需要高度精确控制的长时灵巧操作任务，成功完成11个连续阶段，并持续操作2.5分钟。
英文摘要：
Dexterous manipulation is a cornerstone capability for robotic systems aiming to interact with the physical world in a human-like manner. Although vision-based methods have advanced rapidly, tactile sensing remains crucial for fine-grained control, particularly in unstructured or visually occluded settings. We present ViTacFormer, a representation-learning approach that couples a cross-attention encoder to fuse high-resolution vision and touch with an autoregressive tactile prediction head that anticipates future contact signals. Building on this architecture, we devise an easy-to-challenging curriculum that steadily refines the visual-tactile latent space, boosting both accuracy and robustness. The learned cross-modal representation drives imitation learning for multi-fingered hands, enabling precise and adaptive manipulation. Across a suite of challenging real-world benchmarks, our method achieves approximately 50% higher success rates than prior state-of-the-art systems. To our knowledge, it is also the first to autonomously complete long-horizon dexterous manipulation tasks that demand highly precise control with an anthropomorphic hand, successfully executing up to 11 sequential stages and sustaining continuous operation for 2.5 minutes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
