ContactWorld: 视觉-触觉世界模型中什么对接触丰富操作至关重要
ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulation
AI总结：
通过12项接触丰富操作任务，发现空间结构化和时间连续的表征（如点云）能显著提升规划成功率，且触觉传感的有效性依赖于跨模态表征兼容性。
摘要：
接触丰富操作需要世界模型从多模态感官观测中推理复杂的接触动力学。然而，哪些表征属性从根本上支持接触丰富环境下的稳定长时域规划仍不清楚。在本文中，我们提出了ContactWorld，一个涵盖12项接触丰富操作任务（包括插入、拆卸、拧紧和探索性交互）的基准和系统性实证研究。通过大量实验，我们发现同时具有空间结构化和时间连续性的表征始终能实现最强的规划性能。特别地，点云观测将平均规划成功率从腕部视角观测的20.7%和前方视角观测的22.0%提升至32.1%。我们进一步发现，触觉传感的有效性关键取决于跨模态表征兼容性，而非仅模态规模。将点云观测与保留更丰富空间结构和交互动力学的触觉力场表征相结合，进一步将性能提升至36.1%，在所有评估任务中实现了最强的整体规划性能。此外，在长时域规划目标下，触觉传感变得越来越重要，因为复合预测误差和接触不确定性随时间累积。总之，这些发现强调了表征结构、多模态兼容性和长时域鲁棒性在面向接触丰富机器人操作的视觉-触觉世界模型中的重要性。
英文摘要：
Contact-rich manipulation poses a fundamental challenge for world models: visual and tactile observations capture different aspects of physical interaction, and their utility depends critically on how this information is represented. We introduce ContactWorld, a controlled benchmark and empirical study of vision-tactile representations across 12 contact-rich manipulation tasks. Using a fixed world-model architecture, training procedure, and planning framework, we examine representation effects through three complementary properties: spatial fidelity, motion coherence, and predictive stability. Point clouds preserve task-relevant geometry and track physical motion more reliably than image observations, helping explain their higher average planning success of 32.1%, compared with 20.7% and 22.0% for wrist- and front-view RGB, respectively. Tactile observations generally reduce long-horizon prediction-error accumulation, but these gains do not translate uniformly into task success. Structured tactile force fields provide the most consistent downstream improvements, with PointCloud+TacFF achieving the highest overall simulated success rate of 36.1%. We further validate these trends through 900 physical trials spanning six tasks, two robot platforms, and three tactile sensing systems. Together, ContactWorld identifies sensory representation as a central design factor in vision--tactile world models and provides empirical guidance for predictive planning in contact-rich manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
