发表机构
Fudan University; Shanghai Key Laboratory of Multimodal Embodied AI; Hefei University of Technology; National University of Singapore; Neote AI; China Unicom（复旦大学; 上海多模态具身智能重点实验室; 合肥工业大学; 新加坡国立大学; Neote AI; 中国联通）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
OpenViTac构建了统一的仿真-真实视觉-触觉操作基准，并提出OpenVTLA触觉增强框架，以评估和提升机器人策略性能。
AI 中文摘要
触觉反馈为具身智能体提供了超越视觉观测的物理信息，使其能够与真实世界进行更可靠的交互。然而，尽管视觉-触觉-语言-动作（VTLA）策略取得了快速发展，但在仿真与真实世界中评估触觉赋能机器人操作的统一基准仍然缺乏。为填补这一空白，我们提出了OpenViTac，一个用于在仿真与真实世界中评估机器人策略的视觉-触觉操作基准。OpenViTac将接触密集的操作组织为四个与触觉相关的能力维度，并提供配对的仿真-真实设置，以对VLA、WAM和VTLA策略进行一致评估。基于该基准，我们研究了不同触觉表示和集成策略如何影响预训练VLA模型的性能。相应地，我们提出了OpenVTLA，一个结合了最佳表示和集成策略的触觉增强框架。此外，我们利用配对的基准设置来研究仿真-真实协同训练，并分析影响跨域策略学习的因素。综上，OpenViTac为评估和推进视觉-触觉机器人操作提供了一个统一平台。
英文摘要
Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world. However, despite the rapid progress of vision-tactile-language-action (VTLA) policies, there remains a lack of unified benchmarks for evaluating tactile-enabled robot manipulation across simulation and the real world. To address this gap, we introduce OpenViTac, a visuo-tactile manipulation benchmark for evaluating robot policies across simulation and the real world. OpenViTac organizes contact-rich manipulation into four tactile-relevant capability dimensions and provides paired simulation-real-world settings for consistent evaluation of VLA, WAM, and VTLA policies. Building upon this benchmark, we investigate how different tactile representations and integration strategies affect the performance of pretrained VLA models. Correspondingly, we introduce OpenVTLA, a tactile augmentation framework that combines the best-performing representation and integration strategy. Furthermore, we leverage the paired benchmark setting to study sim-real co-training and analyze factors affecting cross-domain policy learning. Together, OpenViTac provides a unified platform for evaluating and advancing visuo-tactile robot manipulation.
CommentsProject website: https://fvl-repo.github.io/OpenViTac/
