发表机构
NeoteAI Team; Fudan TEAI Team（NeoteAI团队; 复旦大学TEAI团队）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究提出$N_0$-TWAM用于丰富接触操作，通过视觉-触觉联合训练大规模预训练，采用NeoForce表示、引入触觉接触事件及非对称混合变压器架构，在多任务中展现强大能力，为细粒度操作奠定基础，代码等将公开。
AI 中文摘要
我们提出了$N_0$-TWAM，这是一种用于丰富接触操作的触觉原生世界-动作模型，可预测未来视觉和未来接触。据我们所知，它是首个大规模训练的触觉世界-动作模型，在丰富接触任务中表现出强大能力。我们通过跨六个实例和450个任务的丰富触觉演示进行视觉-触觉联合训练，大规模预训练$N_0$-TWAM。使用NeoForce统一的基于力的触觉表示形成基于物理的接触信号来调节动作生成。为改进长时和多阶段操作，引入触觉接触事件进行任务分段并在执行中推进。采用非对称混合变压器架构提高实时效率。在真实和模拟基准上的评估证明了$N_0$-TWAM在一系列丰富接触任务中的能力，展示了数据扩展对精确触觉和动作预测的好处。总之，$N_0$-TWAM赋予世界-动作模型预测视觉、触觉和动作的能力，为开放的丰富接触任务上的细粒度操作奠定坚实基础。代码库和模型检查点将公开，以促进触觉机器人操作的进一步研究和开发。
英文摘要
We present $N_0$-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks. We pre-train $N_0$-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations spanning six embodiments and 450 tasks. We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation. To improve long-horizon and multi-stage manipulation, we introduce tactile contact events for task staging and advance through them during execution. For real-time efficiency, we adopt an asymmetric Mixture-of-Transformers architecture that pairs a full-width expert for video prediction with slim experts for downstream action and tactile prediction. Evaluations on both real and simulated benchmarks justify the capabilities of $N_0$-TWAM across a range of contact-rich tasks, and demonstrate the benefit of data scaling for precise tactile and action prediction. In summary, $N_0$-TWAM endows a world-action model with predictive capabilities to foresee vision, touch and action, building a solid foundation for fine-grained manipulation on open contact-rich tasks. The codebase and model checkpoints will be made publicly available to foster further research and development in tactile-enabled robotic manipulation.
