发表机构
NeoteAI（尼奥特人工智能）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出$\boldsymbol{\tau}_0$-基础模型，构建大规模视触觉数据集及视触觉表征模型，结合真实与模拟套件形成评估基准，助力触觉具身操控研究。
AI 中文摘要
我们提出$\boldsymbol{\tau}_0$-基础模型，这是一种支持触觉的具身操控范式，整合了触觉传感硬件、大规模多模态数据、触觉表征学习及标准化评估。首先，我们构建了可扩展数据采集的基础设施，包括基于视觉的触觉传感器、触觉通用操控接口（UMI），以及支持机器人具身和基于UMI演示的同步视触觉数据采集系统。借助该基础设施，我们构建了NeoData，其包含超过30000小时的同步视觉与触觉演示，涵盖6种具身、450项任务，以及通过真实机器人远程操控和基于UMI的演示混合采集的数十亿对RGB与触觉帧。为推动开放研究，我们进一步发布了NeoData的开源子集OpenNeoData，时长5000小时。该数据集解决了现有操控语料库的核心局限，对可变形物体操控、精密装配、精细力控制及持续表面交互至关重要。利用大规模、异质的触觉测量数据，我们提出了NeoForce，这是一种视触觉表征模型，可学习跨不同传感器设计的可迁移触觉表征。为实现基于我们的基础设施、数据集和触觉表征构建的触觉具身模型的系统评估，我们进一步提出了综合基准，结合真实世界的NeoReal套件与模拟的NeoSim套件进行标准化评估。在两个套件上的实验表明，策略受益于物理接触状态，而非触觉信号的设备特定外观。我们发布该数据集、表征和基准，旨在支持未来触觉具身操控领域的研究工作。
英文摘要
We present $N_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile sensor, a tactile Universal Manipulation Interface (UMI), and a synchronized visuo-tactile data collection system supporting both robot embodiments and UMI-based demonstrations. Leveraging this infrastructure, we construct NeoData, which contains more than 30000 hours of synchronized visual and tactile demonstrations, spanning six embodiments, 450 tasks, and billions of paired RGB and tactile frames collected through a mixture of real-robot teleoperation and UMI-based demonstrations. To facilitate open research, we further release OpenNeoData, a 5000-hour open-source subset of NeoData. The dataset addresses a central limitation of existing manipulation corpora, critical for deformable-object manipulation, precise assembly, delicate force control, and sustained surface interaction. Capitalizing on the large-scale, heterogeneous tactile measurements, we propose NeoForce, a visuo-tactile representation model that learn transferable tactile representations across different sensor designs. To enable systematic evaluation of tactile embodied models built upon our infrastructure, datasets and tactile representations, we further propose a comprehensive benchmark, which combines the real-world NeoReal suite and the simulated NeoSim suite for standardized evaluation. Experiments across both suites show that policies benefit from the physical contact state rather than from the device-specific appearance of the tactile signal. We release the dataset, the representation, and the benchmark, aiming at supporting future work on tactile-enabled embodied manipulation.
Comments13 figures, 5 tables
