arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09617cs.ROcs.AIcs.CV
收藏

AnyTouch 2: 通用光学触觉表征学习用于动态触觉感知

AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perception

Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China(中国人民大学光明学院);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Beijing Jiaotong University(北京交通大学);Shanghai AI Laboratory(上海人工智能实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing University of Posts(北京邮电大学);State Key Laboratory of Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室)

Ruoxuan Feng, Yuxuan Zhou, Siyu Mei, Dongzhan Zhou, Pengwei Wang, Shaowei Cui, Bin Fang, Guocai Yao, Di Hu

AI总结:

AnyTouch 2 是一种通用光学触觉表征学习框架,通过大规模层次化触觉数据集提升动态触觉感知能力,实现物体层面与细粒度力感知的统一学习。

摘要:

现实世界中触觉丰富的操作需求使机器人需要感知时间触觉反馈,捕捉细微的表面变形,并推理解物体属性以及力动态。尽管光学触觉传感器能够提供如此丰富的信息,但现有的触觉数据集和模型仍然有限。这些资源主要关注物体层面的属性(例如材料),而忽略了物理交互过程中精细的触觉时间动态。我们认为,推进动态触觉感知需要一个系统化的动态感知能力层级,以指导数据收集和模型设计。为了解决缺乏富含动态信息的触觉数据的问题,我们提出了ToucHD,一个大规模的层次化触觉数据集,涵盖触觉原子动作、现实世界操作和触觉力配对数据。除了规模外,ToucHD建立了一个全面的触觉动态数据生态系统,从数据角度明确支持层次化的感知能力。在此基础上,我们提出了AnyTouch 2,一个通用的触觉表征学习框架,适用于多种光学触觉传感器,它统一了物体层面的理解与细粒度、力感知的动态感知。该框架在帧间捕捉像素级和动作特定的变形,同时显式建模物理力动态,从而从模型角度学习多级动态感知能力。我们评估了我们的模型在涵盖静态物体属性和动态物理属性以及现实世界操作任务的基准上,涵盖多个层次的动态感知能力—from基础的物体层面理解到力感知的灵巧操作。实验结果表明,模型在传感器和任务上均表现出一致且强劲的性能。

英文摘要:

Real-world contact-rich manipulation demands robots to perceive temporal tactile feedback, capture subtle surface deformations, and reason about object properties as well as force dynamics. Although optical tactile sensors are uniquely capable of providing such rich information, existing tactile datasets and models remain limited. These resources primarily focus on object-level attributes (e.g., material) while largely overlooking fine-grained tactile temporal dynamics during physical interactions. We consider that advancing dynamic tactile perception requires a systematic hierarchy of dynamic perception capabilities to guide both data collection and model design. To address the lack of tactile data with rich dynamic information, we present ToucHD, a large-scale hierarchical tactile dataset spanning tactile atomic actions, real-world manipulations, and touch-force paired data. Beyond scale, ToucHD establishes a comprehensive tactile dynamic data ecosystem that explicitly supports hierarchical perception capabilities from the data perspective. Building on it, we propose AnyTouch 2, a general tactile representation learning framework for diverse optical tactile sensors that unifies object-level understanding with fine-grained, force-aware dynamic perception. The framework captures both pixel-level and action-specific deformations across frames, while explicitly modeling physical force dynamics, thereby learning multi-level dynamic perception capabilities from the model perspective. We evaluate our model on benchmarks that covers static object properties and dynamic physical attributes, as well as real-world manipulation tasks spanning multiple tiers of dynamic perception capabilities-from basic object-level understanding to force-aware dexterous manipulation. Experimental results demonstrate consistent and strong performance across sensors and tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑