arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.04378cs.ROcs.AIcs.CVcs.LG
收藏

基于触觉感知的手-物交互动态建模

Dynamic Modeling of Hand-Object Interactions via Tactile Sensing

Shanghai Jiao Tong University(上海交通大学);Massachusetts Institute of Technology(麻省理工学院)

Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B. Tenenbaum, Wojciech Matusik, Antonio Torralba

摘要:

触觉感知对于人类执行日常任务至关重要。虽然从视觉分析物体抓取已经取得了重大进展,但我们仍然不清楚如何利用触觉感知来推理和建模手-物体相互作用的动力学。在这项工作中,我们采用了高分辨率的触觉手套进行四个不同的互动活动的多样化的一组对象。我们在跨模态学习框架上构建模型,并使用视觉处理管道生成标签来监督触觉模型,然后可以在测试期间单独使用。触觉模型旨在通过结合预测模型和对比学习模块,纯粹从触摸数据预测手和物体的3d位置。该框架可以从触觉数据中推断交互模式,对环境中的变化产生幻觉,估计预测的不确定性,并推广到看不见的对象。我们还提供关于不同系统设计的详细消融研究以及预测轨迹的可视化。这项工作采取了一个步骤,从密集的触觉感知,这为未来的应用程序在活动学习,人机交互,和模仿学习机器人的手-物体交互的动力学建模。

英文摘要:

Tactile sensing is critical for humans to perform everyday tasks. While significant progress has been made in analyzing object grasping from vision, it remains unclear how we can utilize tactile sensing to reason about and model the dynamics of hand-object interactions. In this work, we employ a high-resolution tactile glove to perform four different interactive activities on a diversified set of objects. We build our model on a cross-modal learning framework and generate the labels using a visual processing pipeline to supervise the tactile model, which can then be used on its own during the test time. The tactile model aims to predict the 3d locations of both the hand and the object purely from the touch data by combining a predictive model and a contrastive learning module. This framework can reason about the interaction patterns from the tactile data, hallucinate the changes in the environment, estimate the uncertainty of the prediction, and generalize to unseen objects. We also provide detailed ablation studies regarding different system designs as well as visualizations of the predicted trajectories. This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑