发表机构
Viterbi School of Engineering, University of Southern California; Columbia University; Starpilot（南加州大学维特比工程学院; 哥伦比亚大学; 星航）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对触觉数据稀缺问题，提出FELT框架，利用视觉编码器和查询解码器从RGB观测合成触觉图像，通过单独分支解码左右传感器面板，实验表明该方法能提升策略成功率，潜在特征训练和部署无需真实触觉传感器。
AI 中文摘要
触觉对于操纵至关重要，尤其是在视觉被遮挡或模糊时。尽管视觉与触觉结合可改善操纵，但学习强大的视觉触觉策略需要大量触觉数据。由于触觉传感器脆弱、专门且难以标准化，此类数据比视觉数据稀缺。为解决此问题，我们提出了特征提取潜在触觉（FELT），这是一个基于学习的框架，可从RGB观测中合成每根手指的压力触觉图像，减少对配备触觉的数据收集需求。FELT使用大型冻结视觉编码器和轻量级查询解码器在单次前馈过程中预测触觉信号。为尊重双指触觉传感器的物理拓扑结构，FELT通过单独分支解码左右触觉传感器面板，捕捉擦拭、插入和手中旋转等交互过程中的不对称接触模式。在推理时，FELT仅需RGB数据，使我们能够用触觉观测增强现有的仅视觉数据，无论是生成的触觉图像还是潜在触觉特征。在四个富含接触的操纵任务上的实验表明，生成的触觉图像和潜在触觉特征均比仅视觉基线提高了策略成功率，且潜在特征在策略训练或部署期间无需真实触觉传感器。补充材料可在我们的匿名网站获取：此https URL。
英文摘要
The sense of touch is central to manipulation, especially when vision is occluded or ambiguous. Although combining vision and touch improves manipulation, learning robust visuo-tactile policies requires substantial tactile data. Such data remains scarcer than visual data, because tactile sensors are fragile, specialized, and hard to standardize. To address this, we present Feature-Extracted Latent Tactile (FELT), a learning-based framework that synthesizes per-finger pressure tactile images from RGB observations, reducing the need for tactile-equipped data collection. FELT uses a large frozen visual encoder and a lightweight query decoder to predict tactile signals in a single feed-forward pass. To respect the physical topology of dual-finger tactile sensors, FELT decodes the left and right tactile sensor panels through separate branches, capturing the asymmetric contact patterns during interactions such as wiping, insertion, and in-hand rotation. At inference time, FELT only requires RGB data, allowing us to augment existing vision-only data with tactile observations, either as generated tactile images or as latent tactile features. Experiments on four contact-rich manipulation tasks demonstrate that both generated tactile images and latent tactile features improve policy success over vision-only baselines, with latent feature requiring no real tactile sensor during policy training or deployment. Supplementary material is available on our anonymous website: https://felt-tactile.github.io/.
Comments26 pages, including supplementary material
