Hearing Touch：面向富接触操作的视听预训练
Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation
AI总结：
针对机器人触觉模态缺乏互联网规模预训练数据的问题，提出用接触式麦克风作为替代触觉传感器，借助大规模视听预训练获取表示以提升富接触操作性能，是首个将多感官预训练用于机器人操作的方法。
摘要：
尽管在大量数据上进行预训练对机器人学习有益，但当前范式仅针对视觉表示开展大规模预训练，其他模态的表示则需从头训练。与视觉数据的充足性不同，目前尚不清楚哪些相关的互联网规模数据可用于触觉感知等其他模态的预训练。在机器人应用中常见的低数据量场景下，这类预训练变得愈发关键。本文中，我们通过采用接触式麦克风作为替代触觉传感器来填补这一空白。我们的核心洞见是，接触式麦克风捕获的本质是基于音频的信息，这让我们能够利用大规模视听预训练获取可提升机器人操作性能的表示。据我们所知，我们的方法是首个将大规模多感官预训练应用于机器人操作的方案。更多补充信息（包括真实机器人实验视频）请访问https://sites.google.com/view/hearing-touch。
英文摘要：
Although pre-training on a large amount of data is beneficial for robot learning, current paradigms only perform large-scale pretraining for visual representations, whereas representations for other modalities are trained from scratch. In contrast to the abundance of visual data, it is unclear what relevant internet-scale data may be used for pretraining other modalities such as tactile sensing. Such pretraining becomes increasingly crucial in the low-data regimes common in robotics applications. In this paper, we address this gap by using contact microphones as an alternative tactile sensor. Our key insight is that contact microphones capture inherently audio-based information, allowing us to leverage large-scale audio-visual pretraining to obtain representations that boost the performance of robotic manipulation. To the best of our knowledge, our method is the first approach leveraging large-scale multisensory pre-training for robotic manipulation. For supplementary information including videos of real robot experiments, please see https://sites.google.com/view/hearing-touch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
