添加内部音频感知至内部视觉可使软机器人指尖实现类人手持织物识别
Adding internal audio sensing to internal vision enables human-like in-hand fabric recognition with soft robotic fingertips
AI总结：
本研究通过结合内部视觉和音频感知，使软机器人指尖实现类人织物识别，利用变压器方法在 20 种常见织物上达到 97% 分类准确率。
摘要：
区分光滑丝绸与粗糙棉布的触感是人类日常的 trivial 任务。当探索此类织物时，指尖皮肤能感知时空力模式和由纹理引起的振动，并将二者整合成对探索材料的触觉表征。在机器人中复现这种丰富动态的知觉能力具有挑战性，因为触觉传感器通常无法同时实现高空间分辨率和高时间采样率。在本工作中，我们提出了一种能够感知这两种触觉信息的系统，并研究每种类型如何影响机器人对织物的触觉感知。我们的机器人手的中指和拇指各配备一个柔软的触觉传感器：一个为开源的 Minsight 传感器，利用内部摄像头以 50 Hz 的频率测量指尖变形和力；另一个是我们新的 Minsound 传感器，通过内部 MEMS 麦克风捕获振动，频带从 50 Hz 到 15 kHz。受人类评估织物动作的启发，我们的机器人主动将折叠的织物样本夹在两个敏感手指之间进行摩擦。我们的实验测试了每种传感模式对整体分类性能的影响，显示了音频传感器的高实用性。基于变压器的方法在 20 种常见织物的数据集上实现了 97% 的织物分类准确率。在远离 Minsound 的外部麦克风下，我们的方法在嘈杂的环境噪声条件下更具鲁棒性。为了证明这种音频-视觉触觉感知方法能超越训练数据，我们学习了织物伸缩性、厚度和粗糙度的一般表示。
英文摘要：
Distinguishing the feel of smooth silk from coarse cotton is a trivial everyday task for humans. When exploring such fabrics, fingertip skin senses both spatio-temporal force patterns and texture-induced vibrations that are integrated to form a haptic representation of the explored material. It is challenging to reproduce this rich, dynamic perceptual capability in robots because tactile sensors typically cannot achieve both high spatial resolution and high temporal sampling rate. In this work, we present a system that can sense both types of haptic information, and we investigate how each type influences robotic tactile perception of fabrics. Our robotic hand's middle finger and thumb each feature a soft tactile sensor: one is the open-source Minsight sensor that uses an internal camera to measure fingertip deformation and force at 50 Hz, and the other is our new sensor Minsound that captures vibrations through an internal MEMS microphone with a bandwidth from 50 Hz to 15 kHz. Inspired by the movements humans make to evaluate fabrics, our robot actively encloses and rubs folded fabric samples between its two sensitive fingers. Our results test the influence of each sensing modality on overall classification performance, showing high utility for the audio-based sensor. Our transformer-based method achieves a maximum fabric classification accuracy of 97 % on a dataset of 20 common fabrics. Incorporating an external microphone away from Minsound increases our method's robustness in loud ambient noise conditions. To show that this audio-visual tactile sensing approach generalizes beyond the training data, we learn general representations of fabric stretchiness, thickness, and roughness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
