语言引导的表示学习用于鲁棒的跨传感器材料识别
Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition
浏览论文内容
中文总结 AI 辅助
针对触觉传感器跨设备泛化差的问题，提出语言引导蒸馏框架，利用语言语义对齐触觉表示，在少样本和跨传感器迁移中显著提升准确率。
中文摘要 AI 辅助
机器人需要触觉来安全可靠地操作物体，因为许多属性（如柔软度、纹理和接触稳定性）难以仅从视觉中推断。然而，基于视觉的触觉传感器由于光学、弹性体特性和光照的差异，对同一材料产生不同的观测，导致在单个或多个传感器上训练时泛化能力差。我们提出了一种语言引导的蒸馏框架，用于学习传感器鲁棒的触觉表示。语言编码了触觉的高级语义属性（例如，粗糙、柔软、滑），这些属性在传感硬件上保持不变，提供了自然的与传感器无关的监督信号。我们构建了一个包含39K样本的触觉-语言数据集，带有手工标注的材料标签，并训练一个触觉编码器，将特定于传感器的触觉图像与语言嵌入对齐到共享语义空间中。我们评估了我们的方法在少样本学习和跨传感器迁移中的表现，并在六个现有的触觉数据集上进行了基准测试。我们的方法在100样本设置中达到95%的准确率，将跨传感器迁移的平均准确率提高了13.3%，并在六个现有触觉数据集上实现了高达19%的准确率提升。这些结果表明，语言引导的蒸馏能够实现可扩展且与硬件无关的触觉表示学习。代码和数据集可在以下网址获取：此https URL
英文摘要
Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer from vision alone. However, vision-based tactile sensors yield different observations of the same material due to variations in optics, elastomer properties, and illumination, leading to poor generalization when trained on a single or multiple sensors. We propose a language-guided distillation framework for learning sensor-robust tactile representations. Language encodes high-level semantic properties of touch (e.g., rough, soft, slippery) that remain invariant across sensing hardware, providing a natural sensor-agnostic supervisory signal. We construct a 39K-sample touch-language dataset with human-annotated material labels and train a tactile encoder to align sensor-specific tactile images with language embeddings in a shared semantic space. We evaluate our approach for few-shot learning and cross-sensor transfer and benchmark it on six existing tactile datasets. Our method achieves 95% accuracy in the 100-shot setting, improves cross-sensor transfer by an average of 13.3% accuracy, and yields up to 19% accuracy gains across six existing tactile datasets. These results demonstrate that language-guided distillation enables scalable and hardware-agnostic tactile representation learning. Code and dataset are available at https://mashood3624.github.io/Language_Tactile/
发表机构
- Khalifa University Center for Autonomous Robotic Systems (KUCARS), Khalifa University（哈利法大学自主机器人系统中心（KUCARS），哈利法大学）
机构由 AI 辅助整理，请以论文原文为准。
