FG-CLTP: 用于机器人操作的细粒度对比语言触觉预训练
FG-CLTP: Fine-Grained Contrastive Language Tactile Pretraining for Robotic Manipulation
AI总结：
FG-CLTP通过细粒度对比学习提升机器人触觉感知，实现高精度分类和控制，减少误差并增强跨传感器泛化能力。
摘要：
近期在将触觉传感整合到视觉-语言-动作（VLA）模型中的进展展示了对机器人感知的变革潜力。然而，现有的触觉表示主要依赖于定性描述（例如，纹理），忽略了诸如力的大小、接触几何和主轴方向等定量接触状态，这些对于精细操作至关重要。为弥合这一差距，我们提出了FG-CLTP，一种细粒度对比语言触觉预训练框架。我们首先引入了一个包含超过10万个触觉3D点云-语言对的新型数据集，该数据集明确捕捉了从传感器视角的多维接触状态。然后，我们实现了一种离散化的数值标记机制，以实现定量-语义对齐，有效地将显式的物理度量注入到多模态特征空间中。所提出的FG-CLTP模型在分类准确率上达到95.9%，与最先进的方法相比，回归误差（MAE）减少了52.6%。此外，3D点云表示的整合建立了一个传感器无关的基础，具有最小的仿真到现实差距（3.5%）。在此细粒度表示基础上，我们开发了一种由流匹配策略驱动的3D触觉-语言-动作（3D-TLA）架构，以实现多模态推理和控制。广泛的实验表明，我们的框架在接触丰富的操作任务中显著优于强大的基线方法，为触觉-语言-动作模型提供了稳健且可推广的基础。
英文摘要：
Recent advancements in integrating tactile sensing into vision-language-action (VLA) models have demonstrated transformative potential for robotic perception. However, existing tactile representations predominantly rely on qualitative descriptors (e.g., texture), neglecting quantitative contact states such as force magnitude, contact geometry, and principal axis orientation, which are indispensable for fine-grained manipulation. To bridge this gap, we propose FG-CLTP, a fine-grained contrastive language tactile pretraining framework. We first introduce a novel dataset comprising over 100k tactile 3D point cloud-language pairs that explicitly capture multidimensional contact states from the sensor's perspective. We then implement a discretized numerical tokenization mechanism to achieve quantitative-semantic alignment, effectively injecting explicit physical metrics into the multimodal feature space. The proposed FG-CLTP model yields a 95.9% classification accuracy and reduces the regression error (MAE) by 52.6% compared to state-of-the-art methods. Furthermore, the integration of 3D point cloud representations establishes a sensor-agnostic foundation with a minimal sim-to-real gap of 3.5%. Building upon this fine-grained representation, we develop a 3D tactile-language-action (3D-TLA) architecture driven by a flow matching policy to enable multimodal reasoning and control. Extensive experiments demonstrate that our framework significantly outperforms strong baselines in contact-rich manipulation tasks, providing a robust and generalizable foundation for tactile-language-action models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
