VitaTouch：面向制造领域机器人质量检测的属性感知视觉-触觉-语言模型
VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing
AI总结：
VitaTouch通过融合视觉、触觉与语言信息，提升机器人在制造中对材料属性的识别能力，实现了高精度的质量检测与自然语言描述。
摘要：
在智能制造中的质量检测需要识别超出可见几何的内在材料和表面属性，但仅依赖视觉的方法仍然容易受到遮挡和反射的影响。我们提出了VitaTouch，一个属性感知的视觉-触觉-语言模型，用于材料属性推断和自然语言属性描述。VitaTouch使用模态特定的编码器和双Q-Former来提取与语言相关的视觉和触觉特征，这些特征被压缩成前缀标记以供大型语言模型使用。我们通过对比学习对每个模态与文本对齐，并明确通过对比学习将视觉和触觉联系起来。我们还构建了VitaSet，一个包含186个物体、52,000张图像和5,100对人工验证的指令-回答对的多模态数据集。VitaTouch在HCT和总体TVL基准测试中表现最佳，同时在SSVTP上具有竞争力。在VitaSet上，其达到88.89%的硬度准确性、75.13%的粗糙度准确性和54.81%的描述符召回率；材料描述任务进一步实现了峰值语义相似度为0.9009。通过基于LoRA的微调，VitaTouch在2、3和5类缺陷识别中分别达到100.0%、96.0%和92.0%的准确性，并在100次实验室机器人试验中实现了94.0%的闭环识别准确性和94.0%的端到端排序成功率。更多细节请访问项目页面：https://vitatouch.github.io/
英文摘要：
Quality inspection in smart manufacturing requires identifying intrinsic material and surface properties beyond visible geometry, yet vision-only methods remain vulnerable to occlusion and reflection. We propose VitaTouch, a property-aware vision-tactile-language model for material-property inference and natural-language attribute description. VitaTouch uses modality-specific encoders and a dual Q-Former to extract language-relevant visual and tactile features, which are compressed into prefix tokens for a large language model. We align each modality with text and explicitly couple vision and touch through contrastive learning. We also construct VitaSet, a multimodal dataset with 186 objects, 52k images, and 5.1k human-verified instruction-answer pairs. VitaTouch achieves the best performance on HCT and the overall TVL benchmark, while remaining competitive on SSVTP. On VitaSet, it reaches 88.89% hardness accuracy, 75.13% roughness accuracy, and 54.81% descriptor recall; the material-description task further achieves a peak semantic similarity of 0.9009. With LoRA-based fine-tuning, VitaTouch attains 100.0%, 96.0%, and 92.0% accuracy for 2-, 3-, and 5-category defect recognition, respectively, and delivers 94.0% closed-loop recognition accuracy and 94.0% end-to-end sorting success in 100 laboratory robotic trials. More details are available at the project page: https://vitatouch.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
