arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.09427cs.ROcs.CV
收藏

TransTouch:通过稀疏触觉学习透明物体深度感知

TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touches

Tsinghua University(清华大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Qi Zhi Institute(上海期智研究院)

Liuyu Bian, Pengyang Shi, Weihang Chen, Jing Xu, Li Yi, Rui Chen

AI总结:

本文提出TransTouch方法,利用触觉反馈自动采集稀疏深度标签微调立体网络,并通过效用函数优化探测位置和置信度正则化,显著提升透明物体深度感知精度。

摘要:

透明物体在日常生活中十分常见,然而透明物体的深度感知仍然是一个具有挑战性的问题。虽然基于学习的方法可以利用形状先验来提升感知质量,但现实世界中劳动密集的数据采集以及从仿真到现实的领域差距限制了这些方法的可扩展性。在本文中,我们提出了一种利用带有触觉反馈的探测系统自动采集稀疏深度标签来微调立体网络的方法。我们提出了一种新颖的效用函数来评估触碰的收益。通过近似并优化该效用函数,我们可以在给定触碰预算的情况下优化探测位置,从而更好地提升网络在真实物体上的性能。我们进一步将触觉深度监督与基于置信度的正则化相结合,以防止微调过程中的过拟合。为了评估我们方法的有效性,我们构建了一个包含漫反射物体和透明物体的真实世界数据集。在该数据集上的实验结果表明,我们的方法能够显著提升真实世界深度感知精度,尤其是对于透明物体。

英文摘要:

Transparent objects are common in daily life. However, depth sensing for transparent objects remains a challenging problem. While learning-based methods can leverage shape priors to improve the sensing quality, the labor-intensive data collection in the real world and the sim-to-real domain gap restrict these methods' scalability. In this paper, we propose a method to finetune a stereo network with sparse depth labels automatically collected using a probing system with tactile feedback. We present a novel utility function to evaluate the benefit of touches. By approximating and optimizing the utility function, we can optimize the probing locations given a fixed touching budget to better improve the network's performance on real objects. We further combine tactile depth supervision with a confidence-based regularization to prevent over-fitting during finetuning. To evaluate the effectiveness of our method, we construct a real-world dataset including both diffuse and transparent objects. Experimental results on this dataset show that our method can significantly improve real-world depth sensing accuracy, especially for transparent objects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑