arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.00337cs.CVcs.IR
收藏

DistillGrasp:融合特征相关性与知识蒸馏用于透明物体深度补全

DistillGrasp: Integrating Features Correlation with Knowledge Distillation for Depth Completion of Transparent Objects

College of Computer Science and Technology, Guangdong University of Technology(广东工业大学计算机科学与技术学院);Hasselt University(哈塞尔特大学);Flanders Make(佛兰德制造机构);Zhongguancun Laboratory(中关村实验室)

Yiheng Huang, Junhong Chen, Nick Michiels, Muhammad Asim, Luc Claesen, Wenyin Liu

AI总结:

本文提出DistillGrasp深度补全网络,通过教师分支的位置相关性模块和学生分支的一致性特征相关模块进行知识蒸馏,并设计兼顾距离、结构与边缘的蒸馏损失,在透明物体深度补全和机器人抓取任务中实现高精度与48 FPS实时性能。

摘要:

由于反射和折射的视觉特性,RGB-D相机无法准确捕获透明物体的深度,导致深度图不完整。为了填补缺失点,近期研究倾向于探索新的视觉特征并设计复杂网络来重建深度,然而这些方法极大地增加了计算量,并且不同视觉特征之间的相关性仍然是一个问题。为此,我们提出了一种高效的深度补全网络DistillGrasp,它将知识从教师分支蒸馏到学生分支。具体而言,在教师分支中,我们设计了一个位置相关性模块(PCB),利用RGB图像作为查询和键来搜索相应的值,引导模型在两个特征之间建立正确的对应关系并将其传递到透明区域。对于学生分支,我们提出了一致性特征相关模块(CFCM),根据一致性分别保留RGB图像和深度图的可靠区域,并采用CNN捕获成对关系以进行深度补全。为了避免学生分支仅从教师分支学习区域特征,我们设计了一种蒸馏损失,它不仅考虑距离损失,还考虑物体结构和边缘信息。在ClearGrasp数据集上进行的大量实验表明,我们的教师网络在准确性和泛化性方面优于最先进的方法,学生网络以48 FPS的更高速度取得了有竞争力的结果。此外,在真实机器人抓取系统中的显著改进说明了我们所提出系统的有效性和鲁棒性。

英文摘要:

Due to the visual properties of reflection and refraction, RGB-D cameras cannot accurately capture the depth of transparent objects, leading to incomplete depth maps. To fill in the missing points, recent studies tend to explore new visual features and design complex networks to reconstruct the depth, however, these approaches tremendously increase computation, and the correlation of different visual features remains a problem. To this end, we propose an efficient depth completion network named DistillGrasp which distillates knowledge from the teacher branch to the student branch. Specifically, in the teacher branch, we design a position correlation block (PCB) that leverages RGB images as the query and key to search for the corresponding values, guiding the model to establish correct correspondence between two features and transfer it to the transparent areas. For the student branch, we propose a consistent feature correlation module (CFCM) that retains the reliable regions of RGB images and depth maps respectively according to the consistency and adopts a CNN to capture the pairwise relationship for depth completion. To avoid the student branch only learning regional features from the teacher branch, we devise a distillation loss that not only considers the distance loss but also the object structure and edge information. Extensive experiments conducted on the ClearGrasp dataset manifest that our teacher network outperforms state-of-the-art methods in terms of accuracy and generalization, and the student network achieves competitive results with a higher speed of 48 FPS. In addition, the significant improvement in a real-world robotic grasping system illustrates the effectiveness and robustness of our proposed system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑