arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.04766cs.CV
收藏

基于非对称标注的实时的联合语义分割和深度估计

Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations

School of Computer Science, the University of Adelaide, Australia(澳大利亚阿德莱德大学计算机学院);Monash University, Australia(澳大利亚莫纳什大学)

Vladimir Nekrasov, Thanuja Dharmasiri, Andrew Spek, Tom Drummond, Chunhua Shen, Ian Reid

摘要:

在机器人中部署深度学习模型作为感官信息提取器可能是一项艰巨的任务,即使使用通用GPU卡也是如此。在这里,我们解决了三个最突出的障碍,即,i)一次执行多个任务的单个模型的自适应(在这项工作中,我们认为深度估计和语义分割对于获得场景的几何和语义理解至关重要),而ii)在实时的【形】中这样做,以及iii)使用每个模态具有奇数注释的非对称数据集。为了克服前两个问题,我们适应最近提出的实时的【形】语义分割网络,作出改变,以进一步减少浮点运算的数量。为了解决第三个问题,我们接受了一个简单的解决方案的基础上硬知识蒸馏的假设下,有一个强大的“教师”网络。我们展示了我们的系统如何可以轻松地扩展到处理更多的任务,更多的数据集,一次,执行深度估计和分割室内和室外的一个单一的模型。从数量上看,我们实现了相当于(或优于)当前最先进方法的结果,其中一个前向传递成本仅为13ms,6.5 GFLOPs(640x480输入)。这种效率使我们能够直接将网络的原始预测合并到SemanticFusion框架中,以实现场景的密集3D语义重建。

英文摘要:

Deployment of deep learning models in robotics as sensory information extractors can be a daunting task to handle, even using generic GPU cards. Here, we address three of its most prominent hurdles, namely, i) the adaptation of a single model to perform multiple tasks at once (in this work, we consider depth estimation and semantic segmentation crucial for acquiring geometric and semantic understanding of the scene), while ii) doing it in real-time, and iii) using asymmetric datasets with uneven numbers of annotations per each modality. To overcome the first two issues, we adapt a recently proposed real-time semantic segmentation network, making changes to further reduce the number of floating point operations. To approach the third issue, we embrace a simple solution based on hard knowledge distillation under the assumption of having access to a powerful `teacher' network. We showcase how our system can be easily extended to handle more tasks, and more datasets, all at once, performing depth estimation and segmentation both indoors and outdoors with a single model. Quantitatively, we achieve results equivalent to (or better than) current state-of-the-art approaches with one forward pass costing just 13ms and 6.5 GFLOPs on 640x480 inputs. This efficiency allows us to directly incorporate the raw predictions of our network into the SemanticFusion framework for dense 3D semantic reconstruction of the scene.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑