面向室内环境的高效多任务RGB-D场景分析
Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments
AI总结：
本文提出高效多任务RGB-D场景分析方法EMSANet，在移动平台上用单一神经网络实时完成全景分割、实例朝向估计和场景分类，并扩展NYUv2与SUNRGB-D标注以支持全面评估。
摘要：
语义场景理解对于在各种环境中行动的移动智能体至关重要。尽管语义分割已经提供了大量信息，但关于单个物体以及整体场景的细节仍然缺失，而这些细节对于许多实际应用是必需的。然而，分别解决多个任务代价高昂，并且在移动平台有限的计算和电池能力下无法实时完成。在本文中，我们提出了一种用于RGB-D场景分析的高效多任务方法（EMSANet），该方法同时执行语义分割和实例分割（全景分割）、实例朝向估计以及场景分类。我们表明，所有任务都可以在移动平台上使用单个神经网络实时完成，而不会降低性能——相反，各个任务能够相互受益。为了评估我们的多任务方法，我们扩展了常用RGB-D室内数据集NYUv2和SUNRGB-D的标注，以支持实例分割和朝向估计。据我们所知，我们是首个在NYUv2和SUNRGB-D上为室内场景分析提供如此全面多任务设置结果的工作。
英文摘要：
Semantic scene understanding is essential for mobile agents acting in various environments. Although semantic segmentation already provides a lot of information, details about individual objects as well as the general scene are missing but required for many real-world applications. However, solving multiple tasks separately is expensive and cannot be accomplished in real time given limited computing and battery capabilities on a mobile platform. In this paper, we propose an efficient multi-task approach for RGB-D scene analysis~(EMSANet) that simultaneously performs semantic and instance segmentation~(panoptic segmentation), instance orientation estimation, and scene classification. We show that all tasks can be accomplished using a single neural network in real time on a mobile platform without diminishing performance - by contrast, the individual tasks are able to benefit from each other. In order to evaluate our multi-task approach, we extend the annotations of the common RGB-D indoor datasets NYUv2 and SUNRGB-D for instance segmentation and orientation estimation. To the best of our knowledge, we are the first to provide results in such a comprehensive multi-task setting for indoor scene analysis on NYUv2 and SUNRGB-D.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
