在无先验知识情况下分割已知物体与未见未知物
Segmenting Known Objects and Unseen Unknowns without Prior Knowledge
AI总结：
本文提出整体分割新设定，并用 U3HS 基于不确定性和实例感知嵌入聚类，在无未知类别先验或训练样本时同时分割已知物与未知实例。
摘要：
全景分割方法会为输入中的每个像素分配一个已知类别。即使是最先进的方法，这也不可避免地强制模型作出决策，从而对训练类别之外的物体系统性地产生错误预测。然而，在安全关键场景中，对分布外样本和边角案例的鲁棒性对于避免危险后果至关重要。由于真实世界数据集无法包含足够的数据点来充分采样底层分布的长尾部分，模型还必须能够处理未见且未知的场景。以往方法通过重新识别已经见过但未标注的物体来解决这一问题。在本文中，我们提出了扩展分割的必要步骤：引入一个称为整体分割（holistic segmentation）的新设定。整体分割旨在在对已知类别执行全景分割的同时，在没有任何关于未见、未知类别物体的先验知识的情况下，将其识别并分离为独立实例。我们通过 U3HS 来解决这一新问题，该方法将未知物识别为高度不确定的区域，并将其对应的实例感知嵌入聚类为单个物体。通过这种方式，在含未知物体的全景分割任务中，我们的 U3HS 首次在不使用未知类别进行训练的情况下完成学习，从而减少假设，并使设定保持与现实场景一样不受约束。在 MS COCO、Cityscapes 和 Lost&Found 的公开数据上进行的大量实验，证明了 U3HS 在这一全新、具有挑战性且无假设的整体分割设定中的有效性。项目页面：https://holisticseg.github.io。
英文摘要：
Panoptic segmentation methods assign a known class to each pixel given in input. Even for state-of-the-art approaches, this inevitably enforces decisions that systematically lead to wrong predictions for objects outside the training categories. However, robustness against out-of-distribution samples and corner cases is crucial in safety-critical settings to avoid dangerous consequences. Since real-world datasets cannot contain enough data points to adequately sample the long tail of the underlying distribution, models must be able to deal with unseen and unknown scenarios as well. Previous methods targeted this by re-identifying already-seen unlabeled objects. In this work, we propose the necessary step to extend segmentation with a new setting which we term holistic segmentation. Holistic segmentation aims to identify and separate objects of unseen, unknown categories into instances without any prior knowledge about them while performing panoptic segmentation of known classes. We tackle this new problem with U3HS, which finds unknowns as highly uncertain regions and clusters their corresponding instance-aware embeddings into individual objects. By doing so, for the first time in panoptic segmentation with unknown objects, our U3HS is trained without unknown categories, reducing assumptions and leaving the settings as unconstrained as in real-life scenarios. Extensive experiments on public data from MS COCO, Cityscapes, and Lost&Found demonstrate the effectiveness of U3HS for this new, challenging, and assumptions-free setting called holistic segmentation. Project page: https://holisticseg.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
