利用现成感知模型融合标注室内场景
Labeling Indoor Scenes with Fusion of Out-of-the-Box Perception Models
AI总结：
提出融合SAM、Detic和MaskFormer等现成感知模型及多视角标注融合的方法，低成本生成室内场景伪标签，提升下游任务性能。
摘要：
图像标注阶段是训练和评估目标检测与语义分割模型所需的一个关键且通常最耗时的环节。将现有模型部署到新环境中通常需要检测训练数据中未出现的新语义类别。此外，室内场景包含显著的视角变化，需要训练好的感知模型妥善处理。我们提出利用最近在自底向上分割（SAM）、目标检测（Detic）和语义分割（MaskFormer）等最先进模型上的进展，这些模型均在大规模数据集上训练。我们旨在开发一种高性价比的标注方法，以获取室内环境中语义分割和目标实例检测的伪标签，最终目标是促进各种下游任务轻量级模型的训练。我们还提出一个多视角标注融合阶段，该阶段考虑了场景多视角可用的情况，可用于识别和纠正单视角的不一致性。我们在Active Vision数据集和ADE20K数据集上证明了所提方法的有效性。我们通过与人工标注进行比较来评估标注过程的质量。此外，我们还在目标导航和部件发现等下游任务中证明了所获标签的有效性。在目标目标导航的背景下，与利用大型单体视觉-语言预训练模型的零样本基线相比，我们展示了使用该融合方法带来的性能提升。
英文摘要：
The image annotation stage is a critical and often the most time-consuming part required for training and evaluating object detection and semantic segmentation models. Deployment of the existing models in novel environments often requires detecting novel semantic classes not present in the training data. Furthermore, indoor scenes contain significant viewpoint variations, which need to be handled properly by trained perception models. We propose to leverage the recent advancements in state-of-the-art models for bottom-up segmentation (SAM), object detection (Detic), and semantic segmentation (MaskFormer), all trained on large-scale datasets. We aim to develop a cost-effective labeling approach to obtain pseudo-labels for semantic segmentation and object instance detection in indoor environments, with the ultimate goal of facilitating the training of lightweight models for various downstream tasks. We also propose a multi-view labeling fusion stage, which considers the setting where multiple views of the scenes are available and can be used to identify and rectify single-view inconsistencies. We demonstrate the effectiveness of the proposed approach on the Active Vision dataset and the ADE20K dataset. We evaluate the quality of our labeling process by comparing it with human annotations. Also, we demonstrate the effectiveness of the obtained labels in downstream tasks such as object goal navigation and part discovery. In the context of object goal navigation, we depict enhanced performance using this fusion approach compared to a zero-shot baseline that utilizes large monolithic vision-language pre-trained models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
