arXiv 2309.088162023-09-15cs.CV

EgoObjects：一个用于细粒度物体理解的大规模第一人称数据集

EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding

Meta AI

Chenchen Zhu, Fanyi Xiao, Andres Alvarado, Yasmine Babaei, Jiabo Hu, Hichem El-Mohri, Sean Chang Culatana, Roshan Sumbaly, Zhicheng Yan

备注：: ICCV 2023 final version and supplement. See more details in project page: https://github.com/facebookresearch/EgoObjects

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出大规模第一人称细粒度物体理解数据集 EgoObjects，包含实例级标注，并构建了物体检测与持续学习等基准任务。

摘要：

第一人称视觉数据中的物体理解可以说是第一人称视觉领域的一个基础研究课题。然而，现有的物体数据集要么不是第一人称的，要么在物体类别、视觉内容和标注粒度方面存在局限性。在本工作中，我们介绍了 EgoObjects，一个用于细粒度物体理解的大规模第一人称数据集。其 Pilot 版本包含由来自 50 多个国家的 250 名参与者使用 4 种可穿戴设备采集的超过 9K 个视频，以及来自 368 个物体类别的超过 650K 个物体标注。与先前仅包含物体类别标签的数据集不同，EgoObjects 还为每个物体标注了实例级标识符，并包含超过 14K 个唯一物体实例。EgoObjects 旨在捕捉同一物体在不同背景复杂度、周围物体、距离、光照和相机运动下的表现。在数据收集的同时，我们通过开发一个多阶段联邦标注流程来进行数据标注，以适应数据集不断增长的性质。为了启动 EgoObjects 上的研究，我们围绕第一人称物体理解提出了一套 4 个基准任务，包括一个新的实例级物体检测和经典的类别级物体检测。此外，我们还引入了 2 个新的持续学习物体检测任务。数据集和 API 可在 https://github.com/facebookresearch/EgoObjects 获取。

英文摘要：

Object understanding in egocentric visual data is arguably a fundamental research topic in egocentric vision. However, existing object datasets are either non-egocentric or have limitations in object categories, visual content, and annotation granularities. In this work, we introduce EgoObjects, a large-scale egocentric dataset for fine-grained object understanding. Its Pilot version contains over 9K videos collected by 250 participants from 50+ countries using 4 wearable devices, and over 650K object annotations from 368 object categories. Unlike prior datasets containing only object category labels, EgoObjects also annotates each object with an instance-level identifier, and includes over 14K unique object instances. EgoObjects was designed to capture the same object under diverse background complexities, surrounding objects, distance, lighting and camera motion. In parallel to the data collection, we conducted data annotation by developing a multi-stage federated annotation process to accommodate the growing nature of the dataset. To bootstrap the research on EgoObjects, we present a suite of 4 benchmark tasks around the egocentric object understanding, including a novel instance level- and the classical category level object detection. Moreover, we also introduce 2 novel continual learning object detection tasks. The dataset and API are available at https://github.com/facebookresearch/EgoObjects.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。