我的钱包在哪里？用于自我中心视觉查询定位的物体候选集建模
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
AI总结：
针对自我中心视觉查询定位中的隐式偏差，提出通过扩展标注、动态丢弃物体候选及基于Transformer的CocoFormer模块进行改进，在VQ2D和VQ3D任务及少样本检测中取得领先成绩。
摘要：
本文处理从视觉样本中定位图像和视频数据集中物体的问题。特别地，我们关注具有挑战性的自我中心视觉查询定位问题。我们首先识别出当前查询条件模型设计和视觉查询数据集中存在严重的隐式偏差。然后，我们直接在帧和物体集层面解决这些偏差。具体而言，我们的方法通过扩展有限的标注并在训练期间动态丢弃物体候选来解决这些问题。此外，我们提出了一个新颖的基于 Transformer 的模块，允许在结合查询信息时考虑物体候选集上下文。我们将该模块命名为 Conditioned Contextual Transformer 或 CocoFormer。实验表明，所提出的改进提升了自我中心查询检测，从而在 2D 和 3D 配置下构建了更好的视觉查询定位系统。因此，我们能够将帧级检测性能从 26.28% 提升至 31.26 的 AP，相应地大幅提升了 VQ2D 和 VQ3D 的定位分数。我们改进的上下文感知查询物体检测器在第 2 届 Ego4D 挑战赛的 VQ2D 和 VQ3D 任务中分别获得第一名和第二名。此外，我们展示了所提出模型在 Few-Shot Detection (FSD) 任务中的相关性，并同样取得了 SOTA 结果。代码可在 https://github.com/facebookresearch/vq2d_cvpr 获取。
英文摘要：
This paper deals with the problem of localizing objects in image and video datasets from visual exemplars. In particular, we focus on the challenging problem of egocentric visual query localization. We first identify grave implicit biases in current query-conditioned model design and visual query datasets. Then, we directly tackle such biases at both frame and object set levels. Concretely, our method solves these issues by expanding limited annotations and dynamically dropping object proposals during training. Additionally, we propose a novel transformer-based module that allows for object-proposal set context to be considered while incorporating query information. We name our module Conditioned Contextual Transformer or CocoFormer. Our experiments show the proposed adaptations improve egocentric query detection, leading to a better visual query localization system in both 2D and 3D configurations. Thus, we are able to improve frame-level detection performance from 26.28% to 31.26 in AP, which correspondingly improves the VQ2D and VQ3D localization scores by significant margins. Our improved context-aware query object detector ranked first and second in the VQ2D and VQ3D tasks in the 2nd Ego4D challenge. In addition to this, we showcase the relevance of our proposed model in the Few-Shot Detection (FSD) task, where we also achieve SOTA results. Our code is available at https://github.com/facebookresearch/vq2d_cvpr.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
