arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.00809cs.CVcs.HCcs.IR
收藏

重新审视基于预训练视觉Transformer的人机协同目标检索

Revisiting Human-in-the-Loop Object Retrieval with Pre-Trained Vision Transformers

Institut National de l'Audiovisuel(法国国家视听研究所);Institut National de Recherche et Informatique et Robotique(法国国家信息与机器人研究中心)

Kawtar Zaher, Olivier Buisson, Alexis Joly

AI总结:

本文基于预训练视觉Transformer重新审视人机协同目标检索任务,探讨了在多目标数据集中如何通过主动学习选择样本以提升检索性能,比较了不同表征策略的优劣。

摘要:

在现有方法基础上,我们重新审视人机协同目标检索任务,该任务包括通过用户提供的查询迭代检索包含目标类别的图像。从大规模未标记图像集合开始,目标是仅依赖初始查询和用户的相关反馈快速识别目标类别的多样化实例。检索过程被公式化为二元分类任务,系统通过迭代用户交互持续学习区分相关和不相关图像。此交互由主动学习循环引导:在每次迭代中,系统选择信息量大的样本供用户标注,从而提升检索性能。该任务在多目标数据集中尤其具有挑战性,因为目标可能仅占据图像中复杂、拥挤场景中的小区域。与以对象为中心的设置不同,多目标图像需要更适应的、局部化的描述符。在本文中,我们通过利用预训练的ViT表示重新审视人机协同目标检索任务,并解决关键设计问题,包括在图像中考虑哪些目标实例、注释应采取何种形式、主动选择应如何应用,以及哪种表征策略最能捕捉目标的特征。我们比较了多种表征策略在多目标数据集上的表现,突显了在捕捉全局上下文和聚焦细粒度局部对象细节之间的权衡。我们的结果为基于主动学习的对象类别检索的有效交互检索管道设计提供了实用见解。

英文摘要:

Building on existing approaches, we revisit Human-in-the-Loop Object Retrieval, a task that consists of iteratively retrieving images containing objects of a class-of-interest, specified by a user-provided query. Starting from a large unlabeled image collection, the aim is to rapidly identify diverse instances of an object category relying solely on the initial query and the user's Relevance Feedback, with no prior labels. The retrieval process is formulated as a binary classification task, where the system continuously learns to distinguish between relevant and non-relevant images to the query, through iterative user interaction. This interaction is guided by an Active Learning loop: at each iteration, the system selects informative samples for user annotation, thereby refining the retrieval performance. This task is particularly challenging in multi-object datasets, where the object of interest may occupy only a small region of the image within a complex, cluttered scene. Unlike object-centered settings where global descriptors often suffice, multi-object images require more adapted, localized descriptors. In this work, we formulate and revisit the Human-in-the-Loop Object Retrieval task by leveraging pre-trained ViT representations, and addressing key design questions, including which object instances to consider in an image, what form the annotations should take, how Active Selection should be applied, and which representation strategies best capture the object's features. We compare several representation strategies across multi-object datasets highlighting trade-offs between capturing the global context and focusing on fine-grained local object details. Our results offer practical insights for the design of effective interactive retrieval pipelines based on Active Learning for object class retrieval.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑