利用合成描述进行双曲学习以实现开放世界检测
Hyperbolic Learning with Synthetic Captions for Open-World Detection
AI总结：
针对开放世界检测依赖昂贵人工描述的问题，本文用VLM生成密集合成描述，并通过双曲视觉-语言层级学习抑制幻觉噪声，提出HyperLearner，在多个基准上超过同骨干的GLIP、GLIPv2和Grounding DINO。
摘要：
开放世界检测带来了重大挑战，因为它要求使用对象类别标签或自由形式文本来检测任意对象。现有相关工作通常使用大规模人工标注的描述数据集进行训练，而这些数据集的收集成本极高。为此，我们提出从视觉-语言模型（VLM）迁移知识，以自动丰富开放词汇描述。具体而言，我们利用预训练VLM自举生成密集的合成描述，从而对图像中的不同区域提供丰富描述，并结合这些描述训练一种可泛化到新概念的新型检测器。为缓解合成描述中由幻觉造成的噪声，我们还提出一种新颖的双曲视觉-语言学习方法，在视觉嵌入与描述嵌入之间施加层级结构。我们将该检测器称为“HyperLearner”。我们在多种开放世界检测基准（COCO、LVIS、Object Detection in the Wild、RefCOCO）上进行了大量实验，结果表明，在使用相同骨干网络时，我们的模型始终优于GLIP、GLIPv2和Grounding DINO等现有最先进方法。
英文摘要：
Open-world detection poses significant challenges, as it requires the detection of any object using either object class labels or free-form texts. Existing related works often use large-scale manual annotated caption datasets for training, which are extremely expensive to collect. Instead, we propose to transfer knowledge from vision-language models (VLMs) to enrich the open-vocabulary descriptions automatically. Specifically, we bootstrap dense synthetic captions using pre-trained VLMs to provide rich descriptions on different regions in images, and incorporate these captions to train a novel detector that generalizes to novel concepts. To mitigate the noise caused by hallucination in synthetic captions, we also propose a novel hyperbolic vision-language learning approach to impose a hierarchy between visual and caption embeddings. We call our detector ``HyperLearner''. We conduct extensive experiments on a wide variety of open-world detection benchmarks (COCO, LVIS, Object Detection in the Wild, RefCOCO) and our results show that our model consistently outperforms existing state-of-the-art methods, such as GLIP, GLIPv2 and Grounding DINO, when using the same backbone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
