文本反演：无遗忘地高效适配开放词汇目标检测器
Textual Inversion for Efficient Adaptation of Open-Vocabulary Object Detectors Without Forgetting
AI总结：
提出基于文本反演的开放词汇目标检测适配方法，冻结VLM权重仅学习标记嵌入，用少量样本实现新类检测且不遗忘原有能力，计算开销低。
摘要：
大型预训练视觉语言模型（VLMs）的最新进展已在多个目标检测基准上达到最先进性能，并展现出强大的零样本能力，但为了在特定目标上获得最优性能，某种形式的微调仍然必不可少。虽然初始的VLM权重允许出色的少样本迁移学习，但这通常伴随着原始自然语言查询和零样本能力的丧失。受文本反演（TI）在个性化文本到图像扩散模型中成功应用的启发，我们为开放词汇目标检测提出了类似的公式。TI通过学习新标记或改进现有标记来扩展VLM词汇表，从而仅用少至三个示例就能准确检测新颖或细粒度对象。学习到的标记与原始VLM权重完全兼容，同时保持这些权重冻结，保留了原始模型在基准上的性能，并利用其现有能力，如零样本域迁移（例如，仅用真实照片训练后即可检测对象的草图）。存储和梯度计算仅限于标记嵌入维度，所需计算量远少于全模型微调。我们在广泛的定量和定性实验中评估了该方法是否匹配或优于那些遭受遗忘的基线方法。
英文摘要：
Recent progress in large pre-trained vision language models (VLMs) has reached state-of-the-art performance on several object detection benchmarks and boasts strong zero-shot capabilities, but for optimal performance on specific targets some form of finetuning is still necessary. While the initial VLM weights allow for great few-shot transfer learning, this usually involves the loss of the original natural language querying and zero-shot capabilities. Inspired by the success of Textual Inversion (TI) in personalizing text-to-image diffusion models, we propose a similar formulation for open-vocabulary object detection. TI allows extending the VLM vocabulary by learning new or improving existing tokens to accurately detect novel or fine-grained objects from as little as three examples. The learned tokens are completely compatible with the original VLM weights while keeping them frozen, retaining the original model's benchmark performance, and leveraging its existing capabilities such as zero-shot domain transfer (e.g., detecting a sketch of an object after training only on real photos). The storage and gradient calculations are limited to the token embedding dimension, requiring significantly less compute than full-model fine-tuning. We evaluated whether the method matches or outperforms the baseline methods that suffer from forgetting in a wide variety of quantitative and qualitative experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
