开放集识别在视觉-语言模型时代
Open-Set Recognition in the Age of Vision-Language Models
AI总结：
本研究揭示视觉-语言模型因有限查询集存在封闭集假设，在开放集识别中表现不佳，并重新定义问题、提出新基准与评估协议，评估了不确定性及负嵌入基线方法。
摘要：
视觉-语言模型（VLMs）用于开放词汇感知，是否因其在互联网规模数据集上训练而天然成为开放集模型？我们对此问题的回答是明确的“否”——VLMs通过其有限的查询集引入了封闭集假设，使其易受开放集条件影响。我们系统评估了VLMs在开放集识别中的表现，发现它们经常将查询集中不包含的物体错误分类，导致在高召回率调优时精度极低，反之亦然。我们表明，天真地扩大查询集以包含更多类别并不能缓解此问题，反而导致任务性能和开放集性能下降。我们为VLM时代重新定义了开放集问题，建立了一个新的基准和评估协议，以促进这一重要领域的标准化评估和研究，并在多种开放词汇VLM分类器和目标检测器上评估了基于预测不确定性和专用负嵌入的有前景的基线方法。
英文摘要：
Are vision-language models (VLMs) for open-vocabulary perception inherently open-set models because they are trained on internet-scale datasets? We answer this question with a clear no - VLMs introduce closed-set assumptions via their finite query set, making them vulnerable to open-set conditions. We systematically evaluate VLMs for open-set recognition and find they frequently misclassify objects not contained in their query set, leading to alarmingly low precision when tuned for high recall and vice versa. We show that naively increasing the size of the query set to contain more and more classes does not mitigate this problem, but instead causes diminishing task performance and open-set performance. We establish a revised definition of the open-set problem for the age of VLMs, define a new benchmark and evaluation protocol to facilitate standardised evaluation and research in this important area, and evaluate promising baseline approaches based on predictive uncertainty and dedicated negative embeddings on a range of open-vocabulary VLM classifiers and object detectors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
