CLIP能数星星吗？关于CLIP中数量偏差的实证研究
Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP
AI总结：
本文实证研究了CLIP中的数量偏差问题，通过设计多种实验设置与数据集，从文本、图像及跨模态视角全面评估其对数量的理解，揭示了影响下游任务可靠性的嵌入数量偏差。
摘要：
CLIP在适应图像编辑与生成、视觉问答及视频理解等各种下游任务中展现出极大的通用性。然而，基于CLIP的应用常因对用户意图的误解而出现问题，导致图像生成任务中所需物体数量与实际输出之间存在差异。在这项工作中，我们对CLIP中的数量偏差进行了实证研究。通过精心设计不同的实验设置和数据集，我们从文本、图像和跨模态视角全面评估了CLIP对数量的理解。实验结果揭示了CLIP嵌入中存在的数量偏差，影响了下游任务的可靠性。
英文摘要：
CLIP has demonstrated great versatility in adapting to various downstream tasks, such as image editing and generation, visual question answering, and video understanding. However, CLIP-based applications often suffer from misunderstandings regarding user intent, leading to discrepancies between the required number of objects and the actual outputs in image generation tasks. In this work, we empirically investigate the quantity bias in CLIP. By carefully designing different experimental settings and datasets, we comprehensively evaluate CLIP's understanding of quantity from text, image, and cross-modal perspectives. Our experimental results reveal a quantity bias in CLIP embeddings, impacting the reliability of downstream tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
