GoldiCLIP：平衡显式监督的Goldilocks方法用于语言-图像预训练
GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
AI总结：
GoldiCLIP通过平衡监督信号提出新框架，结合文本条件自蒸馏、编码器解码器与VQA目标及不确定性加权机制，在3000万数据下实现高效预训练，提升多任务检索性能。
摘要：
直到最近，大规模视觉-语言模型（VLMs）的成功主要依赖于十亿样本数据集，成为进展的重大障碍。最新研究开始通过提高监督质量来缩小这一差距，但每项工作只解决对比预训练中的部分不足。我们提出了GoldiCLIP，基于Goldilocks原理寻找监督信号的适当平衡。我们的多维训练框架协同结合三个关键创新：（1）文本条件自蒸馏方法对齐文本无关和文本条件特征；（2）集成解码器的编码器，具有视觉问答（VQA）目标，使编码器能够超越类似描述的查询；（3）基于不确定性的加权机制自动平衡所有异质损失。在仅3000万张图像上训练，比领先方法少300倍的数据，GoldiCLIP在数据高效方法中达到最先进水平，在MSCOCO检索中比最佳可比基线提高2.2点，在细粒度检索中提高2.0，在基于问题的检索中提高5.9，同时与十亿级模型竞争。项目页面：https://petsi.uk/goldiclip.
英文摘要：
Until recently, the success of large-scale vision-language models (VLMs) has primarily relied on billion-sample datasets, posing a significant barrier to progress. Latest works have begun to close this gap by improving supervision quality, but each addresses only a subset of the weaknesses in contrastive pretraining. We present GoldiCLIP, a framework built on a Goldilocks principle of finding the right balance of supervision signals. Our multifaceted training framework synergistically combines three key innovations: (1) a text-conditioned self-distillation method to align both text-agnostic and text-conditioned features; (2) an encoder integrated decoder with Visual Question Answering (VQA) objective that enables the encoder to generalize beyond the caption-like queries; and (3) an uncertainty-based weighting mechanism that automatically balances all heterogeneous losses. Trained on just 30 million images, 300x less data than leading methods, GoldiCLIP achieves state-of-the-art among data-efficient approaches, improving over the best comparable baseline by 2.2 points on MSCOCO retrieval, 2.0 on fine-grained retrieval, and 5.9 on question-based retrieval, while remaining competitive with billion-scale models. Project page: https://petsi.uk/goldiclip.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
