LeVLJEPA：无负样本的端到端视觉语言预训练
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
AI总结：
提出首个完全非对比的端到端视觉语言预训练方法LeVLJEPA，通过跨模态预测与分布正则化学习，无需负样本、温度参数等，生成更强的密集语义特征，在下游任务中优于对比基线。
摘要：
视觉语言预训练仍以对比目标为主，而纯视觉自监督学习已广泛采用非对比方法。同时，视觉语言编码器的角色已转变：它们越来越多地被部署为零样本分类器，而是作为视觉语言模型和密集预测系统的冻结视觉骨干，这些系统消耗完整的补丁令牌网格，而非单个池化嵌入。我们提出LeVLJEPA，这是首个完全非对比的端到端视觉语言预训练方法。LeVLJEPA通过跨模态预测学习，采用停止梯度目标和每模态分布正则化，无需负样本、温度参数、动量编码器或师生调度，并能在大规模下稳定训练。我们发现，由此产生的编码器为下游使用提供了显著更强的密集语义特征：作为冻结的视觉语言模型骨干，LeVLJEPA在两种不同语言模型下的GQA、VQAv2和POPE上是最强的评估编码器，在语义分割上优于对比基线，而在全局读出（如线性探测）上保持相当。这些结果确立了非对比预训练作为产生密集语义视觉特征的有效手段。
英文摘要：
Vision-language pretraining remains dominated by contrastive objectives, whereas vision-only self-supervised learning has largely adopted non-contrastive methods. At the same time, the role of vision-language encoders has shifted: they are increasingly deployed not as zero-shot classifiers but as the frozen visual backbone of vision-language models and dense prediction systems, which consume the full grid of patch tokens rather than a single pooled embedding. We introduce LeVLJEPA, the first fully non-contrastive end-to-end vision-language pretraining method. LeVLJEPA learns through cross-modal prediction with stop-gradient targets and per-modality distributional regularization, without negatives, temperature, momentum encoder, or teacher-student schedule, and trains stably at large scale. We find that the resulting encoder provides markedly stronger dense semantic features for downstream use: as a frozen vision-language-model backbone, LeVLJEPA is the strongest of the evaluated encoders across GQA, VQAv2, and POPE under two distinct language models, and outperforms contrastive baselines on semantic segmentation, while remaining on par on global readouts such as linear probing. These results establish non-contrastive pretraining as an effective means of producing dense semantic vision features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
