arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.05949cs.LGcs.AIcs.CVstat.ML
收藏

高斯嵌入:JEPA如何秘密学习你的数据密度

Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density

Meta-FAIR;Brown University(布朗大学);NYU(纽约大学)

Randall Balestriero, Nicolas Ballas, Mike Rabbat, Yann LeCun

AI总结:

本文发现JEPA的防坍塌项能隐式估计数据密度,提出JEPA-SCORE方法,利用模型雅可比矩阵闭式高效计算样本概率,适用于数据整理、异常检测和密度估计,并在多数据集和方法上验证。

摘要:

联合嵌入预测架构(JEPAs)学习到的表示能够开箱即用地解决众多下游任务。JEPAs结合了两个目标:(i)潜空间预测项,即轻微扰动样本的表示必须能从原始样本的表示中预测出来;(ii)防坍塌项,即并非所有样本都应具有相同的表示。虽然(ii)通常被视为表示坍塌的明显补救措施,但我们揭示出JEPAs的防坍塌项作用远不止于此——它可证明地估计了数据密度。简言之,任何成功训练的JEPA都可以用于获取样本概率,例如用于数据整理、异常检测,或仅用于密度估计。我们的理论发现与所使用的数据集和架构无关——在任何情况下,都可以使用模型在样本$x$处的雅可比矩阵,以闭式形式高效地计算样本$x$的学习概率。我们的发现已在多个数据集(合成、受控和ImageNet)以及属于JEPA家族的不同自监督学习方法(I-JEPA和DINOv2)和多模态模型(如MetaCLIP)上得到实证验证。我们将提取JEPA学习密度的这一方法称为{\f JEPA-SCORE}。

英文摘要:

Joint Embedding Predictive Architectures (JEPAs) learn representations able to solve numerous downstream tasks out-of-the-box. JEPAs combine two objectives: (i) a latent-space prediction term, i.e., the representation of a slightly perturbed sample must be predictable from the original sample's representation, and (ii) an anti-collapse term, i.e., not all samples should have the same representation. While (ii) is often considered as an obvious remedy to representation collapse, we uncover that JEPAs' anti-collapse term does much more--it provably estimates the data density. In short, any successfully trained JEPA can be used to get sample probabilities, e.g., for data curation, outlier detection, or simply for density estimation. Our theoretical finding is agnostic of the dataset and architecture used--in any case one can compute the learned probabilities of sample $x$ efficiently and in closed-form using the model's Jacobian matrix at $x$. Our findings are empirically validated across datasets (synthetic, controlled, and Imagenet) and across different Self Supervised Learning methods falling under the JEPA family (I-JEPA and DINOv2) and on multimodal models, such as MetaCLIP. We denote the method extracting the JEPA learned density as {\bf JEPA-SCORE}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑