多模态对比学习适配共享潜变量的固有维度
Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables
AI总结：
该研究突破线性表示与特定分布的局限，从理论上证明温度优化下的多模态对比学习可适配数据远低于指定维度的固有维度，且经合成与真实数据集实验验证，为其低维信息表示学习能力提供了理论支撑。
摘要：
多模态对比学习作为一种自监督表示学习技术，已在CLIP等基础模型训练中取得巨大成功。本文研究了多模态对比学习所习得表示的理论性质，研究范围不局限于线性表示与特定数据分布。分析表明，在温度优化的作用下，多模态对比学习不仅能最大化模态间的互信息，还能适配数据的固有维度——该维度可能远低于用户指定的表示向量维度。在合成数据集与真实数据集上的实验均证明，对比学习具备学习低维且富含信息的表示的能力，打通了理论洞见与实际性能之间的关联。
英文摘要：
Multi-modal contrastive learning as a self-supervised representation learning technique has achieved great success in foundation model training, such as CLIP~\citep{radford2021learning}. In this paper, we study the theoretical properties of the learned representations from multi-modal contrastive learning beyond linear representations and specific data distributions. Our analysis reveals that, enabled by temperature optimization, multi-modal contrastive learning not only maximizes mutual information between modalities but also adapts to intrinsic dimensions of data, which can be much lower than user-specified dimensions for representation vectors. Experiments on both synthetic and real-world datasets demonstrate the ability of contrastive learning to learn low-dimensional and informative representations, bridging theoretical insights and practical performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
