对比互信息机：用于统一生成式与判别式表示学习的对比互信息框架
Contrastive MIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning
AI总结：
提出cMIM框架，通过对比目标增强MIM，统一生成式与判别式表示学习，无需正样本增强且对批次大小鲁棒，实验表明其在分类回归任务上优于MIM和InfoNCE。
摘要：
学习能够泛化到未知下游任务的表示是表示学习中的一个核心挑战。现有方法如对比学习、自监督掩码和去噪自编码器以不同的权衡应对这一挑战。在本文中，我们引入了对比互信息机（cMIM），这是一个概率框架，通过一种新颖的对比目标增强了互信息机（MIM）。虽然MIM最大化输入与潜变量之间的互信息并鼓励潜码的聚类，但其表示在判别任务上的表现不如最先进的替代方法。cMIM通过强制全局判别结构同时保留MIM的生成优势来解决这一局限性。我们提出两个主要贡献：（1）我们提出了cMIM，它是MIM的对比扩展，消除了对正样本数据增强的需求，并且与基于InfoNCE的方法不同，它对批次大小具有鲁棒性；（2）我们引入了信息丰富嵌入，这是一种从编码器-解码器模型中提取增强表示的通用技术，无需额外训练即可显著提升判别性能，并且其应用范围广泛，不仅限于MIM。实验结果表明，cMIM在分类和回归任务上始终优于MIM和InfoNCE，同时保持了相当的重建质量。这些发现表明，cMIM提供了一个统一的框架，用于学习同时适用于判别和生成应用的表示。
英文摘要：
Learning representations that generalize well to unknown downstream tasks is a central challenge in representation learning. Existing approaches such as contrastive learning, self-supervised masking, and denoising auto-encoders address this challenge with varying trade-offs. In this paper, we introduce the {contrastive Mutual Information Machine} (cMIM), a probabilistic framework that augments the Mutual Information Machine (MIM) with a novel contrastive objective. While MIM maximizes mutual information between inputs and latent variables and encourages clustering of latent codes, its representations underperform on discriminative tasks compared to state-of-the-art alternatives. cMIM addresses this limitation by enforcing global discriminative structure while retaining MIM's generative strengths. We present two main contributions: (1) we propose cMIM, a contrastive extension of MIM that eliminates the need for positive data augmentation and is robust to batch size, unlike InfoNCE-based methods; (2) we introduce {informative embeddings}, a general technique for extracting enriched representations from encoder--decoder models that substantially improve discriminative performance without additional training, and which apply broadly beyond MIM. Empirical results demonstrate that cMIM consistently outperforms MIM and InfoNCE in classification and regression tasks, while preserving comparable reconstruction quality. These findings suggest that cMIM provides a unified framework for learning representations that are simultaneously effective for discriminative and generative applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
