为对比学习最大化增量信息熵
Maximizing Incremental Information Entropy for Contrastive Learning
AI总结：
本文提出IE-CL框架，通过优化增强视图间的熵增来提升对比学习性能，在CIFAR-10/100、STL-10和ImageNet数据集上验证了其有效性，并能无缝集成到现有框架中。
摘要：
对比学习在自监督表示学习中取得了显著成功，通常由信息论目标如互信息最大化引导。受静态增强和刚性不变性约束的限制，我们提出了IE-CL（增量熵对比学习）框架，明确优化增强视图间的熵增，同时保持语义一致性。我们的理论框架将编码器视为信息瓶颈，并提出联合优化两个组件：用于熵生成的可学习转换和用于其保持的编码器正则化器。在CIFAR-10/100、STL-10和ImageNet上的实验表明，IE-CL在小批量设置下一致提升了性能。此外，我们的核心模块可以无缝集成到现有框架中。这项工作连接了理论原则与实践，为对比学习提供了新的视角。
英文摘要：
Contrastive learning has achieved remarkable success in self-supervised representation learning, often guided by information-theoretic objectives such as mutual information maximization. Motivated by the limitations of static augmentations and rigid invariance constraints, we propose IE-CL (Incremental-Entropy Contrastive Learning), a framework that explicitly optimizes the entropy gain between augmented views while preserving semantic consistency. Our theoretical framework reframes the challenge by identifying the encoder as an information bottleneck and proposes a joint optimization of two components: a learnable transformation for entropy generation and an encoder regularizer for its preservation. Experiments on CIFAR-10/100, STL-10, and ImageNet demonstrate that IE-CL consistently improves performance under small-batch settings. Moreover, our core modules can be seamlessly integrated into existing frameworks. This work bridges theoretical principles and practice, offering a new perspective in contrastive learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
