多对称集成:通过相反对称性提升多样性与泛化能力
Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries
AI总结:
提出多对称集成(MSE)框架,通过沿对称轴捕获相反假设(不变与等变)构建多样集成,超越随机扰动,提升ImageNet等大数据集上的分类性能、不确定性量化与迁移泛化。
摘要:
深度集成(DE)通过随机初始化的随机性学习多样化的成员,在提升模型性能方面取得了成功。尽管近期的工作尝试通过超参数或正则化损失函数进一步促进DE中的多样性,但这些方法主要仍依赖随机方法来探索假设空间。在本工作中,我们提出了多对称集成(MSE),一种通过沿对称轴捕获假设的多样性来构建多样化集成的框架,该框架探索了超越模型权重和超参数随机扰动的假设空间。我们利用对比表示学习的最新进展,创建分别捕获不变和等变功能类别的相反假设的模型,并提出一种简单的集成方法,以高效地为给定任务组合适当的假设。我们展示了MSE有效地捕获了在像ImageNet这样的大规模、多样化数据集中通常需要的冲突假设的多样性。由于其固有的多样性,MSE在一系列迁移任务中提升了分类性能、不确定性量化以及泛化能力。
英文摘要:
Deep ensembles (DE) have been successful in improving model performance by learning diverse members via the stochasticity of random initialization. While recent works have attempted to promote further diversity in DE via hyperparameters or regularizing loss functions, these methods primarily still rely on a stochastic approach to explore the hypothesis space. In this work, we present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes, which explore the hypothesis space beyond stochastic perturbations of model weights and hyperparameters. We leverage recent advances in contrastive representation learning to create models that separately capture opposing hypotheses of invariant and equivariant functional classes and present a simple ensembling approach to efficiently combine appropriate hypotheses for a given task. We show that MSE effectively captures the multiplicity of conflicting hypotheses that is often required in large, diverse datasets like ImageNet. As a result of their inherent diversity, MSE improves classification performance, uncertainty quantification, and generalization across a series of transfer tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。