SIDDA:面向等变神经网络图像分类的Sinkhorn动态域适应
SIDDA: SInkhorn Dynamic Domain Adaptation for Image Classification with Equivariant Neural Networks
AI总结:
针对协变量偏移下神经网络泛化差、现有域适应方法超参数调优成本高的问题,提出基于Sinkhorn散度的SIDDA算法,兼容多架构,搭配等变神经网络可显著提升分类准确率与模型校准度。
摘要:
现代神经网络(NNs)在存在“协变量偏移”的情况下往往泛化能力不佳;所谓协变量偏移,是指训练数据与测试数据的分布存在差异,但分类标签的条件分布保持不变的场景。在这类情况下,神经网络的泛化问题可归结为学习更具域不变性的特征的问题。域适应(DA)方法包含一系列旨在实现这一目标的技术;但这类方法存在需要大量超参数调优的痛点,进而会产生高昂的计算成本。\n在本研究中,我们提出了SIDDA——一种基于Sinkhorn散度的开箱即用域适应训练算法,它仅需极少的超参数调优和计算开销即可实现有效的域对齐。我们在多个复杂度各异的模拟数据集和真实数据集上验证了该方法的有效性,涵盖简单形状、手写数字以及真实天文观测数据。SIDDA可兼容多种神经网络架构,当与等变神经网络(ENNs)搭配使用时,在提升分类准确率和模型校准度方面表现尤为出色。\n我们发现,SIDDA增强了神经网络的泛化能力,在无标签目标数据上的分类准确率提升最高可达约40%。我们还针对二面体群$D_N$的不同群阶,研究了域适应在等变神经网络上的效果,发现模型性能会随着等变程度的提升而改善。最后,我们发现SIDDA提升了源数据和目标数据上的模型校准度——在期望校准误差(ECE)和Brier得分上实现了超过一个数量级的提升。SIDDA的多功能性结合其自动化的域对齐方式,有望通过支持高泛化性模型的开发,推动多数据集研究的发展。
英文摘要:
Modern neural networks (NNs) often do not generalize well in the presence of a "covariate shift"; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more domain-invariant features. Domain adaptation (DA) methods include a range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SIDDA, an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, and real astronomical observations. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with equivariant neural networks (ENNs). We find that SIDDA enhances the generalization capabilities of NNs, achieving up to a $\approx40\%$ improvement in classification accuracy on unlabeled target data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group $D_N$, and find that the model performance improves as the degree of equivariance increases. Finally, we find that SIDDA enhances model calibration on both source and target data--achieving over an order of magnitude improvement in the ECE and Brier score. SIDDA's versatility, combined with its automated approach to domain alignment, has the potential to advance multi-dataset studies by enabling the development of highly generalizable models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。