鱼与熊掌兼得:通过格拉斯曼类表示提升准确率与迁移性
Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation
AI总结:
研究将神经网络类向量推广为Grassmann流形子空间,提出GCR方法联合优化类子空间与模型参数,在ImageNet-1K上显著降低top-1误差并提升下游任务的线性迁移准确率。
摘要:
我们将神经网络中的类向量推广为线性子空间(即Grassmann流形上的点),并证明Grassmann类表示(GCR)能够同时提升准确率和特征迁移性。在GCR中,每个类是一个子空间,logit定义为特征投影到类子空间的范数。我们将Riemannian SGD集成到深度学习框架中,使得Grassmannian中的类子空间与模型其余参数联合优化。与向量形式相比,子空间的表示能力更强大。我们发现在ImageNet-1K上,ResNet50-D、ResNeXt50、Swin-T和Deit3-S的top-1误差分别降低了5.6%、4.5%、3.0%和3.5%。子空间也为特征变化提供了自由度,我们观察到随着子空间维度增加,类内特征变异性增大。因此,我们发现GCR特征质量更适用于下游任务。对于ResNet50-D,在6个数据集上的平均线性迁移准确率相比vanilla softmax的强基线从77.98%提升至79.70%。对于Swin-T,从81.5%提升至83.4%;对于Deit3,从73.8%提升至81.4%。基于这些令人鼓舞的结果,我们相信更多应用可受益于Grassmann类表示。代码发布于https://github.com/innerlee/GCR。
英文摘要:
We generalize the class vectors found in neural networks to linear subspaces (i.e.~points in the Grassmann manifold) and show that the Grassmann Class Representation (GCR) enables the simultaneous improvement in accuracy and feature transferability. In GCR, each class is a subspace and the logit is defined as the norm of the projection of a feature onto the class subspace. We integrate Riemannian SGD into deep learning frameworks such that class subspaces in a Grassmannian are jointly optimized with the rest model parameters. Compared to the vector form, the representative capability of subspaces is more powerful. We show that on ImageNet-1K, the top-1 error of ResNet50-D, ResNeXt50, Swin-T and Deit3-S are reduced by 5.6%, 4.5%, 3.0% and 3.5%, respectively. Subspaces also provide freedom for features to vary and we observed that the intra-class feature variability grows when the subspace dimension increases. Consequently, we found the quality of GCR features is better for downstream tasks. For ResNet50-D, the average linear transfer accuracy across 6 datasets improves from 77.98% to 79.70% compared to the strong baseline of vanilla softmax. For Swin-T, it improves from 81.5% to 83.4% and for Deit3, it improves from 73.8% to 81.4%. With these encouraging results, we believe that more applications could benefit from the Grassmann class representation. Code is released at https://github.com/innerlee/GCR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。