arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07519cs.CV
收藏

白化持续提升自监督学习性能

Whitening Consistently Improves Self-Supervised Learning

Budapest University of Technology and Economics(布达佩斯技术与经济大学)

András Kalapos, Bálint Gyires-Tóth

AI总结:

本研究提出将ZCA白化作为自监督学习编码器最后一层,验证了其对各类SSL方法、架构和数据集的通用提升效果,可使探测准确率提升1-5%,还提出了特征分析指标以识别坍缩模式。

摘要:

自监督学习(SSL)已被证实是一种学习视觉表征的强大方法。本研究提出将ZCA白化(零相位成分分析白化)作为自监督学习编码器的最后一层,通过对学习到的特征进行归一化和去相关来提升其质量。尽管以往研究已在SSL中使用过白化,但它对任意SSL模型的通用提升潜力尚未被探究。我们证明,在SSL预训练编码器的最后一层添加白化操作与自监督学习方法、编码器架构无关,因此能在多种编码器架构和数据集上提升各类SSL方法的性能。实验表明,白化可将线性探测和k近邻(k-NN)探测准确率提升1-5%。此外,我们提出了可对学习到的特征进行全面分析的指标,能为表征质量提供洞见,并帮助识别坍缩模式。

英文摘要:

Self-supervised learning (SSL) has been shown to be a powerful approach for learning visual representations. In this study, we propose incorporating ZCA whitening as the final layer of the encoder in self-supervised learning to enhance the quality of learned features by normalizing and decorrelating them. Although whitening has been utilized in SSL in previous works, its potential to universally improve any SSL model has not been explored. We demonstrate that adding whitening as the last layer of SSL pretrained encoders is independent of the self-supervised learning method and encoder architecture, thus it improves performance for a wide range of SSL methods across multiple encoder architectures and datasets. Our experiments show that whitening is capable of improving linear and k-NN probing accuracy by 1-5%. Additionally, we propose metrics that allow for a comprehensive analysis of the learned features, provide insights into the quality of the representations and help identify collapse patterns.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑