arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14525cs.NEcs.AIcs.LG
收藏

基于对比自监督学习的深度神经网络进化研究

Towards evolution of Deep Neural Networks through contrastive Self-Supervised learning

University of Coimbra(科英布拉大学)

Adriano Vinhas, João Correia, Penousal Machado

AI总结:

针对深度神经网络设计耗时长、依赖标注数据的问题,提出结合神经进化与自监督学习的框架,在CIFAR-10上验证可进化出合格网络且降低标注依赖,还发现自监督进化网络结构受标注数据量影响更小。

摘要:

深度神经网络(DNNs)已成功应用于各类问题,但普遍存在两大主要局限:一是设计周期长,二是严重依赖标注数据,而标注数据有时获取成本高、难度大。针对第一个问题,神经进化已被证实是实现DNN自动化设计的可行方案;针对第二个问题,自监督学习可利用无标注数据学习表征。本研究旨在探索神经进化如何助力自监督学习,缩小其与监督学习在性能上的差距。本文提出一个能借助自监督学习进化深度神经网络的框架。在CIFAR-10数据集上的实验结果表明,该框架能够进化出性能合格的神经网络,同时降低对标注数据的依赖。此外,对进化后网络结构的分析显示,与依赖监督学习的个体相比,通过自监督学习的网络,其结构受输入标注数据量的影响更小。

英文摘要:

Deep Neural Networks (DNNs) have been successfully applied to a wide range of problems. However, two main limitations are commonly pointed out. The first one is that they require long time to design. The other is that they heavily rely on labelled data, which can sometimes be costly and hard to obtain. In order to address the first problem, neuroevolution has been proved to be a plausible option to automate the design of DNNs. As for the second problem, self-supervised learning has been used to leverage unlabelled data to learn representations. Our goal is to study how neuroevolution can help self-supervised learning to bridge the gap to supervised learning in terms of performance. In this work, we propose a framework that is able to evolve deep neural networks using self-supervised learning. Our results on the CIFAR-10 dataset show that it is possible to evolve adequate neural networks while reducing the reliance on labelled data. Moreover, an analysis to the structure of the evolved networks suggests that the amount of labelled data fed to them has less effect on the structure of networks that learned via self-supervised learning, when compared to individuals that relied on supervised learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑