arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1808.01766cs.NE
收藏

基于进化计算的深度卷积神经网络优化研究

On Optimizing Deep Convolutional Neural Networks by Evolutionary Computing

M. U. B. Dias, D. D. N. De Silva, S. Fernando

摘要:

深度网络优化是目前非常活跃的研究领域。随着神经网络变得越来越深,手动优化网络的能力变得越来越困难。提出了小批量归一化、有效域的识别、动量更新、残差块的引入、学习率的采用等方法来加快人工训练过程中的收敛速度,同时保持较高的精度水平。然而,对于给定的问题,寻找最优的拓扑结构的问题正在成为一个具有挑战性的任务,需要立即解决。很少有研究人员试图使用进化计算方法来优化网络结构。其中,很少有人成功进化出具有强化学习和长短期记忆的网络。很少有人将进化编程应用于深度卷积神经网络。这些尝试主要是对网络结构进行改进,然后对网络的超参数进行优化。然而,仍然缺乏在当前在手动过程中实践的技术下演化深层网络结构的机制。将这种技术结合到染色体水平的进化计算中,当然可以使我们获得更好的拓扑深层结构。本文最后指出了基于进化的深度神经网络和深度神经网络之间的差距。此外,它还提出了一些使用进化计算技术优化深度神经网络的见解。

英文摘要:

Optimization for deep networks is currently a very active area of research. As neural networks become deeper, the ability in manually optimizing the network becomes harder. Mini-batch normalization, identification of effective respective fields, momentum updates, introduction of residual blocks, learning rate adoption, etc. have been proposed to speed up the rate of convergent in manual training process while keeping the higher accuracy level. However, the problem of finding optimal topological structure for a given problem is becoming a challenging task need to be addressed immediately. Few researchers have attempted to optimize the network structure using evolutionary computing approaches. Among them, few have successfully evolved networks with reinforcement learning and long-short-term memory. A very few has applied evolutionary programming into deep convolution neural networks. These attempts are mainly evolved the network structure and then subsequently optimized the hyper-parameters of the network. However, a mechanism to evolve the deep network structure under the techniques currently being practiced in manual process is still absent. Incorporation of such techniques into chromosomes level of evolutionary computing, certainly can take us to better topological deep structures. The paper concludes by identifying the gap between evolutionary based deep neural networks and deep neural networks. Further, it proposes some insights for optimizing deep neural networks using evolutionary computing techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑