arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.05988cs.NE
收藏

用多重搜索神经进化优化深度神经网络

Optimizing Deep Neural Networks with Multiple Search Neuroevolution

Ahmed Aly, David Weikersdorfer, Claire Delaunay

摘要:

本文提出了一种称为多搜索神经进化(MSN)的进化元启发式算法来优化深度神经网络。该算法试图同时在搜索空间中搜索多个有希望的区域,并保持它们之间足够的距离。通过对两个任务的神经网络训练,对该算法进行了测试,并与其他优化算法进行了比较。第一个任务是解决具有挑战性拓扑的全局优化函数。我们发现MSN优于经典的优化算法,如演化策略,减少了至少2倍的优化步骤。第二个任务是在流行的MNIST数据集上训练卷积神经网络(CNN)。使用3.33%的训练集,MSN达到了90%的验证准确率。随机梯度下降(SGD)能够匹配相同的精度数字,同时减少7倍的优化步骤。尽管滞后,但MSN元数据库训练4.7M参数的CNN的事实表明了未来发展的希望。这是迄今为止使用仅50个样本池开发的最大网络。

英文摘要:

This paper presents an evolutionary metaheuristic called Multiple Search Neuroevolution (MSN) to optimize deep neural networks. The algorithm attempts to search multiple promising regions in the search space simultaneously, maintaining sufficient distance between them. It is tested by training neural networks for two tasks, and compared with other optimization algorithms. The first task is to solve Global Optimization functions with challenging topographies. We found to MSN to outperform classic optimization algorithms such as Evolution Strategies, reducing the number of optimization steps performed by at least 2X. The second task is to train a convolutional neural network (CNN) on the popular MNIST dataset. Using 3.33% of the training set, MSN reaches a validation accuracy of 90%. Stochastic Gradient Descent (SGD) was able to match the same accuracy figure, while taking 7X less optimization steps. Despite lagging, the fact that the MSN metaheurisitc trains a 4.7M-parameter CNN suggests promise for future development. This is by far the largest network ever evolved using a pool of only 50 samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑