arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.03842cs.LGcs.NEstat.ML
收藏

机器学习的通用激活函数

Universal Activation Function For Machine Learning

University of Victoria(维多利亚大学)

Brosnan Yuen, Minh Tu Hoang, Xiaodai Dong, Tao Lu

摘要:

本文提出了一种通用激活函数(UAF),它在量化、分类和强化学习(RL)问题中实现了接近最佳的性能。对于任何给定的问题,优化算法能够通过调整UAF的参数将UAF进化为合适的激活函数。对于CIFAR-10分类和VGG-8,UAF收敛到类似Mish的激活函数,当与其他激活函数相比时,其具有接近最佳性能$F_{1} = 0.9017\pm0.0040$。对于在30 dB信噪比(SNR)环境中的模拟9种气体混合物的量化,UAF收敛到恒等函数,其具有接近最优的均方根误差$0.4888 \pm 0.0032$ $μM$。在BipedalWalker-v2 RL数据集中,UAF在$961 \pm 193$ epoch中实现了250奖励,这证明了UAF在最低数量的epoch中收敛。此外,UAF收敛到BipedalWalker-v2 RL数据集中的新激活函数。

英文摘要:

This article proposes a Universal Activation Function (UAF) that achieves near optimal performance in quantification, classification, and reinforcement learning (RL) problems. For any given problem, the optimization algorithms are able to evolve the UAF to a suitable activation function by tuning the UAF's parameters. For the CIFAR-10 classification and VGG-8, the UAF converges to the Mish like activation function, which has near optimal performance $F_{1} = 0.9017\pm0.0040$ when compared to other activation functions. For the quantification of simulated 9-gas mixtures in 30 dB signal-to-noise ratio (SNR) environments, the UAF converges to the identity function, which has near optimal root mean square error of $0.4888 \pm 0.0032$ $μM$. In the BipedalWalker-v2 RL dataset, the UAF achieves the 250 reward in $961 \pm 193$ epochs, which proves that the UAF converges in the lowest number of epochs. Furthermore, the UAF converges to a new activation function in the BipedalWalker-v2 RL dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑