arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.09580cs.CVcs.AI
收藏

不要惧怕奇特的激活函数:EUAF 以及更多

Don't Fear Peculiar Activation Functions: EUAF and Beyond

Center of Mathematical Artificial Intelligence(数学人工智能中心);Department of Applied Mathematics(应用数学系);Department of Biomedical Engineering(生物医学工程系);Institute of Medical Technology(医学技术研究所);Independent Researcher(独立研究者)

Qianchao Wang, Shijun Zhang, Dong Zeng, Zhaoheng Xie, Hengtao Guo, Feng-Lei Fan, Tieyong Zeng

AI总结:

本文提出了一种新的超表达激活函数PEUAF,并通过系统实验验证其在多个数据集上的有效性,同时扩展了超表达激活函数家族,证明任何连续函数可通过固定大小网络和特定超表达激活函数近似。

摘要:

在本文中,我们提出了一种名为参数基本通用激活函数(PEUAF)的新超表达激活函数。我们通过系统且全面的实验,在各种工业和图像数据集上,包括CIFAR10、Tiny-ImageNet和ImageNet,展示了PEUAF的有效性。此外,我们显著扩展了超表达激活函数家族,其存在已在一些近期工作中得到证明,通过展示任何连续函数都可以通过具有特定超表达激活函数的固定大小网络近似到任意所需的精度。具体来说,我们的工作解决了阻碍超表达激活函数发展的两个主要瓶颈:超表达函数的有限识别,这引发了对其广泛应用的怀疑,以及它们通常奇特的形式,这导致对其可扩展性和实际应用中的实用性存疑。

英文摘要:

In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEUAF through systematic and comprehensive experiments on various industrial and image datasets, including CIFAR10, Tiny-ImageNet, and ImageNet. Moreover, we significantly generalize the family of super-expressive activation functions, whose existence has been demonstrated in several recent works by showing that any continuous function can be approximated to any desired accuracy by a fixed-size network with a specific super-expressive activation function. Specifically, our work addresses two major bottlenecks in impeding the development of super-expressive activation functions: the limited identification of super-expressive functions, which raises doubts about their broad applicability, and their often peculiar forms, which lead to skepticism regarding their scalability and practicality in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑