arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.11778cs.LGeess.SPstat.ML
收藏

具有生成神经元的自组织操作神经网络

Self-Organized Operational Neural Networks with Generative Neurons

Qatar University(卡塔尔大学);Izmir University of Economics(伊兹密尔经济大学);Aarhus University(奥胡斯大学);Tampere University(坦佩雷大学)

Serkan Kiranyaz, Junaid Malik, Habib Ben Abdallah, Turker Ince, Alexandros Iosifidis, Moncef Gabbouj

摘要:

最近提出了操作神经网络(ONN)以解决传统卷积神经网络(CNN)的众所周知的限制和缺点,诸如具有唯一线性神经元模型的网络同质性。ONN是具有广义神经元模型的异构网络,该广义神经元模型可以封装任何非线性算子集合以提高多样性并以最小的网络复杂度和训练数据学习高度复杂和多模态的函数或空间。然而,贪婪迭代搜索(GIS)方法,这是一种用于在ONN中找到最佳算子的搜索方法,需要许多训练会话才能找到每层的单个算子集。这不仅在计算上要求很高,而且网络的异质性也受到限制,因为同一组运算符将用于每一层中的所有神经元。此外,ONN的性能直接取决于所使用的算子集库,这引入了一定的性能降低的风险,特别是当特定任务所需的最佳算子集从库中缺失时。为了解决这些问题,并实现最终的异质性水平,以提高网络的多样性和计算效率,在这项研究中,我们提出了具有生成神经元的自组织ONNs(Self-ONNs),这些神经元能够在训练过程中适应(优化)每个连接的节点运算符。因此,Self-ONNs可以具有手头的学习问题所需的最大异质性水平。此外,这种能力避免了需要具有固定的运算符集合库和在库内的先前运算符搜索以便找到最佳可能的运算符集合。我们进一步制定了训练方法,通过Self-ONNs的操作层反向传播错误。

英文摘要:

Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitations and drawbacks of conventional Convolutional Neural Networks (CNNs) such as network homogeneity with the sole linear neuron model. ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, Greedy Iterative Search (GIS) method, which is the search method used to find optimal operators in ONNs takes many training sessions to find a single operator set per layer. This is not only computationally demanding, but the network heterogeneity is also limited since the same set of operators will then be used for all neurons in each layer. Moreover, the performance of ONNs directly depends on the operator set library used, which introduces a certain risk of performance degradation especially when the optimal operator set required for a particular task is missing from the library. In order to address these issues and achieve an ultimate heterogeneity level to boost the network diversity along with computational efficiency, in this study we propose Self-organized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection during the training process. Therefore, Self-ONNs can have an utmost heterogeneity level required by the learning problem at hand. Moreover, this ability voids the need of having a fixed operator set library and the prior operator search within the library in order to find the best possible set of operators. We further formulate the training method to back-propagate the error through the operational layers of Self-ONNs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑