arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.07294cs.LGcs.AIcs.NE
收藏

面向深度神经网络的计算与存储高效二次神经元

Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks

Technical University of Munich(慕尼黑工业大学);Technical University of Darmstadt(达姆施塔特工业大学);Zhejiang University(浙江大学)

Chuangtao Chen, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li

AI总结:

本文提出一种高效二次神经元架构,通过增强对二阶计算信息的利用提升表达能力,使DNNs以更少神经元和计算成本达到相似准确率,并在多种任务中展现出优于传统线性和非线性神经元的计算与存储效率。

摘要:

深度神经网络(DNNs)已被广泛应用于计算机视觉和自然语言处理等多个领域。然而,DNNs的卓越成就伴随着巨大的计算需求,从而阻碍了其在资源受限设备上的适用性。为应对这一挑战,许多研究人员将重点放在作为神经网络基本构建块的基本神经元结构上,以降低计算和存储成本。本文提出了一种高效的二次神经元架构,其特点在于增强了对二阶计算信息的利用。得益于更好的表达能力,采用所提二次神经元的DNNs能够以更少的神经元和计算成本达到相似的准确率。实验结果表明,与先前工作中的线性和非线性神经元相比,所提二次神经元结构在多种任务中展现出更优的计算和存储效率。

英文摘要:

Deep neural networks (DNNs) have been widely deployed across diverse domains such as computer vision and natural language processing. However, the impressive accomplishments of DNNs have been realized alongside extensive computational demands, thereby impeding their applicability on resource-constrained devices. To address this challenge, many researchers have been focusing on basic neuron structures, the fundamental building blocks of neural networks, to alleviate the computational and storage cost. In this work, an efficient quadratic neuron architecture distinguished by its enhanced utilization of second-order computational information is introduced. By virtue of their better expressivity, DNNs employing the proposed quadratic neurons can attain similar accuracy with fewer neurons and computational cost. Experimental results have demonstrated that the proposed quadratic neuron structure exhibits superior computational and storage efficiency across various tasks when compared with both linear and non-linear neurons in prior work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑