arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1701.03452cs.NEstat.ML
收藏

循环神经网络的简化最小门控单元变分

Simplified Minimal Gated Unit Variations for Recurrent Neural Networks

Joel Heck, Fathi M. Salem

摘要:

具有各种类型的隐藏单元的循环神经网络已被用于解决涉及序列数据的各种问题。最近的两个提议,门控经常性单位(GRU)和最小门控单位(MGU),在示例公共数据集上显示出相当有前途的结果。在本文中,我们介绍了三个模型的最小门控单元(MGU),进一步简化了设计,通过减少遗忘门动态方程中的参数的数量。这三种模型变体,简称为MGU1,MGU2和MGU3,在MNIST数据集和路透社新闻主题(RNT)数据集生成的序列上进行了测试。新模型显示出与MGU模型相似的精度,同时使用更少的参数,从而降低了训练费用。一种模型变体,即MGU2,在所考虑的数据集上表现优于MGU,因此可以在循环神经网络中用作MGU或GRU的替代。

英文摘要:

Recurrent neural networks with various types of hidden units have been used to solve a diverse range of problems involving sequence data. Two of the most recent proposals, gated recurrent units (GRU) and minimal gated units (MGU), have shown comparable promising results on example public datasets. In this paper, we introduce three model variants of the minimal gated unit (MGU) which further simplify that design by reducing the number of parameters in the forget-gate dynamic equation. These three model variants, referred to simply as MGU1, MGU2, and MGU3, were tested on sequences generated from the MNIST dataset and from the Reuters Newswire Topics (RNT) dataset. The new models have shown similar accuracy to the MGU model while using fewer parameters and thus lowering training expense. One model variant, namely MGU2, performed better than MGU on the datasets considered, and thus may be used as an alternate to MGU or GRU in recurrent neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑