加法器视角下的循环神经网络：超前进位RNN
Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN
摘要：
递归网络结构是序列建模中广泛使用的模型，但其串行依赖性阻碍了计算的并行化，使得运算效率低下。在数字电子学的早期阶段，串行加法器也遇到了同样的问题。在本文中，我们讨论了循环神经网络（RNN）和串行加法器之间的相似之处。受超前进位加法器的启发，我们将超前进位模块引入到RNN中，使RNN的并行运行成为可能。然后设计了RNN的并行计算方法，最后提出了Carry-lookahead RNN（CL-RNN）。CL-RNN具有并行性和灵活的感受野等优点。通过一组全面的测试，我们验证了CL-RNN在专门为RNN设计的序列建模任务中可以比现有的典型RNN表现得更好。
英文摘要：
The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
