用神经元级模糊记忆方案节省RNN计算
Saving RNN Computations with a Neuron-Level Fuzzy Memoization Scheme
摘要：
递归神经网络（RNN）是自动语音识别或机器翻译等应用的关键技术。与传统的前馈DNN不同，RNN记住过去的信息以提高未来预测的准确性，因此它们对于序列处理问题非常有效。对于每个应用程序运行，递归层被执行多次，以处理潜在的大输入序列（单词，图像，音频帧等）。在本文中，我们观察到神经元的输出在连续调用中表现出微小的变化。我们利用这个属性来建立一个神经元级的模糊记忆计划，动态缓存每个神经元的输出，并重用它，每当它被预测，当前的输出将类似于以前计算的结果，避免以这种方式的输出计算。该方案的主要挑战是确定新神经元对序列中当前输入的输出是否与最近计算的结果相似。为此，我们用一个更简单的逐位神经网络（BNN）扩展了递归层，并证明了BNN和RNN的输出是高度相关的：如果两个BNN的输出非常相似，那么原始RNN层中相应的输出可能会出现可以忽略不计的变化。BNN提供了一种低成本和有效的机制，用于决定何时可以应用模糊记忆，对精度的影响很小。我们在最先进的RNN加速器上评估我们的记忆方案，用于来自多个应用领域的各种不同的神经网络。我们表明，我们的技术避免了超过26.7\%的计算，从而节省了21\%的能源和1.4倍的加速平均。
英文摘要：
Recurrent Neural Networks (RNNs) are a key technology for applications such as automatic speech recognition or machine translation. Unlike conventional feed-forward DNNs, RNNs remember past information to improve the accuracy of future predictions and, therefore, they are very effective for sequence processing problems. For each application run, recurrent layers are executed many times for processing a potentially large sequence of inputs (words, images, audio frames, etc.). In this paper, we observe that the output of a neuron exhibits small changes in consecutive invocations.~We exploit this property to build a neuron-level fuzzy memoization scheme, which dynamically caches each neuron's output and reuses it whenever it is predicted that the current output will be similar to a previously computed result, avoiding in this way the output computations. The main challenge in this scheme is determining whether the new neuron's output for the current input in the sequence will be similar to a recently computed result. To this end, we extend the recurrent layer with a much simpler Bitwise Neural Network (BNN), and show that the BNN and RNN outputs are highly correlated: if two BNN outputs are very similar, the corresponding outputs in the original RNN layer are likely to exhibit negligible changes. The BNN provides a low-cost and effective mechanism for deciding when fuzzy memoization can be applied with a small impact on accuracy. We evaluate our memoization scheme on top of a state-of-the-art accelerator for RNNs, for a variety of different neural networks from multiple application domains. We show that our technique avoids more than 26.7\% of computations, resulting in 21\% energy savings and 1.4x speedup on average.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
