arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.00915cs.ET
收藏

二值化神经网络硬件实现的随机计算

Stochastic Computing for Hardware Implementation of Binarized Neural Networks

Tifenn Hirtzlin, Bogdan Penkovsky, Marc Bocquet, Jacques-Olivier Klein, Jean-Michel Portal, Damien Querlioz

摘要:

二进制神经网络是最近发现的一类具有最小存储器需求且不依赖乘法的神经网络,是实现紧凑且节能的推理硬件的绝佳机会。然而,这样的神经网络通常不是完全二进制化的:它们的第一层保持固定点输入。在这项工作中,我们提出了一个随机计算版本的二进制神经网络,其中输入也被二进制化。对Fashion-MNIST和CIFAR-10数据集的仿真结果表明,该网络的性能接近传统二值化神经网络。我们的证据表明,培训过程中应适应随机计算使用。最后,ASIC实现我们的计划进行了调查,在一个系统中,密切相关的逻辑和存储器,实现自旋力矩磁阻随机存取存储器。该分析表明,随机计算方法可以允许在面积方面相对于传统的二值化神经网络有相当大的节省(在Fashion-MNIST任务上减少62%的面积)。如果我们接受合理的精度降低,它还可以在能源消耗方面实现重要的节省:例如,Fashion-MNIST测试精度可以节省2.1倍,成本为1.4%。这些结果突出了二值化神经网络在硬件实现方面的高潜力,并且使它们适应硬件约束可以提供重要的好处。

英文摘要:

Binarized Neural Networks, a recently discovered class of neural networks with minimal memory requirements and no reliance on multiplication, are a fantastic opportunity for the realization of compact and energy efficient inference hardware. However, such neural networks are generally not entirely binarized: their first layer remains with fixed point input. In this work, we propose a stochastic computing version of Binarized Neural Networks, where the input is also binarized. Simulations on the example of the Fashion-MNIST and CIFAR-10 datasets show that such networks can approach the performance of conventional Binarized Neural Networks. We evidence that the training procedure should be adapted for use with stochastic computing. Finally, the ASIC implementation of our scheme is investigated, in a system that closely associates logic and memory, implemented by Spin Torque Magnetoresistive Random Access Memory. This analysis shows that the stochastic computing approach can allow considerable savings with regards to conventional Binarized Neural networks in terms of area (62% area reduction on the Fashion-MNIST task). It can also allow important savings in terms of energy consumption, if we accept reasonable reduction of accuracy: for example a factor 2.1 can be saved, with the cost of 1.4% in Fashion-MNIST test accuracy. These results highlight the high potential of Binarized Neural Networks for hardware implementation, and that adapting them to hardware constrains can provide important benefits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑