哪种缩放规则适用于人工神经网络
Which scaling rule applies to Artificial Neural Networks
摘要：
经验表明，包括分离的单处理器的协作和通信计算系统具有严重的性能限制。冯·诺依曼在他的经典著作《初稿》中警告说，使用“太快的处理器”会破坏他的简单“程序”（但不是他的计算模型！）;此外，使用经典的计算范例来模仿神经元的操作是不合理的。Amdahl补充说，由许多处理器组成的大型机器有一个固有的缺点。鉴于ANN的组件彼此之间进行大量通信，它们是由大量设计/制造用于传统计算的组件构建的，此外，它们试图使用不适当的技术解决方案来模拟生物操作，它们可实现的有效载荷计算性能在概念上是适度的。基于人工智能的系统产生的工作负载类型导致了异常低的有效载荷计算性能，并且它们的设计/技术将它们的大小限制在“玩具”级别系统之上：基于处理器的ANN系统的缩放是强非线性的。鉴于人工神经网络系统的扩散和不断增长的规模，我们提出了提前估计设备或应用程序的效率的想法。通过分析已发表的测量结果，我们提供的证据表明，数据传输时间的作用极大地影响了ANN的性能和可行性。它讨论了一些主要的理论限制因素，人工神经网络的层次结构和他们的通信技术实现的方法如何影响其效率。本文从冯·诺依曼的原始模型出发，在不忽略加工时间以外的传递时间的情况下，导出了对阿姆达尔定律的适当解释和处理。它表明，在这种解释中，Amdahl定律正确地描述了ANN。
英文摘要：
The experience shows that cooperating and communicating computing systems, comprising segregated single processors, have severe performance limitations. In his classic "First Draft" von Neumann warned that using a "too fast processor" vitiates his simple "procedure" (but not his computing model!); furthermore, that using the classic computing paradigm for imitating neuronal operations, is unsound. Amdahl added that large machines, comprising many processors, have an inherent disadvantage. Given that ANN's components are heavily communicating with each other, they are built from a large number of components designed/fabricated for use in conventional computing, furthermore they attempt to mimic biological operation using improper technological solutions, their achievable payload computing performance is conceptually modest. The type of workload that AI-based systems generate leads to an exceptionally low payload computational performance, and their design/technology limits their size to just above the "toy" level systems: the scaling of processor-based ANN systems is strongly nonlinear. Given the proliferation and growing size of ANN systems, we suggest ideas to estimate in advance the efficiency of the device or application. Through analyzing published measurements we provide evidence that the role of data transfer time drastically influences both ANNs performance and feasibility. It is discussed how some major theoretical limiting factors, ANN's layer structure and their methods of technical implementation of communication affect their efficiency. The paper starts from von Neumann's original model, without neglecting the transfer time apart from processing time; derives an appropriate interpretation and handling for Amdahl's law. It shows that, in that interpretation, Amdahl's Law correctly describes ANNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
