arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.08980cs.LGcs.AR
收藏

神经网络推理对高数据率、低延迟科学应用的架构影响

Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications

UC San Diego(加州大学圣迭戈分校);Fermi National Accelerator Laboratory(费米国家加速器实验室)

Olivia Weng, Alexander Redding, Nhan Tran, Javier Mauricio Duarte, Ryan Kastner

AI总结:

该研究指出高数据率、低延迟科学应用要求神经网络参数全部存储在芯片上,并常需定制或可重构逻辑,极端情况下需定制芯片以满足严格约束。

摘要:

随着越来越多的科学领域依赖神经网络(NN)以极高吞吐量和极低延迟处理传入数据,开发所有参数均存储在芯片上的神经网络变得至关重要。在许多此类应用中,没有足够的时间访问片外并检索权重。更甚者,诸如DRAM之类的片外存储器不具备以数据产生速度(例如每25纳秒)处理这些神经网络所需的带宽。因此,这些极端的延迟和带宽要求对旨在运行这些神经网络的硬件产生了架构影响:1)所有神经网络参数必须适配在芯片上,2)通常需要协同设计定制/可重构逻辑以满足这些延迟和带宽约束。在我们的工作中,我们表明许多科学神经网络应用必须完全在芯片上运行,在极端情况下需要定制芯片以满足如此严格的约束。

英文摘要:

With more scientific fields relying on neural networks (NNs) to process data incoming at extreme throughputs and latencies, it is crucial to develop NNs with all their parameters stored on-chip. In many of these applications, there is not enough time to go off-chip and retrieve weights. Even more so, off-chip memory such as DRAM does not have the bandwidth required to process these NNs as fast as the data is being produced (e.g., every 25 ns). As such, these extreme latency and bandwidth requirements have architectural implications for the hardware intended to run these NNs: 1) all NN parameters must fit on-chip, and 2) codesigning custom/reconfigurable logic is often required to meet these latency and bandwidth constraints. In our work, we show that many scientific NN applications must run fully on chip, in the extreme case requiring a custom chip to meet such stringent constraints.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑