ONNXim:一款快速的周期级多核 NPU 仿真器
ONNXim: A Fast, Cycle-level Multi-core NPU Simulator
AI总结:
本文提出 ONNXim,一款基于 ONNX、周期级建模 DRAM/NoC 并简化确定性计算的多核 NPU 仿真器,最高较 Accel-sim 加速 384 倍,支持多租户 DNN 服务研究。
摘要:
随着 DNN 被广泛应用于各个应用领域,同时对计算和内存的需求不断提高,设计高效且高性能的 NPU(神经处理单元)正变得愈发重要。然而,现有的架构级 NPU 仿真器缺乏对高速仿真、多核建模、多租户场景、详细 DRAM/NoC 建模以及不同深度学习框架的支持,或在这些方面存在不足。为解决这些局限,本工作提出 ONNXim,这是一款面向 DNN 服务系统中多核 NPU 的快速周期级仿真器。它接收由各种深度学习框架生成、以 ONNX 图格式表示的 DNN 模型,以便于仿真。此外,基于典型 NPU 核以确定性计算延迟处理来自片上暂存存储器的张量分片这一观察,我们放弃了对计算的详细建模,同时仍保持仿真精度。ONNXim 还保留了计算与分片 DMA 之间的依赖关系。同时,DRAM 和 NoC 以周期级精度建模,以恰当地建模多个核之间的争用;这些核可为多租户执行不同的 DNN 模型。因此,ONNXim 显著快于现有仿真器(例如,相比 Accel-sim 最高可达 384 倍),并支持多种案例研究,例如多租户 NPU;此前由于速度较慢和/或功能缺失,这类研究并不现实。ONNXim 已公开发布于 https://github.com/PSAL-POSTECH/ONNXim。
英文摘要:
As DNNs are widely adopted in various application domains while demanding increasingly higher compute and memory requirements, designing efficient and performant NPUs (Neural Processing Units) is becoming more important. However, existing architectural NPU simulators lack support for high-speed simulation, multi-core modeling, multi-tenant scenarios, detailed DRAM/NoC modeling, and/or different deep learning frameworks. To address these limitations, this work proposes ONNXim, a fast cycle-level simulator for multi-core NPUs in DNN serving systems. It takes DNN models represented in the ONNX graph format generated from various deep learning frameworks for ease of simulation. In addition, based on the observation that typical NPU cores process tensor tiles from on-chip scratchpad memory with deterministic compute latency, we forgo a detailed modeling for the computation while still preserving simulation accuracy. ONNXim also preserves dependencies between compute and tile DMAs. Meanwhile, the DRAM and NoC are modeled in cycle-level to properly model contention among multiple cores that can execute different DNN models for multi-tenancy. Consequently, ONNXim is significantly faster than existing simulators (e.g., by up to 384x over Accel-sim) and enables various case studies, such as multi-tenant NPUs, that were previously impractical due to slow speed and/or lack of functionalities. ONNXim is publicly available at https://github.com/PSAL-POSTECH/ONNXim.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。