DeepStack:可扩展且准确的分布式3D堆叠AI加速器设计空间探索
DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference
AI总结:
DeepStack通过高效模型和工具实现分布式3D堆叠AI系统的早期设计空间探索,利用双阶段网络抽象和计算通信重叠技术,提升性能模拟效率,并揭示批量大小对架构设计的根本影响。
摘要:
混合键合和封装技术的进步推动了更高内存带宽和容量的3D DRAM堆叠加速器的研究。随着LLM参数规模扩大到数百亿甚至万亿级别,跨多个3D芯片的分布式推断变得至关重要。针对跨堆栈协同设计日益关键的需求,我们提出了DeepStack,一种准确且高效的性能模型和工具,用于支持分布式3D堆叠AI系统的早期系统-硬件协同设计空间探索(DSE)。在硬件层面,DeepStack捕捉了细粒度的3D内存语义,如事务感知带宽、银行激活约束、缓冲限制和热功率建模。在系统层面,DeepStack集成了全面的并行化策略和执行调度,用于分布式LLM推断。通过双阶段网络抽象和片级计算通信重叠等新建模技术,我们实现了比最先进的仿真器快100,000倍的运行时间,准确率在我们内部3D设计、NS-3后端(2.12%)和vLLM服务在8xB200 GPU上的验证下保持一致。通过分层设计空间搜索,DeepStack能够高效探索跨越3D堆叠DRAM层、DRAM垂直连接、互连、计算-内存分配和分布式调度的2.5x10^14个设计点。与基线设计相比,DeepStack通过协同优化并行性和3D架构搜索,实现了最高9.5倍的吞吐量提升。我们的DSE进一步揭示,批量大小比prefill/decode区分驱动更根本的架构划分,且并行策略和硬件架构紧密相关——不完整的调度搜索导致永久性的硅不可恢复的软件调优。我们计划开源DeepStack以支持未来研究。
英文摘要:
Advances in hybrid bonding and packaging have driven growing interest in 3D DRAM-stacked AI accelerators. As large language models (LLMs) scale to hundreds of billions or trillions of parameters, distributed inference across multiple 3D chips has become essential for AI serving. This trend makes cross-stack co-design critical because system-level parallelization and scheduling choices are tightly coupled with hardware characteristics such as memory organization, interconnects, and thermal constraints. We present DeepStack, an accurate performance model and efficient design space exploration (DSE) framework for distributed 3D-stacked LLM inference. At the hardware level, DeepStack captures transaction-aware memory bandwidth, bank activation constraints, buffering limitations, and thermal and power behavior. At the system level, it incorporates comprehensive parallelization strategies and execution scheduling. Through a dual-stage network abstraction and tile-level compute-communication overlap modeling, DeepStack achieves up to 100,000x faster evaluation than state-of-the-art simulators at comparable accuracy. We cross-validate DeepStack against our in-house 3D designs, an NS-3 backend with 2.12% error, and vLLM serving on eight B200 GPUs with 12.92% error. Combined with hierarchical search, DeepStack efficiently explores about 2.5 x 10^14 design points spanning the number of stacked DRAM layers, DRAM vertical connectivity, interconnects, compute-memory allocation, and distributed scheduling under thermal and area constraints. A search-space ablation shows that restricted DSE baselines can miss up to 9.5x modeled throughput. Beyond modeling and DSE, DeepStack derives design implications for distributed 3D AI systems and guides performance optimization across the stack. Source code and artifacts are available at https://github.com/tile-ai/DeepStack/tree/ae.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。