SpaDA:一种空间数据流架构编程语言
SpaDA: A Spatial Dataflow Architecture Programming Language
AI总结:
SpaDA为空间数据流架构提供高阶编程接口,通过抽象低层细节,实现对数据布局、数据流模式和异步操作的精确控制,使复杂并行模式在更少代码下实现高性能计算。
摘要:
类似于Cerebras Wafer-Scale Engine的空间数据流架构通过在数以万计的处理元素上分布暂存内存,在AI和科学计算中提供卓越性能。然而,编程这些架构仍然困难:没有共享内存,数据移动需要显式配置,异步任务管理引入了大量复杂性。我们提出了SpaDA,一种提供对数据布局、数据流模式和异步操作的精确控制的编程语言,同时抽象低层架构细节。我们设计并实现了一个针对Cerebras CSL的编译器,通过多级降低和独特的优化步骤。SpaDA作为高层次编程接口和领域特定语言(DSL)的中间表示,此处通过GT4Py stencil DSL演示。SpaDA使具有复杂并行模式的操作能够以14.09倍更少的代码行表达——包括流水线集体操作、多维Stencil和密集线性代数——在单个设备上730,000个处理元素上达到超过260 TFlop/s。
英文摘要:
Spatial dataflow architectures like the Cerebras Wafer-Scale Engine deliver exceptional performance in AI and scientific computing by distributing scratchpad memory across hundreds of thousands of processing elements (PEs). Yet programming these architectures remains difficult: with no shared memory, data movement requires explicit configuration, and asynchronous task management introduces substantial complexity. We present SpaDA, a programming language that offers precise control over data placement, dataflow patterns, and asynchronous operations while abstracting low-level architectural details. We design and implement a compiler targeting Cerebras CSL through multi-level lowering and unique optimization passes. SpaDA functions as a high-level programming interface and an intermediate representation for domain-specific languages (DSLs), demonstrated here with the GT4Py stencil DSL. SpaDA enables concise expression of operations with complex parallel patterns -- including pipelined collective operations, multi-dimensional stencils, and dense linear algebra -- in 14.09x fewer lines than CSL, achieving over 260 TFlop/s across 730,000 PEs on a single device.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。