CacheFlex：支持可扩展向量的软件直接管理高级缓存访问
CacheFlex: Direct Software-Managed Access to Higher-Level Cache for Scalable Vector Support
AI总结：
CacheFlex是ISA与微架构协同设计方案，通过软件直接访问私有L2缓存，在向量长度扩展时，可提升AI工作负载的加速比并降低内存子系统能耗，且面积开销仅2.02%。
摘要：
更宽的向量/SIMD通道可成倍提升计算吞吐量，但缓存改进仍面临挑战。我们发现L1数据缓存容量是影响性能和能效的最关键参数；L1容量受流水线耦合和延迟要求限制，而私有L2缓存的容量是其一个数量级以上，但软件无法直接访问。本研究提出CacheFlex，一种ISA（指令集架构）与微架构协同设计方案，通过允许运行时将部分L2转换为暂存存储器（SPM），实现软件对私有L2 SRAM的直接、软件管理访问。通过轻量ISA扩展，软件可显式控制L2内的数据放置和生命周期。CacheFlex集成到乱序流水线中，具备跨域内存一致性和操作系统上下文切换支持，在12nm工艺后布局实现中，仅产生2.02%的面积开销。我们基于Armv8-A SVE ISA实现CacheFlex原型，在gem5中建模，并使用优化的通用矩阵乘法（GEMM）核（含多种模型衍生形状）、基于FlashAttention的融合注意力机制，以及LLaMA-3.2-1B和BERT-Base在预填充阶段的全层工作负载进行评估。当向量长度（VL）从VL=4（512位）扩展到VL=16（2048位）时，CacheFlex的核级几何平均加速比和内存子系统能耗降低幅度从1.12倍、26%提升至1.44倍、40%，缩小了单指令数据需求与内存性能间不断扩大的差距。在结合CacheFlex与FlashAttention的完整Transformer层上，该设计实现最高1.78倍加速比和45%内存子系统能耗节省（BERT-Base，序列长度T=4096，VL=16；LLaMA-3.2-1B：1.64倍和38%）。CacheFlex与硬件预取技术互补，其效益随计算密集型AI核的向量宽度和峰值计算吞吐量提升而增长。
英文摘要：
Wider vector/SIMD lanes multiply compute throughput, but cache improvements remain challenging. We identify L1 data cache capacity as the most impactful parameter for performance and efficiency. L1 capacity is constrained by pipeline coupling and latency requirements; private L2 offers an order of magnitude more capacity but is inaccessible from software. This work proposes CacheFlex, an ISA and microarchitecture co-design that enables software-managed, direct access to private L2 SRAM by allowing runtime conversion of parts of L2 into a scratchpad memory (SPM). Through a lightweight ISA extension, software explicitly controls data placement and lifetime within the L2. CacheFlex integrates into an out-of-order pipeline with cross-domain memory consistency and OS context-switch support, with merely 2.02% area overhead relative to the L2 in a post-layout 12 nm implementation. We prototype CacheFlex based on the Armv8-A SVE ISA, model it in gem5, and evaluate it using optimized general matrix multiplication (GEMM) kernels with diverse model-derived shapes, FlashAttention-based fused attention, and full-layer workloads from LLaMA-3.2-1B and BERT-Base in the prefill phase. As the vector length (VL) scales from VL = 4 (512-bit) to VL = 16 (2048-bit), CacheFlex's kernel-level geometric-mean speedup and memory-subsystem energy reduction grow from 1.12x and 26% to 1.44x and 40%, respectively, bridging the widening gap between per-instruction data demand and memory performance. On full transformer layers combining CacheFlex with FlashAttention, the design delivers up to 1.78x speedup and 45% memory-subsystem energy savings (BERT-Base, sequence length T=4096, VL = 16; LLaMA-3.2-1B: 1.64x and 38%). CacheFlex is complementary to hardware prefetching, and its benefit grows as vector width and peak compute throughput increase for compute-intensive AI kernels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
