arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.05034cs.ARcs.DC
收藏

面向高效机器学习训练的存内计算即时量化方法

Just-in-time Quantization with Processing-In-Memory for Efficient ML Training

Mohamed Assem Ibrahim, Shaizeen Aga, Ada Li, Suchita Pati, Mahzabeen Islam

AI总结:

针对机器学习训练中多精度权重副本占用大量内存的问题,提出基于存内计算的即时量化方案,仅存高精度权重并按需实时生成低精度版本,以2.4%以内的吞吐量损失换取最高24%的内存容量节省,提升训练效率。

摘要:

数据格式创新一直是机器学习(ML)规模扩展的关键,而规模扩展又推动了突破性的ML能力发展。然而,即便有低精度格式可用,训练期间模型权重通常仍需同时存储高精度和低精度两个版本。此外,随着新兴的定向数据格式(如MX9、MX6等)出现,可能需要多份低精度权重副本。为降低权重的内存容量需求,我们探索了即时量化(JIT-Q)方案:仅在内存中存储高精度权重,仅在需要时才生成低精度权重。为高效执行JIT-Q,本研究评估了新兴的存内计算(PIM)技术以执行量化操作。借助PIM,我们可将量化任务卸载到内存内计算单元,使量化操作无需产生高昂的数据移动开销,同时可与加速器计算并行执行。我们提出的PIM卸载量化方案能够跟上GPU的计算速度,可节省高达24%的内存容量,而吞吐量损失仅最高2.4%。上述内存容量节省可带来诸多益处,例如在同一系统中容纳更大的模型、降低模型并行度需求,以及提升整体ML训练效率。

英文摘要:

Data format innovations have been critical for machine learning (ML) scaling, which in turn fuels ground-breaking ML capabilities. However, even in the presence of low-precision formats, model weights are often stored in both high-precision and low-precision during training. Furthermore, with emerging directional data formats (e.g., MX9, MX6, etc.) multiple low-precision weight copies can be required. To lower memory capacity needs of weights, we explore just-in-time quantization (JIT-Q) where we only store high-precision weights in memory and generate low-precision weights only when needed. To perform JIT-Q efficiently, in this work, we evaluate emerging processing-in-memory (PIM) technology to execute quantization. With PIM, we can offload quantization to in-memory compute units enabling quantization to be performed without incurring costly data movement while allowing quantization to be concurrent with accelerator computation. Our proposed PIM-offloaded quantization keeps up with GPU compute and delivers considerable capacity savings (up to 24\%) at marginal throughput loss (up to 2.4\%). Said memory capacity savings can unlock several benefits such as fitting larger model in the same system, reducing model parallelism requirement, and improving overall ML training efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑