arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.17367cs.ARcs.LG
收藏

通过低秩适配器实现高效的类比存内计算变压器适应

Efficient transformer adaptation for analog in-memory computing via low-rank adapters

Department of Engineering, King's College London, United Kingdom(伦敦大学工程学院,英国)

Chen Li, Elena Ferro, Corey Lammie, Manuel Le Gallo, Irem Boybat, Bipin Rajendran

AI总结:

本文提出AHWA-LoRA方法,通过引入轻量级LoRA模块,解决类比存内计算中模型适应和任务调整的挑战,验证了其在SQuAD和GLUE基准上的有效性,并展示了在实际部署中的高效性。

摘要:

类比存内计算(AIMC)提供了一种解决冯·诺依曼瓶颈的有希望的解决方案。然而,将变压器模型部署到AIMC上仍然具有挑战性,因为它们本质上需要在多样化任务中保持灵活性和适应性。为了充分实现AIMC的优势,静态向量-矩阵乘法的权重必须以权重静止的方式映射和编程到类比设备上。这提出了两个挑战:(i)传统类比硬件感知(AHWA)训练需要重新训练整个模型;(ii)重新编程类比设备既耗时又耗能。为了解决这些问题,我们提出了类比硬件感知低秩适配(AHWA-LoRA)训练,这是一种新的方法,用于高效地将变压器适应到AIMC硬件上。AHWA-LoRA训练将类比权重保持为元权重,并引入轻量级外部LoRA模块,用于硬件和任务适应。我们验证了AHWA-LoRA训练在SQuAD v1.1和GLUE基准上的有效性,展示了其扩展性到更大模型的能力,并展示了其在指令调优和强化学习中的有效性。我们进一步评估了一个实际部署场景,该场景平衡了AIMC瓷砖延迟与数字LoRA处理,使用优化的流水线策略,通过基于RISC-V的可编程多核加速器。这种混合架构仅在每层增加4%的开销即可实现高效的变压器推断,相比完全AIMC实现而言。

英文摘要:

Analog In-Memory Computing (AIMC) offers a promising solution to the von Neumann bottleneck. However, deploying transformer models on AIMC remains challenging due to their inherent need for flexibility and adaptability across diverse tasks. For the benefits of AIMC to be fully realized, weights of static vector-matrix multiplications must be mapped and programmed to analog devices in a weight-stationary manner. This poses two challenges for adapting a base network to hardware and downstream tasks: (i) conventional analog hardware-aware (AHWA) training requires retraining the entire model, and (ii) reprogramming analog devices is both time- and energy-intensive. To address these issues, we propose Analog Hardware-Aware Low-Rank Adaptation (AHWA-LoRA) training, a novel approach for efficiently adapting transformers to AIMC hardware. AHWA-LoRA training keeps the analog weights fixed as meta-weights and introduces lightweight external LoRA modules for both hardware and task adaptation. We validate AHWA-LoRA training on SQuAD v1.1 and the GLUE benchmark, demonstrate its scalability to larger models, and show its effectiveness in instruction tuning and reinforcement learning. We further evaluate a practical deployment scenario that balances AIMC tile latency with digital LoRA processing using optimized pipeline strategies, with RISC-V-based programmable multi-core accelerators. This hybrid architecture achieves efficient transformer inference with only a 4% per-layer overhead compared to a fully AIMC implementation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑