1-bit AI Infra:第1.1部分,在CPU上实现快速且无损的BitNet b1.58推理
1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
AI总结:
本文提出bitnet.cpp软件栈,通过专用内核在CPU上支持三值BitNet b1.58的快速无损推理,并在x86与ARM CPU上取得显著加速。
摘要:
1-bit大语言模型(LLM)的最新进展(例如BitNet和BitNet b1.58)为提升LLM在速度和能耗方面的效率提供了一种颇具前景的方法。这些进展还使得在广泛设备上进行本地LLM部署成为可能。在本工作中,我们推出bitnet.cpp,这是一个旨在充分释放1-bit LLM潜力的定制软件栈。具体而言,我们开发了一组内核,以支持在CPU上对三值BitNet b1.58 LLM进行快速且无损的推理。大量实验表明,在各种模型规模下,bitnet.cpp在x86 CPU上实现了2.37倍至6.17倍的显著加速,在ARM CPU上实现了1.37倍至5.07倍的显著加速。代码可在https://github.com/microsoft/BitNet获取。
英文摘要:
Recent advances in 1-bit Large Language Models (LLMs), such as BitNet and BitNet b1.58, present a promising approach to enhancing the efficiency of LLMs in terms of speed and energy consumption. These developments also enable local LLM deployment across a broad range of devices. In this work, we introduce bitnet.cpp, a tailored software stack designed to unlock the full potential of 1-bit LLMs. Specifically, we develop a set of kernels to support fast and lossless inference of ternary BitNet b1.58 LLMs on CPUs. Extensive experiments demonstrate that bitnet.cpp achieves significant speedups, ranging from 2.37x to 6.17x on x86 CPUs and from 1.37x to 5.07x on ARM CPUs, across various model sizes. The code is available at https://github.com/microsoft/BitNet.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。