ScaleBITS: 可扩展的位宽搜索用于硬件对齐的混合精度大语言模型
ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
AI总结：
ScaleBITS通过硬件对齐的块级权重分区和约束优化方法，实现高效混合精度量化，提升低比特下的模型性能。
摘要：
后训练权重量化对于减少大型语言模型（LLMs）的内存和推理成本至关重要，但将平均精度降至4位以下仍然具有挑战性，因为权重敏感性高度非均匀且缺乏系统性的精度分配原则。现有解决方案使用不规则的细粒度混合精度，具有较高的运行时开销，或依赖启发式或高度受限的精度分配策略。在本工作中，我们提出了ScaleBITS，一种混合精度量化框架，能够在内存预算内实现自动化、细粒度的位宽分配，同时保持硬件效率。受新的敏感性分析引导，我们引入了一种硬件对齐的块级权重分区方案，通过双向通道重排实现。我们将全局位宽分配建模为一个约束优化问题，并开发了一种可扩展的贪心算法近似方法，实现端到端的系统性分配。实验表明，ScaleBITS在均匀精度量化方面显著改进（最高+36%），并在超低比特 regime 中优于最先进的敏感性aware基线（最高+13%），且不增加运行时开销。
英文摘要：
Post-training weight quantization is crucial for reducing the memory and inference cost of large language models (LLMs), yet pushing the average precision below 4 bits remains challenging due to highly non-uniform weight sensitivity and the lack of principled precision allocation. Existing solutions use irregular fine-grained mixed-precision with high runtime overhead or rely on heuristics or highly constrained precision allocation strategies. In this work, we propose ScaleBITS, a mixed-precision quantization framework that enables automated, fine-grained bitwidth allocation under a memory budget while preserving hardware efficiency. Guided by a new sensitivity analysis, we introduce a hardware-aligned, block-wise weight partitioning scheme, powered by bi-directional channel reordering. We formulate global bitwidth allocation as a constrained optimization problem and develop a scalable approximation to the greedy algorithm, enabling end-to-end principled allocation. Experiments show that ScaleBITS significantly improves over uniform-precision quantization (up to +36%) and outperforms state-of-the-art sensitivity-aware baselines (up to +13%) in ultra-low-bit regime, without adding runtime overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
