Bielik-Q2-Sharp:极端2位量化方法在波兰11B语言模型上的比较研究
Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model
AI总结:
Bielik-Q2-Sharp通过比较六种极端2位量化方法,评估其在波兰11B语言模型上的性能,发现QuIP# E8P12在高阶推理保留方面表现更优,同时在规模和效率上具有竞争力。
摘要:
我们介绍了Bielik-Q2-Sharp,这是首个系统性的学术评估,用于评估极端2位量化应用于波兰大语言模型。使用Bielik-11B-v2.3-Instruct(11B参数,Mistral架构)作为基础模型,我们比较了六种最先进的后训练量化方法——QuIP#、SpinQuant+GPTQ、ButterflyQuant、QTIP、VPTQ和AQLM——这些方法均在波兰语语料库(CulturaX-PL)上进行校准,并共享Hessian矩阵。我们的最佳变体(QuIP# E8P12)在22个波兰基准测试中达到71.92%,与IQ2_XXS基线的72.07%相差在统计噪声范围内,仅在规模上略大(3.26 GB vs. ~2.6 GB)。在eq_bench上,我们的方法得分为47.14,而基线为43.53(+3.6个百分点),表明对高阶推理的保留更优。QTIP在每比特效率上表现最佳(79.4% MC acc_norm在约2.4 bpw,3.27 GB),与VPTQ的质量相当,但规模小35%。我们还记录了一种MC生成脱节现象,其中基于旋转的方法在对数似然质量上得以保持,但在自回归生成中却惨遭失败。整个项目由单一独立研究者在云GPU(vast.ai)上进行,预算为285美元。所有模型、Hessian矩阵和评估日志均已公开。
英文摘要:
We present Bielik-Q2-Sharp, the first systematic academic evaluation of extreme 2-bit quantization applied to a Polish large language model. Using Bielik-11B-v2.3-Instruct (11B parameters, Mistral architecture) as our base model, we compare six state-of-the-art post-training quantization methods -- QuIP#, SpinQuant+GPTQ, ButterflyQuant, QTIP, VPTQ, and AQLM -- all calibrated on a Polish-language corpus (CulturaX-PL) with shared Hessian matrices. Our best variant (QuIP# E8P12) achieves 71.92% across 22 Polish benchmarks versus 72.07% for the IQ2_XXS baseline -- within statistical noise, at a modest size premium (3.26 GB vs. ~2.6 GB). On eq_bench, our method scores 47.14 versus 43.53 (+3.6pp), suggesting superior preservation of higher-order reasoning. QTIP achieves the best per-bit efficiency (79.4% MC acc_norm at ~2.4 bpw, 3.27 GB), matching VPTQ's quality at 35% smaller size. We additionally document a MC-generation dissociation phenomenon where rotation-based methods preserve log-likelihood quality but fail catastrophically at autoregressive generation. The entire project was conducted by a single independent researcher on cloud GPUs (vast.ai) within a $285 budget. All models, Hessians, and evaluation logs are publicly available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。