从扫描到缝合：交错跨块后训练量化
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
浏览论文内容
中文总结 AI 辅助
该研究针对大语言模型的分块后训练量化，提出交错跨块量化方法，通过重新处理块边界降低量化困惑度，提升了三比特及GPTQ等量化方案的性能。
中文摘要 AI 辅助
将大语言模型压缩至2比特或更少的水平，通过分块后训练量化（PTQ）正变得越来越可行；跨块变体在移动窗口内重构相邻的Transformer块。本文研究的固定双块设置中，匹配的顺序基线方法将该窗口在网络中移动一次，因此扫描早期引入的误差不会被重新处理。我们提出交错跨块量化（ICBQ），这是一种调度修改方法，会重新处理连续块之间的边界对。每个缝合对会被细化两次：第一次在一个块的末尾，第二次在下一个块的开头。该方法保留了局部双块目标，并复用现有分块PTQ流水线的校准输入。在给定的局部收缩和平滑假设下，我们推导出一种深度方向的上界比较，其中缝合处的重新处理会使传播项成倍增加，而残差则与深度无关地保持有界。在报告的实验中，相对于匹配的顺序CBQ基线，ICBQ降低了三比特量化的困惑度，在基线出现严重退化的配置中产生有限的困惑度，还可与3比特和2比特GPTQ配合使用。
英文摘要
Compressing large language models to two bits or fewer is increasingly feasible through block-wise post-training quantization; cross-block variants reconstruct neighboring Transformer blocks within a moving window. In the fixed two-block setting studied here, the matched sequential baseline moves this window through the network once, so errors introduced early in the sweep are not revisited. We propose Interleaved Cross-Block Quantization (ICBQ), a scheduling modification that revisits the boundary pair between consecutive chunks. Each seam pair is refined twice: first at the end of one chunk and again at the start of the next. The method retains the local two-block objective and reuses the calibration inputs of existing block-wise PTQ pipelines. Under stated local contraction and smoothness assumptions, we derive a depth-wise upper-bound comparison in which seam revisits multiply the propagated term while the residual remains bounded independently of depth. In the reported experiments, ICBQ reduces ternary-quantization perplexity relative to the matched Sequential CBQ baseline, yields finite perplexity in configurations where the baseline has severe degradation, and can also be used with 3-bit and 2-bit GPTQ.
发表机构
- Fujitsu Limited（富士通公司）
- RIKEN Center for AIP（理化学研究所先进智能项目中心）
- Tokai University（东海大学）
机构由 AI 辅助整理，请以论文原文为准。
