OrbitQuant: 图像和视频扩散Transformer的数据无关量化
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
AI总结：
提出数据无关量化方法OrbitQuant，通过归一化旋转基和Lloyd-Max码本，无需校准数据即可对扩散Transformer的权重和激活进行低比特量化，在多个模型和模态上达到最优。
摘要：
扩散Transformer（DiT）在图像和视频生成中达到最先进水平，但其多步采样和不断增长的参数数量使得推理成本高昂。训练后量化（PTQ）是自然的补救措施，然而DiT的激活在不同时间步、提示和引导分支之间发生变化，迫使先前的方法为每个新的检查点或模态重新拟合校准数据。我们提出OrbitQuant，一种数据无关的权重激活量化器，通过在归一化旋转基中进行量化来绕过范围估计。在这个基中，随机排列块哈达玛（RPBH）旋转将每个坐标集中在一个固定的、已知的边缘分布周围，而与输入无关，因此单个Lloyd-Max码本即可服务于给定输入维度的所有时间步、提示和层。我们将相同的量化器离线扩展到权重行，将旋转吸收到权重中，使得在每个线性层内部旋转抵消，运行时仅保留激活上的前向旋转。同样的方法从图像迁移到视频，无需每模态调优。在FLUX.1、Z-Image-Turbo、Wan 2.1和CogVideoX上，它在多个低比特设置下达到了PTQ的最先进水平。它还将图像扩散Transformer的PTQ推进到W2A4，并具有可用的生成质量。
英文摘要：
Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation concentrates each coordinate around one fixed, known marginal regardless of the input, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video with no per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image diffusion transformers to W2A4 with usable generation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
