AI 中文总结
该研究针对模拟存算非理想性下扩散变换器的CFG残差问题，提出采样器侧引导尺度重新校准方法，可大幅消除CIM导致的FID差距，提升生成质量。
AI 中文摘要
扩散变换器（DiTs）因采样时需反复评估以线性操作为主的大型去噪器，产生了高内存流量与能量成本。模拟存算（CIM）可通过在存储权重的内存阵列内执行线性操作来缓解这些成本，但CIM非理想性会扰动有效权重，误差会沿依赖状态的去噪轨迹累积，其与无分类器引导（CFG）的相互作用仍未得到充分探索。本文中，我们表征了模拟CIM非理想性对DiT采样的影响：尽管条件预测与无条件预测各自可保持与干净版本相近，但它们的差值（即CFG残差）会不成比例地衰减与旋转。将该残差识别为可控失效通道，我们提出一种无需重新训练、仅在采样器侧进行的重新校准方法，该方法仅针对给定CIM条件调整CFG尺度。轨迹级分析显示，适度的重新校准可增强失真残差中保留的面向目标的分量，使其更早锁定到与提示一致的语义区域；反之，过度引导会放大全部噪声残差并降低质量，进而产生有限的、依赖噪声的最优值。在PixArt-Sigma、PixArt-alpha和DiT-XL/2上开展的大量实验表明，最优引导尺度随CIM噪声增大而升高；在每个条件下使用30000个样本时，引导重新校准可在模拟CIM映射上持续恢复生成质量，在CIM噪声水平为0.20时，至少消除了87%的CIM诱导FID差距，使PixArt-Sigma的FID从59.22降至20.49，PixArt-alpha从72.37降至21.12，DiT-XL/2从20.89降至6.62。
英文摘要
Diffusion Transformers (DiTs) incur high memory traffic and energy costs because sampling repeatedly evaluates large denoisers dominated by linear operations. Analog compute-in-memory (CIM) can alleviate these costs by executing linear operations within weight-storing memory arrays. However, CIM nonidealities perturb effective weights, with errors accumulating along the state-dependent denoising trajectory; their interaction with classifier-free guidance (CFG) remains underexplored. In this paper, we characterize the impact of analog CIM nonidealities on DiT sampling. Although conditional and unconditional predictions can each remain close to their clean counterparts, their difference (the CFG residual) is disproportionately attenuated and rotated. Identifying this residual as a controllable failure channel, we propose a retraining-free, sampler-side recalibration that adjusts only the CFG scale for a given CIM condition. Trajectory-level analysis shows that moderate recalibration strengthens the target-oriented component preserved in the distorted residual, enabling earlier commitment to a prompt-consistent semantic region. In contrast, excessive guidance amplifies the full noisy residual and degrades quality, resulting in a finite, noise-dependent optimum. Extensive experiments on PixArt-Sigma, PixArt-alpha, and DiT-XL/2 show that the optimal guidance scale increases with CIM noise. Using 30,000 samples per condition, guidance recalibration consistently restores generation quality across simulated CIM mappings, closing at least 87% of the CIM-induced FID gap at a CIM noise level of 0.20. It reduces FID from 59.22 to 20.49 on PixArt-Sigma, 72.37 to 21.12 on PixArt-alpha, and 20.89 to 6.62 on DiT-XL/2.
Comments9 pages, 8 figures, 3 tables
