ASSERT：面向模拟存算一体硬件的鲁棒扩散模型的自适应随机采样
ASSERT: Adaptive Stochastic Sampling for Robust Diffusion Models on Analog Compute-in-Memory Hardware
浏览论文内容
中文总结 AI 辅助
针对模拟存算一体硬件中扩散模型受空间内存变异影响的问题，提出无需训练的自适应随机采样器ASSERT，通过调整不同去噪阶段的随机性，显著降低FID且不改变模型参数。
中文摘要 AI 辅助
扩散模型在图像生成质量上表现出色，但存在迭代去噪成本高的问题。模拟存算一体（CIM）可加速矩阵-向量乘法，然而空间内存变异会扰动权重并在采样过程中累积。与传统神经网络不同，扩散模型对硬件噪声的时间敏感性尚未得到充分探索。我们使用针对多个物理CIM芯片测量数据校准和验证的噪声模型，研究扩散推理。结果表明，早期高噪声去噪阶段比最终细化阶段脆弱得多，一阶轨迹分析将此行为归因于固定硬件映射诱导的相关预测误差的反复传播。基于此观察，我们提出ASSERT，一种无需训练的采样器，其在早期使用更高的随机性，并平滑过渡到确定性去噪。注入的随机性改变了后续激活轨迹，从而减少了其与持续空间误差的对齐。在评估设置中，ASSERT在高分辨率数据集上比确定性DDIM实现了高达2.58倍的FID降低，在CIFAR-10步数研究中实现了7.68倍的FID降低，且无需更改模型参数或网络评估次数。
英文摘要
Diffusion models achieve strong image generation quality but incur high iterative denoising costs. Analog compute-in-memory (CIM) can accelerate matrix-vector multiplications, yet spatial memory variations perturb weights and accumulate during sampling. Unlike conventional neural networks, diffusion models' temporal sensitivity to hardware noise remains underexplored. We investigate diffusion inference using a noise model calibrated and validated against measurements collected from multiple physical CIM chips. Our results show that the early, high-noise denoising stage is substantially more vulnerable than the final refinement stage. A first-order trajectory analysis attributes this behavior to the repeated propagation of correlated prediction errors induced by a fixed hardware mapping. Based on this observation, we propose ASSERT, a training-free sampler that uses higher stochasticity early and smoothly transitions to deterministic denoising. The injected stochasticity changes subsequent activation trajectories and thereby reduces their alignment with persistent spatial errors. Across the evaluated settings, ASSERT achieves up to 2.58$\times$ lower FID than deterministic DDIM on high-resolution datasets and 7.68$\times$ lower FID in the CIFAR-10 step-count study, without changing model parameters or the number of network evaluations.
发表机构
- School of Integrated Circuit Science and Engineering, Beihang University（北京航空航天大学集成电路科学与工程学院）
- Zhicun Research Lab（知存研究院）
- School of Integrated Circuits, Peking University（北京大学集成电路学院）
- Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电机电子工程学系）
机构由 AI 辅助整理，请以论文原文为准。
