arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.00313eess.AScs.SD
收藏

面向音频扩散Transformer的训练后量化

Post-Training Quantization for Audio Diffusion Transformers

Courant Institute of Mathematical Sciences, New York University, NY, USA(Courant数学科学研究所,纽约大学);MARL, New York University, NY, USA(MARL,纽约大学);IDM, New York University, NY, USA(IDM,纽约大学)

Tanmay Khandelwal, Magdalena Fuentes

AI总结:

针对音频扩散Transformer部署时计算与存储成本高的问题,本文评估训练后量化技术,提出两种优化方法,实验证明低精度量化可降内存达79%且保持生成质量。

摘要:

扩散Transformer(Diffusion Transformers, DiTs)可实现高质量音频合成,但通常计算密集且需要大量存储空间,这限制了其实际部署。本文对音频DiT的训练后量化(post-training quantization, PTQ)技术开展了综合评估,分析了静态与动态量化方案之间的权衡关系。我们探索了两种实用扩展方法:(1)一种去噪时间步感知平滑方法,它针对每个输入通道和时间步调整量化尺度,以缓解激活离群值问题;(2)一种基于奇异值分解(singular value decomposition, SVD)的轻量级低秩适配器(low-rank adapter, LoRA)分支,用于补偿残余权重误差。我们使用Stable Audio Open数据集,在客观指标和人类感知评分上对W8A8和W4A8配置进行了基准测试。结果表明,动态量化即使在较低精度下也能保持保真度,而静态方法延迟更低,性能仍具竞争力。总体而言,研究发现低精度DiT可在保持高保真生成效果的同时,将内存使用量降低多达79%。

英文摘要:

Diffusion Transformers (DiTs) enable high-quality audio synthesis but are often computationally intensive and require substantial storage, which limits their practical deployment. In this paper, we present a comprehensive evaluation of post-training quantization (PTQ) techniques for audio DiTs, analyzing the trade-offs between static and dynamic quantization schemes. We explore two practical extensions (1) a denoising-timestep-aware smoothing method that adapts quantization scales per-input-channel and timestep to mitigate activation outliers, and (2) a lightweight low-rank adapter (LoRA)-based branch derived from singular value decomposition (SVD) to compensate for residual weight errors. Using Stable Audio Open we benchmark W8A8 and W4A8 configurations across objective metrics and human perceptual ratings. Our results show that dynamic quantization preserves fidelity even at lower precision, while static methods remain competitive with lower latency. Overall, our findings show that low-precision DiTs can retain high-fidelity generation while reducing memory usage by up to 79%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑