arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.05339cs.CV
收藏

JAQ:通过硬件-软件协同探索进行联合高效架构设计与低位量化

JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration

Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);BNRist, Tsinghua University(清华大学BNRist)

Mingzi Wang, Yuan Meng, Chen Tang, Weixiang Zhang, Yijian Qin, Yang Yao, Yingxin Li, Tongtong Feng, Xin Wang, Xun Guan, Zhi Wang, Wenwu Zhu

AI总结:

提出JAQ软硬件协同探索框架,通过CSQ降低低位量化训练内存开销,并利用BatchTile加速编译器映射搜索,在ImageNet上提升约7% Top-1准确率,迭代硬件搜索仅0.15秒。

摘要:

神经网络架构、量化精度和硬件加速器的协同设计为实现性能与效率之间的最佳平衡提供了一种有前景的方法,尤其是对于资源受限边缘设备上的模型部署。在本工作中,我们提出了JAQ框架,该框架联合优化这三个关键维度。然而,在这三个维度的庞大搜索空间中有效自动化设计过程带来了重大挑战,尤其是在追求极低比特量化时。具体而言,主要挑战包括:(1)软件侧的内存开销:低精度量化感知训练可能因存储用于反向传播的大型中间特征和潜在权重而导致显著的内存使用,可能造成内存耗尽。(2)硬件侧的搜索耗时:硬件参数的离散特性以及编译器优化与各个算子之间复杂的相互作用,使加速器搜索非常耗时。为解决这些问题,JAQ通过通道级稀疏量化(CSQ)方案缓解内存开销,在优化期间有选择地对模型最敏感的组件应用量化。此外,JAQ设计了BatchTile,它采用硬件生成网络对所有可能的分块模式进行编码,从而加快对最佳编译器映射策略的搜索。大量实验证明了JAQ的有效性,与先前方法相比,其在ImageNet上实现了约7%的Top-1准确率提升,并将每次迭代的硬件搜索时间缩短至0.15秒。

英文摘要:

The co-design of neural network architectures, quantization precisions, and hardware accelerators offers a promising approach to achieving an optimal balance between performance and efficiency, particularly for model deployment on resource-constrained edge devices. In this work, we propose the JAQ Framework, which jointly optimizes the three critical dimensions. However, effectively automating the design process across the vast search space of those three dimensions poses significant challenges, especially when pursuing extremely low-bit quantization. Specifical, the primary challenges include: (1) Memory overhead in software-side: Low-precision quantization-aware training can lead to significant memory usage due to storing large intermediate features and latent weights for back-propagation, potentially causing memory exhaustion. (2) Search time-consuming in hardware-side: The discrete nature of hardware parameters and the complex interplay between compiler optimizations and individual operators make the accelerator search time-consuming. To address these issues, JAQ mitigates the memory overhead through a channel-wise sparse quantization (CSQ) scheme, selectively applying quantization to the most sensitive components of the model during optimization. Additionally, JAQ designs BatchTile, which employs a hardware generation network to encode all possible tiling modes, thereby speeding up the search for the optimal compiler mapping strategy. Extensive experiments demonstrate the effectiveness of JAQ, achieving approximately 7% higher Top-1 accuracy on ImageNet compared to previous methods and reducing the hardware search time per iteration to 0.15 seconds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑