arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14404cs.LG
收藏

预测误差概率以结合量化与早退:QuEE

Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE

McGill University(麦吉尔大学);ILLS(学习系统国际实验室);MILA(蒙特利尔学习算法研究所);Technical University of Munich(慕尼黑工业大学);École de technologie supérieure(高等技术学院)

Florence Regol, Joud Chataoui, Bertrand Charpentier, Mark Coates, Pablo Piantanida, Stephan Gunnemann

AI总结:

针对机器学习模型推理计算量大的问题,提出结合量化与早退的通用动态网络QuEE,通过有原则的公式化方法解决软早退决策问题,在4个分类数据集上验证了有效性并探究了成功条件。

摘要:

机器学习模型能够解决复杂任务,但推理过程中通常需要大量计算资源。这催生了多种训练后计算缩减方法,它们以不同方式解决该问题,例如降低权重精度与算术运算精度的quantization(量化),以及根据样本调整计算量的dynamic networks(动态网络)。本研究提出一种更通用的动态网络QuEE,可结合量化与早退动态网络两种技术。我们的算法可被视为一种软早退或依赖输入的压缩形式。我们引入了以缩减计算量继续运行的可能性,而非在退出或继续之间做二元决策。这让传统的早退问题变得更复杂,我们通过一个有理论依据的公式化方法解决了该问题。我们方法的关键在于准确预测通过进一步计算可获得的潜在精度提升。我们通过实证评估验证了方法的有效性,并在4个分类数据集上探究了其成功的条件。

英文摘要:

Machine learning models can solve complex tasks but often require significant computational resources during inference. This has led to the development of various post-training computation reduction methods that tackle this issue in different ways, such as quantization which reduces the precision of weights and arithmetic operations, and dynamic networks which adapt computation to the sample at hand. In this work, we propose a more general dynamic network that can combine both quantization and early exit dynamic network: QuEE. Our algorithm can be seen as a form of soft early exiting or input-dependent compression. Rather than a binary decision between exiting or continuing, we introduce the possibility of continuing with reduced computation. This complicates the traditionally considered early exiting problem, which we solve through a principled formulation. The crucial factor of our approach is accurate prediction of the potential accuracy improvement achievable through further computation. We demonstrate the effectiveness of our method through empirical evaluation, as well as exploring the conditions for its success on 4 classification datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑