arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.01928cs.CV
收藏

DSConv:高效卷积算子

DSConv: Efficient Convolution Operator

University of Oxford(牛津大学);Intel Corporation(英特尔公司)

Marcelo Gennari, Roger Fawcett, Victor Adrian Prisacariu

摘要:

量化是提高卷积神经网络(CNN)速度和降低内存使用率的一种流行方法。当标记的训练数据可用时,网络权重和激活已成功量化到1位。当标记的训练数据不可用时,例如,当量化预训练模型时,不能说同样的情况,其中当前方法最多显示在8位量化时没有准确性损失。我们引入了DSConv,这是一种灵活的量化卷积运算符,它用成本低得多的整数运算替代了单精度运算,同时保持了内核权重和输出的概率分布。我们在最流行的神经网络架构ResNet、DenseNet、GoogLeNet、AlexNet和VGG-Net上测试我们的模型作为标准卷积的即插即用替代品,并展示了最先进的结果,准确度损失不到1%,无需重新训练,仅使用4位量化。我们还展示了如何使用未标记数据的基于蒸馏的自适应阶段可以进一步改善结果。

英文摘要:

Quantization is a popular way of increasing the speed and lowering the memory usage of Convolution Neural Networks (CNNs). When labelled training data is available, network weights and activations have successfully been quantized down to 1-bit. The same cannot be said about the scenario when labelled training data is not available, e.g. when quantizing a pre-trained model, where current approaches show, at best, no loss of accuracy at 8-bit quantizations. We introduce DSConv, a flexible quantized convolution operator that replaces single-precision operations with their far less expensive integer counterparts, while maintaining the probability distributions over both the kernel weights and the outputs. We test our model as a plug-and-play replacement for standard convolution on most popular neural network architectures, ResNet, DenseNet, GoogLeNet, AlexNet and VGG-Net and demonstrate state-of-the-art results, with less than 1% loss of accuracy, without retraining, using only 4-bit quantization. We also show how a distillation-based adaptation stage with unlabelled data can improve results even further.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑