arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1908.06314cs.CV
收藏

贝叶斯优化的1位CNN

Bayesian Optimized 1-Bit CNNs

Beihang University(北京航空航天大学);Baidu Research(百度研究院);National Engineering Laboratory for Deep Learning Technology and Application(深度学习技术及应用国家工程实验室);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Xiamen University(厦门大学);Peng Cheng Lab(鹏城实验室)

Jiaxin Gu, Junhe Zhao, Xiaolong Jiang, Baochang Zhang, Jianzhuang Liu, Guodong Guo, Rongrong Ji

摘要:

深度卷积神经网络(DCNN)通过建立各种破纪录的模型,主导了计算机视觉的最新发展。然而,在资源有限的环境中实现强大的DCNN仍然是一个巨大的挑战,例如在嵌入式设备和智能手机上。研究人员已经意识到,1位CNN可能是解决这个问题的一个可行的解决方案;然而,与全精度DCNN相比,它们的性能较差。在本文中,我们提出了一种新的方法,称为贝叶斯优化的1位CNN(表示为BONN),利用贝叶斯学习(一种针对困难问题的成熟策略)来显着提高极端1位CNN的性能。我们将全精度内核和特征的先验分布纳入贝叶斯框架,以端到端的方式构建1位CNN,这在以前的任何相关方法中都没有考虑过。贝叶斯损失的实现与理论支持,同时优化网络在连续和离散空间,聚合不同的损失联合,以提高模型的能力。在ImageNet和CIFAR数据集上进行的大量实验表明,与最先进的1位CNN相比,BONN实现了最佳的分类性能。

英文摘要:

Deep convolutional neural networks (DCNNs) have dominated the recent developments in computer vision through making various record-breaking models. However, it is still a great challenge to achieve powerful DCNNs in resource-limited environments, such as on embedded devices and smart phones. Researchers have realized that 1-bit CNNs can be one feasible solution to resolve the issue; however, they are baffled by the inferior performance compared to the full-precision DCNNs. In this paper, we propose a novel approach, called Bayesian optimized 1-bit CNNs (denoted as BONNs), taking the advantage of Bayesian learning, a well-established strategy for hard problems, to significantly improve the performance of extreme 1-bit CNNs. We incorporate the prior distributions of full-precision kernels and features into the Bayesian framework to construct 1-bit CNNs in an end-to-end manner, which have not been considered in any previous related methods. The Bayesian losses are achieved with a theoretical support to optimize the network simultaneously in both continuous and discrete spaces, aggregating different losses jointly to improve the model capacity. Extensive experiments on the ImageNet and CIFAR datasets show that BONNs achieve the best classification performance compared to state-of-the-art 1-bit CNNs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑