arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.04469cs.LGcs.AImath.OC
收藏

驯服二值化神经网络与混合整数规划

Taming Binarized Neural Networks and Mixed-Integer Programs

Czech Technical University(捷克技术大学);HSBC Holdings(汇丰控股)

Johannes Aspman, Georgios Korpas, Jakub Marecek

AI总结:

本文将二值化神经网络训练重构为混合整数规划的次可加对偶,证明其具有驯服表示,从而借助隐式微分框架实现反向传播,并推广至更广泛的混合整数规划。

摘要:

近年来,二值化神经网络因其可解释性而受到广泛关注。与此同时,诸如反向传播之类的自动微分算法在二值化神经网络中失效,这限制了它们的适用性。通过将训练二值化神经网络的问题重新表述为混合整数规划的次可加对偶问题,我们证明了二值化神经网络具有驯服(tame)表示。这反过来使得使用 Bolte 等人的隐式微分框架成为可能,从而为在二值化神经网络背景下实际实现反向传播提供了可能性。该方法还可用于更广泛的混合整数规划类别,不仅限于二值化神经网络的训练,例如在人工智能的符号方法及其他领域中所遇到的混合整数规划。

英文摘要:

There has been a great deal of recent interest in binarized neural networks, especially because of their explainability. At the same time, automatic differentiation algorithms such as backpropagation fail for binarized neural networks, which limits their applicability. By reformulating the problem of training binarized neural networks as a subadditive dual of a mixed-integer program, we show that binarized neural networks admit a tame representation. This, in turn, makes it possible to use the framework of Bolte et al. for implicit differentiation, which offers the possibility for practical implementation of backpropagation in the context of binarized neural networks. This approach could also be used for a broader class of mixed-integer programs, beyond the training of binarized neural networks, as encountered in symbolic approaches to AI and beyond.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑