arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.11201cs.LGcs.AIcs.LOmath.OC
收藏

基于不确定性和的高效神经网络分析

Efficient Neural Network Analysis with Sum-of-Infeasibilities

Stanford University(斯坦福大学);The Hebrew University of Jerusalem(耶路撒冷希伯来大学)

Haoze Wu, Aleksandar Zeljić, Guy Katz, Clark Barrett

摘要:

受凸优化中的非线性和方法的启发,我们提出了一种新的方法来分析具有分段线性激活函数的神经网络上的验证查询。给定一个过逼近非凸激活函数的凸松弛,我们将激活函数的违规行为编码为代价函数,并相对于凸松弛进行优化。成本函数被称为无效和(SoI),其设计使得其最小值为零,并且仅当满足所有激活函数时才实现。我们提出了一个随机过程,DeepSoI,有效地减少SoI。通过用DeepSoI替换每个搜索状态下执行的凸过程,可以实现对基于规范案例分析的完整搜索过程的扩展。使用DeepSoI扩展完整的搜索可以同时实现多个目标:1)它引导搜索朝向反例; 2)它可以实现更明智的分支决策; 3)它为绑定派生创造了额外的机会。在不同的基准和求解器的广泛评估表明所提出的技术的好处。特别是,我们证明了SoI显着提高了现有的完整的搜索过程的性能。此外,基于SoI的实现优于其他国家的最先进的完整的验证。我们还表明,我们的技术可以有效地提高扰动界由最近的对抗攻击算法。

英文摘要:

Inspired by sum-of-infeasibilities methods in convex optimization, we propose a novel procedure for analyzing verification queries on neural networks with piecewise-linear activation functions. Given a convex relaxation which over-approximates the non-convex activation functions, we encode the violations of activation functions as a cost function and optimize it with respect to the convex relaxation. The cost function, referred to as the Sum-of-Infeasibilities (SoI), is designed so that its minimum is zero and achieved only if all the activation functions are satisfied. We propose a stochastic procedure, DeepSoI, to efficiently minimize the SoI. An extension to a canonical case-analysis-based complete search procedure can be achieved by replacing the convex procedure executed at each search state with DeepSoI. Extending the complete search with DeepSoI achieves multiple simultaneous goals: 1) it guides the search towards a counter-example; 2) it enables more informed branching decisions; and 3) it creates additional opportunities for bound derivation. An extensive evaluation across different benchmarks and solvers demonstrates the benefit of the proposed techniques. In particular, we demonstrate that SoI significantly improves the performance of an existing complete search procedure. Moreover, the SoI-based implementation outperforms other state-of-the-art complete verifiers. We also show that our technique can efficiently improve upon the perturbation bound derived by a recent adversarial attack algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑