arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.03258cs.LGmath.OC
收藏

用ReLU网络表示的目标函数的全局优化

Global Optimization of Objective Functions Represented by ReLU Networks

Stanford University(斯坦福大学);The Hebrew University of Jerusalem(耶路撒冷希伯来大学)

Christopher A. Strong, Haoze Wu, Aleksandar Zeljić, Kyle D. Julian, Guy Katz, Clark Barrett, Mykel J. Kochenderfer

摘要:

神经网络可以学习复杂的非凸函数,并且在安全关键环境中保证其正确行为是具有挑战性的。存在许多方法来发现网络中的故障(例如,对抗性的例子),但这些不能保证没有失败。验证算法解决了这一需求,并通过回答“是或否”的问题来提供关于神经网络的正式保证。例如,它们可以回答是否在一定范围内存在违规行为。然而,个别的“是或否”问题不能回答定性的问题,如“在这些界限内最大的误差是什么”;这些问题的答案在于优化领域。因此,我们提出了扩展现有验证器的策略来执行优化并找到:(i)给定输入区域中的最极端故障和(ii)导致故障所需的最小输入扰动。一个天真的方法,使用二分搜索与现成的验证结果在许多昂贵的和重叠的调用验证。相反,我们提出了一种方法,紧密集成的优化过程中的验证过程,实现更好的运行时性能比天真的方法。我们评估我们的方法实现的扩展Marabou,一个国家的最先进的神经网络验证,并比较其性能与二分法和MIPVerify,基于优化的验证。我们观察到Marabou和MIPVerify扩展之间的互补性能。

英文摘要:

Neural networks can learn complex, non-convex functions, and it is challenging to guarantee their correct behavior in safety-critical contexts. Many approaches exist to find failures in networks (e.g., adversarial examples), but these cannot guarantee the absence of failures. Verification algorithms address this need and provide formal guarantees about a neural network by answering "yes or no" questions. For example, they can answer whether a violation exists within certain bounds. However, individual "yes or no" questions cannot answer qualitative questions such as "what is the largest error within these bounds"; the answers to these lie in the domain of optimization. Therefore, we propose strategies to extend existing verifiers to perform optimization and find: (i) the most extreme failure in a given input region and (ii) the minimum input perturbation required to cause a failure. A naive approach using a bisection search with an off-the-shelf verifier results in many expensive and overlapping calls to the verifier. Instead, we propose an approach that tightly integrates the optimization process into the verification procedure, achieving better runtime performance than the naive approach. We evaluate our approach implemented as an extension of Marabou, a state-of-the-art neural network verifier, and compare its performance with the bisection approach and MIPVerify, an optimization-based verifier. We observe complementary performance between our extension of Marabou and MIPVerify.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑