Verifying that input-output relationships of a neural network conform to prescribed operational specifications is a key enabler towards deploying these networks in safety-critical applications. Semidefinite programming (SDP)-based approaches to Rectified Linear Unit (ReLU) network verification transcribe this problem into an optimization problem, where the accuracy of any such formulation reflects the level of fidelity in how the neural network computation is represented, as well as the relaxations of intractable constraints. While the literature contains much progress on improving the tightness of SDP formulations while maintaining tractability, comparatively little work has been devoted to the other extreme, i.e., how to most accurately capture the original verification problem before SDP relaxation. In this work, we develop an exact, convex formulation of verification as a completely positive program (CPP), and provide analysis showing that our formulation is minimal -- the removal of any constraint fundamentally misrepresents the neural network computation. We leverage our formulation to provide a unifying view of existing approaches, and give insight into the source of large relaxation gaps observed in some cases.
Electrical Engineering and Computer Sciences UC Berkeley（加州大学伯克利分校电气工程与计算机科学学院）；Electrical Engineering and Computer Sciences Mechanical Engineering UC Berkeley（加州大学伯克利分校电气工程与计算机科学学院、机械工程学院）
Although neural networks have been applied to several systems in recent years, they still cannot be used in safety-critical systems due to the lack of efficient techniques to certify their robustness. A number of techniques based on convex optimization have been proposed in the literature to study the robustness of neural networks, and the semidefinite programming (SDP) approach has emerged as a leading contender for the robust certification of neural networks. The major challenge to the SDP approach is that it is prone to a large relaxation gap. In this work, we address this issue by developi…
We suppose the existence of an oracle which solves any semidefinite programming (SDP) problem satisfying Slater's condition simultaneously at its primal and dual sides. We note that such an oracle might not be able to directly solve general SDPs even after certain regularization schemes are applied. In this work we fill this gap and show how to use such an oracle to "completely solve" an arbitrary SDP. Completely solving an SDP, includes, for example, distinguishing between weak/strong feasibility/infeasibility and detecting when the optimal value is attained or not. We will employ several too…
We present a novel analysis of semidefinite programs (SDPs) with positive duality gaps, i.e. different optimal values in the primal and dual problems. These SDPs are extremely pathological, often unsolvable, and also serve as models of more general pathological convex programs. However, despite their allure, they are not well understood even when they have just two variables. We first completely characterize two variable SDPs with positive gaps; in particular, we transform them into a standard form that makes the positive gap trivial to recognize. The transformation is very simple, as it mostl…
Certifying the safety or robustness of neural networks against input uncertainties and adversarial attacks is an emerging challenge in the area of safe machine learning and control. To provide such a guarantee, one must be able to bound the output of neural networks when their input changes within a bounded set. In this paper, we propose a semidefinite programming (SDP) framework to address this problem for feed-forward neural networks with general activation functions and input uncertainty sets. Our main idea is to abstract various properties of activation functions (e.g., monotonicity, bound…
A power system unit commitment (UC) problem considering uncertainties of renewable energy sources is investigated in this paper, through a distributionally robust optimization approach. We assume that the first and second order moments of stochastic parameters can be inferred from historical data, and then employed to model the set of probability distributions. The resulting problem is a two-stage distributionally robust unit commitment with second order moment constraints, and we show that it can be recast as a mixed-integer semidefinite programming (MI-SDP) with finite constraints. The solut…
There has been an increasing interest in using neural networks in closed-loop control systems to improve performance and reduce computational costs for on-line implementation. However, providing safety and stability guarantees for these systems is challenging due to the nonlinear and compositional structure of neural networks. In this paper, we propose a novel forward reachability analysis method for the safety verification of linear time-varying systems with neural networks in feedback interconnection. Our technical approach relies on abstracting the nonlinear activation functions by quadrati…
We study the safety verification problem for a class of distributed parameter systems described by partial differential equations (PDEs), i.e., the problem of checking whether the solutions of the PDE satisfy a set of constraints at a particular point in time. The proposed method is based on an extension of barrier certificates to infinite-dimensional systems. In this respect, we introduce barrier functionals, which are functionals of the dependent and independent variables. Given a set of initial conditions and an unsafe set, we demonstrate that if such a functional exists satisfying two (int…
We study optimization programs given by a bilinear form over non-commutative variables subject to linear inequalities. Problems of this form include the entangled value of two-prover games, entanglement-assisted coding for classical channels and quantum-proof randomness extractors. We introduce an asymptotically converging hierarchy of efficiently computable semidefinite programming (SDP) relaxations for this quantum optimization. This allows us to give upper bounds on the quantum advantage for all of these problems. Compared to previous work of Pironio, Navascues and Acin, our hierarchy has a…
Optimal power flow (OPF) is an important problem in the operation of electric power systems. Due to the OPF problem's non-convexity, there may exist multiple local optima. Certifiably obtaining the global solution is important for certain applications of OPF problems. Many global optimization techniques compute an optimality gap that compares the achievable objective value corresponding to the feasible point from a local solution algorithm with the objective value bound from a convex relaxation technique. Rather than the traditional practice of completely separating the local solution and conv…
CMU（卡内基梅隆大学）；Columbia University（哥伦比亚大学）；Drexel University（德雷塞尔大学）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；UCLA（加州大学洛杉矶分校）；Bosch Center for AI（博世人工智能中心）
Bound propagation methods, when combined with branch and bound, are among the most effective methods to formally verify properties of deep neural networks such as correctness, robustness, and safety. However, existing works cannot handle the general form of cutting plane constraints widely accepted in traditional solvers, which are crucial for strengthening verifiers with tightened convex relaxations. In this paper, we generalize the bound propagation procedure to allow the addition of arbitrary cutting plane constraints, including those involving relaxed integer variables that do not appear i…
Optimal control of stochastic Volterra integral equations with completely monotone kernels and stochastic differential equations on Hilbert spaces with unbounded control and diffusion operators
Sapienza Università di Roma（罗马大学）；Technische Universität Berlin（柏林工业大学）
