我们研究检测连续分段仿射（PA）函数近似平稳点的基本计算问题。我们的贡献涵盖复杂性、正则性和算法等多个方面。具体而言，我们证明：检验由常用广义次微分定义的一阶近似平稳性概念，在计算上是不可行的，除非 P=NP。为促进可计算性，我们通过不严格使用凸次微分和法则，考虑了一种可在多项式时间内求解的松弛，并建立了其精确性的紧刻画。此外，为解决由在有限时间内终止次梯度方法的需求所引发的一个开放问题，我们提出了首个 oracle-多项式时间算法，用于检测 PA 函数所谓的近近似平稳点。我们在正则性方面的研究的一个显著副产品是首个关于等式型（Clarke）次微分和法则成立的充要条件。我们的技术围绕凸多面体的两个新几何概念展开，这些概念在非光滑分析中可能具有独立价值。此外，我们关于平稳性检验的复杂性和算法工作的若干推论解决了文献中的开放问题。为展示我们结果的多功能性，我们还将研究结果应用于一系列结构化分段光滑函数，包括 $\ ho$-margin-loss SVM、分段仿射回归和非光滑神经网络，以补充我们的发现。
英文摘要：
We study the basic computational problem of detecting approximate stationary points for continuous piecewise affine (PA) functions. Our contributions span multiple aspects, including complexity, regularity, and algorithms. Specifically, we show that testing first-order approximate stationarity concepts, as defined by commonly used generalized subdifferentials, is computationally intractable unless P=NP. To facilitate computability, we consider a polynomial-time solvable relaxation by abusing the convex subdifferential sum rule and establish a tight characterization of its exactness. Furthermore, addressing an open issue motivated by the need to terminate the subgradient method in finite time, we introduce the first oracle-polynomial-time algorithm to detect so-called near-approximate stationary points for PA functions.
A notable byproduct of our development in regularity is the first necessary and sufficient condition for the validity of an equality-type (Clarke) subdifferential sum rule. Our techniques revolve around two new geometric notions for convex polytopes and may be of independent interest in nonsmooth analysis. Moreover, some corollaries of our work on complexity and algorithms for stationarity testing address open questions in the literature. To demonstrate the versatility of our results, we complement our findings with applications to a series of structured piecewise smooth functions, including $ρ$-margin-loss SVM, piecewise affine regression, and nonsmooth neural networks.
A standard theorem in nonsmooth analysis states that a piecewise affine function $F:\mathbb R^n\rightarrow\mathbb R^n$ is surjective if it is coherently oriented in that the linear parts of its selection functions all have the same nonzero determinant sign. In this note we prove that surjectivity already follows from coherent orientation of the selection functions which are active on the unbounded sets of a polyhedral subdivision of the domain corresponding to $F$. A side bonus of the argumentation is a short proof of the classical statement that an injective piecewise affine function is coher…
We consider the problem of designing piecewise affine policies for two-stage adjustable robust linear optimization problems under right-hand side uncertainty. It is well known that a piecewise affine policy is optimal although the number of pieces can be exponentially large. A significant challenge in designing a practical piecewise affine policy is constructing good pieces of the uncertainty set. Here we address this challenge by introducing a new framework in which the uncertainty set is "approximated" by a "dominating" simplex. The corresponding policy is then based on a mapping from the un…
We study the reach control problem for affine systems on simplices, and the focus is on cases when it is known that the problem is not solvable by continuous state feedback. We examine from a geometric viewpoint the structural properties of the system which make continuous state feedbacks fail. This structure is encoded by so-called reach control indices, which are defined and developed in the paper. Based on these indices, we propose a subdivision algorithm and associated piecewise affine feedback. The method is shown to solve the reach control problem in all remaining cases, assuming it is s…
Piecewise affine functions are widely used to approximate nonlinear and discontinuous functions. However, most, if not all existing models only deal with fitting continuous functions. In this paper, we investigate the problem of fitting a discontinuous piecewise affine function to given data that lie in an orthogonal grid, where no restriction on the partition is enforced (i.e., its geometric shape can be nonconvex). This is useful for segmentation and denoising when data corresponding to images. We propose a novel Mixed Integer Program (MIP) formulation for the piecewise affine fitting proble…
This note presents an extension to the adaptive control strategy presented in [1] able to counter eventual instability due to disturbances at the input of an otherwise $\mathcal{L}_2$ stable closed-loop system. These disturbances are due to the presence of affine terms in the plant and reference model. The existence of a common Lyapunov function is used to prove global convergence of the error system, even in the presence of sliding solutions, as well as boundedness of all the adaptive gains.
In this paper, we study the solution uniqueness of an individual feasible vector of a class of convex optimization problems involving convex piecewise affine functions and subject to general polyhedral constraints. This class of problems incorporates many important polyhedral constrained $\ell_1$ recovery problems arising from sparse optimization, such as basis pursuit, LASSO, and basis pursuit denoising, as well as polyhedral gauge recovery. By leveraging the max-formulation of convex piecewise affine functions and convex analysis tools, we develop dual variables based necessary and sufficien…
The class of nonsmooth codifferentiable functions was introduced by professor V.F.~Demyanov in the late 1980s. He also proposed a method for minimizing these functions called the method of codifferential descent (MCD). However, until now almost no theoretical results on the performance of this method on particular classes of nonsmooth optimization problems were known. In the first part of the paper, we study the performance of the method of codifferential descent on a class of nonsmooth convex functions satisfying some regularity assumptions, which in the smooth case are reduced to the Lipschi…
We propose a learning-based method for Lyapunov stability analysis of piecewise affine dynamical systems in feedback with piecewise affine neural network controllers. The proposed method consists of an iterative interaction between a learner and a verifier, where in each iteration, the learner uses a collection of samples of the closed-loop system to propose a Lyapunov function candidate as the solution to a convex program. The learner then queries the verifier, which solves a mixed-integer program to either validate the proposed Lyapunov function candidate or reject it with a counterexample, …
This paper considers the class of deterministic continuous-time optimal control problems (OCPs) with piecewise-affine (PWA) vector field, polynomial Lagrangian and semialgebraic input and state constraints. The OCP is first relaxed as an infinite-dimensional linear program (LP) over a space of occupation measures. This LP, a particular instance of the generalized moment problem, is then approached by an asymptotically converging hierarchy of linear matrix inequality (LMI) relaxations. The relaxed dual of the original LP returns a polynomial approximation of the value function that solves the H…
A popular class of algorithms to optimize the dual LP relaxation of the discrete energy minimization problem (a.k.a.\ MAP inference in graphical models or valued constraint satisfaction) are convergent message-passing algorithms, such as max-sum diffusion, TRW-S, MPLP and SRMP. These algorithms are successful in practice, despite the fact that they are a version of coordinate minimization applied to a convex piecewise-affine function, which is not guaranteed to converge to a global minimizer. These algorithms converge only to a local minimizer, characterized by local consistency known from con…
In this paper, we introduce a family of second-order sigma delta quantization schemes for analog-to-digital conversion which are `quiet' : quantization output is guaranteed to fall to zero at the onset of vanishing input. In the process, we prove that the origin is a globally attractive fixed point for the related family of asymmetrically-damped piecewise affine maps. Our proof of convergence is twofold: first, we construct a trapping set using a Lyapunov-type argument; we then take advantage of the asymmetric structure of the maps under consideration to prove convergence to the origin from wi…
Differential privacy is a recently proposed notion of privacy that provides strong privacy guarantees without any assumptions on the adversary. The paper studies the problem of computing a differentially private solution to convex optimization problems whose objective function is piecewise affine. Such problem is motivated by applications in which the affine functions that define the objective function contain sensitive user information. We propose several privacy preserving mechanisms and provide analysis on the trade-offs between optimality and the level of privacy for these mechanisms. Nume…
Extending a fundamental result for (indefinite) quadratic programs, this paper shows that certain non-convex piecewise programs have only a finite number of directional stationary values, and thus, possess only finitely many locally minimum values. We present various special cases of our main results, in particular, an application to a least-squares piecewise affine regression problem for which every directional stationary point is locally minimizing.
In this paper, we present a novel approach to decompose a given piecewise affine (PWA) function into two convex PWA functions. Convex decompositions are useful to speed up or distribute evaluations of PWA functions. Different approaches to construct a convex decomposition have already been published. However, either the two resulting convex functions have very high or very different complexities, which is often undesirable, or the decomposition procedure is inapplicable even for simple cases. Our novel methodology significantly reduces these drawbacks in order to extend the applicability of co…
In this manuscript, we investigate symbolic abstractions that capture the behavior of piecewise-affine systems under input constraints and bounded external noise. This is accomplished by considering local affine feedback controllers that are jointly designed with the symbolic model, which ensures that an alternating simulation relation between the system and the abstraction holds. The resulting symbolic system is called a state-feedback abstraction and we show that it can be deterministic even when the original piecewise-affine system is unstable and non-deterministic. One benefit of this appr…
In this paper, we consider the problem of piecewise affine abstraction of nonlinear systems, i.e., the overapproximation of its nonlinear dynamics by a pair of piecewise affine functions that "includes" the dynamical characteristics of the original system. As such, guarantees for controllers or estimators based on the affine abstraction also apply to the original nonlinear system. Our approach consists of solving a linear programming (LP) problem that over-approximates the nonlinear function at only the grid points of a mesh with a given resolution and then accounting for the entire domain via…
Motivated by a growing list of nontraditional statistical estimation problems of the piecewise kind, this paper provides a survey of known results supplemented with new results for the class of piecewise linear-quadratic programs. These are linearly constrained optimization problems with piecewise linear-quadratic (PLQ) objective functions. Starting from a study of the representation of such a function in terms of a family of elementary functions consisting of squared affine functions, squared plus-composite-affine functions, and affine functions themselves, we summarize some local properties …
Hybrid systems, and especially piecewise affine (PWA) systems, are often used to model gene regulatory networks. In this paper we elaborate on previous work about control problems for this class of models, using also some recent results guaranteeing the existence and uniqueness of limit cycles, based solely on a discrete abstraction of the system and its interaction structure. Our aim is to control the transition graph of the PWA system to obtain an oscillatory behaviour, which is indeed of primary functional importance in numerous biological networks; we show how it is possible to control the…
We study the computational problem of the stationarity test for the empirical loss of neural networks with ReLU activation functions. Our contributions are: Hardness: We show that checking a certain first-order approximate stationarity concept for a piecewise linear function is co-NP-hard. This implies that testing a certain stationarity concept for a modern nonsmooth neural network is in general computationally intractable. As a corollary, we prove that testing so-called first-order minimality for functions in abs-normal form is co-NP-complete, which was conjectured by Griewank and Walther (2…
Fast and Exact Least Absolute Deviations Line Fitting via Piecewise Affine Lower-Bounding
Institute of Digital Engineering and the Faculty of Applied Natural Sciences and Humanities, Technische Hochschule Würzburg-Schweinfurt（数字工程研究所和应用自然科学与人文学院，图宾根-施维林技术大学）
We consider a nonlinear ordinary differential equation and want to control its behavior so that it reaches a target by minimizing a cost function. Our approach is to use hybrid systems to solve this problem: the complex dynamic is replaced by piecewise affine approximations which allow an analytical resolution. The sequence of affine models then forms a sequence of states of a hybrid automaton. Given a sequence of states, we introduce an hybrid approximation of the nonlinear controllable domain and propose a new algorithm computing a controllable, piecewise convex approximation. The same way t…
