We investigate the problem of optimal transport in the so-called Beckmann form, i.e. given two Radon measures on a compact set, we seek an optimal flow field which is a vector valued Radon measure on the same set that describes a flow between these two measures and minimizes a certain linear cost function.
We consider $L^α$ regularization of the problem, which guarantees uniqueness and forces the solution to be an integrable function rather than a Radon measure. This regularization naturally gives rise to a semi-smooth Newton scheme that can be used to solve the problem numerically. Besides motivating and developing the numerical scheme, we also include approximation results for vanishing regularization in the continuous setting.
In this paper we consider the application of several gradient methods to the traffic assignment problem: we search equilibria in the stable dynamics model (Nesterov and De Palma, 2003) and the Beckmann model. Unlike the celebrated Frank--Wolfe algorithm widely used for the Beckmann model, these gradients methods solve the dual problem and then reconstruct a solution to the primal one. We deal with the universal gradient method, the universal method of similar triangles, and the method of weighted dual averages, and estimate their complexity for the problem. Due to the primal-dual nature of the…
We consider the problem of revenue-maximizing Bayesian auction design with several bidders having independent private values over several items. We show that it can be reduced to the problem of continuous optimal transportation introduced by Beckmann (1952) where the optimal transportation flow generalizes the concept of ironed virtual valuations to the multi-item setting. We establish the strong duality between the two problems and the existence of solutions. The results rely on insights from majorization and optimal transportation theories and on the characterization of feasible interim mech…
The paper is devoted to the regularization of linear Copositive Programming problems which consists of transforming a problem to an equivalent form, where the Slater condition is satisfied and the strong duality holds. We describe here two regularization algorithms based on the concept of immobile indices and an understanding of the important role these indices play in the feasible sets' characterization. These algorithms are compared to some regularization procedures developed for a more general case of convex problems and based on a facial reduction approach. We show that the immobile-index-…
This paper is concerned with an optimization problem governed by the Kantorovich optimal transportation problem. This gives rise to a bilevel optimization problem, which can be reformulated as a mathematical problem with complementarity constraints in the space of regular Borel measures. Because of the non-smoothness induced by the complementarity relations, problems of this type are frequently regularized. Here we apply a quadratic regularization of the Kantorovich problem. As the title indicates, this is the first part in a series of three papers. It addresses the existence of optimal soluti…
We consider regularization of non-convex optimization problems involving a non-linear least-squares objective. By adding an auxiliary set of variables, we introduce a novel regularization framework whose corresponding objective function is not only provably invex, but it also satisfies the highly desirable Polyak--Lojasiewicz inequality for any choice of the regularization parameter. Although our novel framework is entirely different from the classical $\ell_2$-regularization, an interesting connection is established for the special case of under-determined linear least-squares. In particular,…
Tikhonov regularization is a common technique used when solving poorly behaved optimization problems. Often, and with good reason, this technique is applied by practitioners in an ad hoc fashion. In this note, we systematically illustrate the role of Tikhonov regularizations in two simple, yet instructive examples. In one example, we use regular perturbation theory to predict the impact Tikhonov regularizations have on condition numbers of symmetric, positive semi-definite matrices. We then use a numerical example to confirm our result. In another example, we construct an exactly solvable opti…
We extend the convergence analysis of the Scholtes-type regularization method for cardinality-constrained optimization problems. Its behavior is clarified in the vicinity of saddle points, and not just of minimizers as it has been done in the literature before. This becomes possible by using as an intermediate step the recently introduced regularized continuous reformulation of a cardinality-constrained optimization problem. We show that the Scholtes-type regularization method is well-defined locally around a nondegenerate T-stationary point of this regularized continuous reformulation. Moreov…
We analyze a potentially risk-averse convex stochastic optimization problem, where the control is deterministic and the state is a Banach-valued essentially bounded random variable. We obtain strong forms of necessary and sufficient optimality conditions for problems subject to equality and conical constraints. We propose a Moreau--Yosida regularization for the conical constraint and show consistency of the optimality conditions for the regularized problem as the regularization parameter is taken to infinity.
In this paper we study the limit of the value function for a two-scale, infinite-dimensional, stochastic controlled system with cylindrical noise and possibly degenerate diffusion. The limit is represented as the value function of a new reduced control problem (on a reduced state space). The presence of a cylindrical noise prevents representation of the limit by viscosity solutions of HJB equations, while degeneracy of diffusion coefficients prevents representation as a classical BSDE. We use a vanishing noise regularization technique.
In this paper, we focus on the local convergence rate analysis of the proximal iteratively reweighted $\ell_1$ algorithms for solving $\ell_p$ regularization problems, which are widely applied for inducing sparse solutions. We show that if the Kurdyka-Lojasiewicz (KL) property is satisfied, the algorithm converges to a unique first-order stationary point; furthermore, the algorithm has local linear convergence or local sublinear convergence. The theoretical results we derived are much stronger than the existing results for iteratively reweighted $\ell_1$ algorithms.
We consider a Beckmann formulation of an unbalanced optimal transport (UOT) problem. The $Γ$-convergence of this formulation of UOT to the corresponding optimal transport (OT) problem is established as the balancing parameter $α$ goes to infinity. The discretization of the problem is further shown to be asymptotic preserving regarding the same limit, which ensures that a numerical method can be applied uniformly and the solutions converge to the one of the OT problem automatically. Particularly, there exists a critical value, which is independent of the mesh size, such that the discrete proble…
This paper is concerned with an optimization problem that is constrained by the Kantorovich optimal transportation problem. This bilevel optimization problem can be reformulated as a mathematical problem with complementarity constraints in the space of regular Borel measures. Because of the non-smoothness induced by the complementarity relations, problems of this type are frequently regularized. Here we apply a quadratic regularization of the Kantorovich problem. As the title indicates, this is the second part in a series of three papers. While the existence of optimal solutions to both the bi…
We study a source identification problem for a prototypical elliptic PDE from Dirichlet boundary data. This problem is ill-posed, and the involved forward operator has a significant nullspace. Standard Tikhonov regularization yields solutions which approach the minimum $L^2$-norm least-squares solution as the regularization parameter tends to zero. We show that this approach 'always' suggests that the unknown local source is very close to the boundary of the domain of the PDE, regardless of the position of the true local source. We propose an alternative regularization procedure, realized in t…
In this work we analyze regularized optimal transport problems in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, the aim is to find a transport plan, which is another Radon measure on the product of the sets, that has these two measures as marginals and minimizes the sum of a certain linear cost function and a regularization term. We focus on regularization terms where a Young's function applied to the (density of the) transport plan is integrated against a product measure. This forces the transport plan to belong to a certain Orlicz space. The predual probl…
We introduce and study a mathematical framework for a broad class of regularization functionals for ill-posed inverse problems: Regularization Graphs. Regularization graphs allow to construct functionals using as building blocks linear operators and convex functionals, assembled by means of operators that can be seen as generalizations of classical infimal convolution operators. This class of functionals exhaustively covers existing regularization approaches and it is flexible enough to craft new ones in a simple and constructive way. We provide well-posedness and convergence results with the …
Along this work we study an indefinite abstract smoothing problem. After establishing necessary and sufficient conditions for the existence of solutions to this problem, the set of admissible parameters is discussed in detail. Then, its relationship with a linearly constrained interpolation problem is analyzed.
Iterative regularization exploits the implicit bias of an optimization algorithm to regularize ill-posed problems. Constructing algorithms with such built-in regularization mechanisms is a classic challenge in inverse problems but also in modern machine learning, where it provides both a new perspective on algorithms analysis, and significant speed-ups compared to explicit regularization. In this work, we propose and study the first iterative regularization procedure able to handle biases described by non smooth and non strongly convex functionals, prominent in low-complexity regularization. O…
In this paper we the formulation of inverse problems as constrained minimization problems and their iterative solution by gradient or Newton type. We carry out a convergence analysis in the sense of regularization methods and discuss applicability to the problem of identifying the spatially varying diffusivity in an elliptic PDE from different sets of observations. Among these is a novel hybrid imaging techology known as impedance acoustic tomography, for which we provide numerical experiments.
Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
This paper discusses basic results and recent developments on variational regularization methods, as developed for inverse problems. In a typical setup we review basic properties needed to obtain a convergent regularization scheme and further discuss the derivation of quantitative estimates respectively needed ingredients such as Bregman distances for convex functionals. In addition to the approach developed for inverse problems we will also discuss variational regularization in machine learning and work out some connections to the classical regularization theory. In particular we will discuss…
Regularization and finite element error estimates for elliptic distributed optimal control problems with energy regularization and state or control constraints
