We consider joint Tikhonov- and Lavrentiev-regularization of control problems with pointwise control- and state-constraints. We derive error estimates for the error which is introduced by the Tikhonov regularization. With the help of this results we show, that if the solution of the unconstrained problem has no active constraints, the same holds for the Tikhonov-regularized solution if the regularization parameter is small enough and a certain source condition is fulfilled.
In this paper, elliptic control problems with pointwise box constraints on the state is considered, where the corresponding Lagrange multipliers in general only represent regular Borel measure functions. To tackle this difficulty, the Lavrentiev regularization is employed to deal with the state constraints. To numerically discretize the resulted problem, since the weakness of variational discretization in numerical implementation, full piecewise linear finite element discretization is employed. Estimation of the error produced by regularization and discretization is done. The error order of fu…
The minimum value function appearing in Tikhonov regularization technique is very useful in determining the regularization parameter, both theoretically and numerically. In this paper, we discuss the properties of the minimum value function. We also propose an efficient method to determine the regularization parameter. A new criterion for the determination of the regularization parameter is also discussed.
We investigate the asymptotic behavior of solutions to a second order differential equation with vanishing damping term, convex potential and regularizing Tikhonov term.
Tikhonov regularization is a common technique used when solving poorly behaved optimization problems. Often, and with good reason, this technique is applied by practitioners in an ad hoc fashion. In this note, we systematically illustrate the role of Tikhonov regularizations in two simple, yet instructive examples. In one example, we use regular perturbation theory to predict the impact Tikhonov regularizations have on condition numbers of symmetric, positive semi-definite matrices. We then use a numerical example to confirm our result. In another example, we construct an exactly solvable opti…
It is well known that Tikhonov regularization is one of the most commonly used methods for solving ill-posed problems. One of the most widely applied approaches is based on constructing a new dataset whose sample size is greater than the original one. The enlarged sample size may bring additional computational difficulties. In this paper, we aim to make full use of Tikhonov regularization and develop a dual based semismooth Newton (DSSN) method without destroying the structure of dataset. From the point of view of theory, we will show that \blue{the DSSN method is a globally convergent method …
We consider Tikhonov regularization of control-constrained optimal control problems. We present new a-priori estimates for the regularization error assuming measure and source-measure conditions. In the special case of bang-bang solutions, we introduce another assumption to obtain the same convergence rates. This new condition turns out to be useful in the derivation of error estimates for the discretized problem. The necessity of the just mentioned assumptions to obtain certain convergence rates is analyzed. Finally, a numerical example confirms the analytical findings.
In this article, we consider the Tikhonov regularization of an optimal control problem of semilinear partial differential equations with box constraints on the control. We derive a-priori regularization error estimates for the control under suitable conditions. These conditions comprise second-order sufficient optimality conditions as well as regularity conditions on the control, which consists of a source condition and a condition on the active sets. In addition, we show that these conditions are necessary for convergence rates under certain conditions. We also consider sparse optimal control…
We provide a modified augmented Lagrange method coupled with a Tikhonov regularization for solving ill-posed state-constrained elliptic optimal control problems with sparse controls. We consider a linear quadratic optimal control problem without any additional $L^2$ regularization terms. The sparsity is guaranteed by an additional $L^1$ term. Here, the modification of the classical augmented Lagrange method guarantees us uniform boundedness of the multiplier that corresponds to the state constraints. We present a coupling between the regularization parameter introduced by the Tikhonov regulari…
In this paper, we propose a Tikhonov-like regularization for dynamical systems associated with non-expansive operators defined in closed and convex sets of a Hilbert space. We prove the well-posedness and the strong convergence of the proposed dynamical systems to a fixed point of the non-expansive operator. We apply the obtained result to dynamical system associated with the problem of finding the zeros of the sum of a cocoercive operator with the subdifferential of a convex function.
In a Hilbert setting, for convex differentiable optimization, we consider accelerated gradient dynamics combining Tikhonov regularization with Hessian-driven damping. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria. The presence of the Tikhonov regularization term induces a strong convexity property which vanishes asymptotically. To take advantage of the exponential convergence rates attached to the heavy ball method in the strongly convex case, we consider the inertial dynamic where the viscous damping coefficient is taken…
In a Hilbert framework, for convex differentiable optimization, we consider accelerated gradient methods obtained by combining temporal scaling and averaging techniques with Tikhonov regularization. We start from the continuous steepest descent dynamic with an additional Tikhonov regularization term whose coefficient vanishes asymptotically. We provide an extensive Lyapunov analysis of this first-order evolution equation. Then we apply to this dynamic the method of time scaling and averaging recently introduced by Attouch, Bot and Nguyen. We thus obtain an inertial dynamic which involves visco…
In this manuscript we would like to address the classical optimization problem of minimizing a proper, convex and lower semicontinuous function via the second order in time dynamics, combining viscous and Hessian-driven damping with a Tikhonov regularization technique. In our analysis we heavily exploit the Moreau envelope of the objective function and its properties as well as Tikhonov properties, which we extend to a nonsmooth case. We introduce the setting, which at the same time guarantees the fast convergence of the function (and Moreau envelope) values and strong convergence of the traje…
In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed to tend towards zero, not too fast, which is a key property to make the trajectories strongly converge towards the minimizer of $f$ of minimum norm. According to the structure of the heavy ball method …
A Nesterov type algorithm with double Tikhonov regularization: fast convergence of the function values and strong convergence to the minimal norm solution
We study whether a modified version of Tikhonov regularization can be used to identify several local sources from Dirichlet boundary data for a prototypical elliptic PDE. This paper extends the results presented in [5]. It turns out that the possibility of distinguishing between two, or more, sources depends on the smoothing properties of a second or fourth order PDE. Consequently, the geometry of the involved domain, as well as the position of the sources relative to the boundary of this domain, determines the identifiability. We also present a uniqueness result for the identification of a si…
Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems
It is common to have to process signals, whose values are points on the 3-D sphere. We consider a Tikhonov-type regularization model to smoothen or interpolate sphere-valued signals defined on arbitrary graphs. We propose a convex relaxation of this nonconvex problem as a semidefinite program, which is easy to solve numerically and is efficient in practice.
In a Hilbertian framework, for the minimization of a general convex differentiable function $f$, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of $f$ with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time $t$, is associated with the strongly convex function obtained by adding to $f$ a Tikhonov regularization term with vanishing coefficient $ε(t)$. In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regulariza…
We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system with Hessian driven damping and a Tikhonov regularization term in connection with the minimization of a smooth convex function in Hilbert spaces. We obtain fast convergence results for the function values along the trajectories. The Tikhonov regularization term enables the derivation of strong convergence results of the trajectory to the minimizer of the objective function of minimum norm.
Fast convergence rates and trajectory convergence of a Tikhonov regularized inertial primal\mbox{-}dual dynamical system with time scaling and vanishing damping
Tikhonov regularization of monotone operator flows not only ensures strong convergence of the trajectories but also speeds up the vanishing of the residuals
In a Hilbert space setting $\mathcal H$, we study the convergence properties as $t \to + \infty$ of the trajectories of the second-order differential equation \begin{equation*} \mbox{(AVD)}_{α, ε} \quad \quad \ddot{x}(t) + \fracα{t} \dot{x}(t) + \nabla Φ(x(t)) + ε(t) x(t) =0, \end{equation*} where $\nablaΦ$ is the gradient of a convex continuously differentiable function $Φ: \mathcal H \to \mathbb R$, $α$ is a positive parameter, and $ε(t) x(t)$ is a Tikhonov regularization term, with $\lim_{t \to \infty}ε(t) =0$. In this damped inertial system, the damping coefficient $\fracα{t}$ vanishes asy…
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
It is common to have to process signals or images whose values are cyclic and can be represented as points on the complex circle, like wrapped phases, angles, orientations, or color hues. We consider a Tikhonov-type regularization model to smoothen or interpolate circle-valued signals defined on arbitrary graphs. We propose a convex relaxation of this nonconvex problem as a semidefinite program, and an efficient algorithm to solve it.
This work is concerned with linear inverse problems where a distributed parameter is known a priori to only take on values from a given discrete set. This property can be promoted in Tikhonov regularization with the aid of a suitable convex but nondifferentiable regularization term. This allows applying standard approaches to show well-posedness and convergence rates in Bregman distance. Using the specific properties of the regularization term, it can be shown that convergence (albeit without rates) actually holds pointwise. Furthermore, the resulting Tikhonov functional can be minimized effic…
Stanford University（斯坦福大学）；Chinese University of Hong Kong（香港中文大学）
Distributionally robust optimization has been shown to offer a principled way to regularize learning models. In this paper, we find that Tikhonov regularization is distributionally robust in an optimal transport sense (i.e., if an adversary chooses distributions in a suitable optimal transport neighborhood of the empirical measure), provided that suitable martingale constraints are also imposed. Further, we introduce a relaxation of the martingale constraints which not only provides a unified viewpoint to a class of existing robust methods but also leads to new regularization tools. To realize…
Combining Strong Convergence, Values Fast Convergence and Vanishing of Gradients for a Proximal Point Algorithm Using Tikhonov Regularization in a Hilbert Space
This paper deals with a second order dynamical system with vanishing damping that contains a Tikhonov regularization term, in connection to the minimization problem of a convex Fréchet differentiable function $g$. We show that for appropriate Tikhonov regularization parameters the value of the objective function in a generated trajectory converges fast to the global minimum of the objective function and a trajectory generated by the dynamical system converges weakly to a minimizer of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral es…
A learning approach to selecting regularization parameters in multi-penalty Tikhonov regularization is investigated. It leads to a bilevel optimization problem, where the lower level problem is a Tikhonov regularized problem parameterized in the regularization parameters. Conditions which ensure the existence of solutions to the bilevel optimization problem of interest are derived, and these conditions are verified for two relevant examples. Difficulties arising from the possible lack of convexity of the lower level problems are discussed. Optimality conditions are given provided that a reason…
We study a source identification problem for a prototypical elliptic PDE from Dirichlet boundary data. This problem is ill-posed, and the involved forward operator has a significant nullspace. Standard Tikhonov regularization yields solutions which approach the minimum $L^2$-norm least-squares solution as the regularization parameter tends to zero. We show that this approach 'always' suggests that the unknown local source is very close to the boundary of the domain of the PDE, regardless of the position of the true local source. We propose an alternative regularization procedure, realized in t…
