Sparse signal restoration is usually formulated as the minimization of a quadratic cost function $\|y-Ax\|_2^2$, where A is a dictionary and x is an unknown sparse vector. It is well-known that imposing an $\ell_0$ constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the $\ell_0$-norm is replaced by the $\ell_1$-norm. Among the many efficient $\ell_1$ solvers, the homotopy algorithm minimizes $\|y-Ax\|_2^2+λ\|x\|_1$ with respect to x for a continuum of $λ$'s. It is inspired by the piecewise regularity of the $\ell_1$-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem $\|y-Ax\|_2^2+λ\|x\|_0$ for a continuum of $λ$'s and propose two heuristic search algorithms for $\ell_0$-homotopy. Continuation Single Best Replacement is a forward-backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for $\ell_0$-minimization at a given $λ$. The adaptive search of the $λ$-values is inspired by $\ell_1$-homotopy. $\ell_0$ Regularization Path Descent is a more complex algorithm exploiting the structural properties of the $\ell_0$-regularization path, which is piecewise constant with respect to $λ$. Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection.
We propose an adaptive finite element algorithm to approximate solutions of elliptic problems whose forcing data is locally defined and is approximated by regularization (or mollification). We show that the energy error decay is quasi-optimal in two dimensional space and sub-optimal in three dimensional space. Numerical simulations are provided to confirm our findings.
In this paper we propose a primal-dual homotopy method for $\ell_1$-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path for the dual problem. We show that by solving a small linear program, one can jump to the next primal break point and then, solving another small linear program, a new optimal dual solution is calculate…
Approximations of the Dirac delta distribution are commonly used to create sequences of smooth functions approximating nonsmooth (generalized) functions, via convolution. In this work, we show a priori rates of convergence of this approximation process in standard Sobolev norms, with minimal regularity assumptions on the approximation of the Dirac delta distribution. The application of these estimates to the numerical solution of elliptic problems with singularly supported forcing terms allows us to provide sharp $H^1$ and $L^2$ error estimates for the corresponding regularized problem. As an …
We consider polynomial approximation over the interval $[-1,1]$ by regularized weighted discrete least squares methods with $\ell_2-$ or $\ell_1-$regularization, respectively. As the set of nodes we use Gauss quadrature points (which are zeros of orthogonal polynomials). The number of Gauss quadrature points is $N+1$. For $2L\leq2N+1$, with the aid of Gauss quadrature, we obtain approximation polynomials of degree $L$ in closed form without solving linear algebra or optimization problems. In fact, these approximation polynomials can be expressed in the form of the barycentric interpolation for…
We prove the support recovery for a general class of linear and nonlinear evolutionary partial differential equation (PDE) identification from a single noisy trajectory using $\ell_1$ regularized Pseudo-Least Squares model~($\ell_1$-PsLS). In any associative $\mathbb{R}$-algebra generated by finitely many differentiation operators that contain the unknown PDE operator, applying $\ell_1$-PsLS to a given data set yields a family of candidate models with coefficients $\mathbf{c}(λ)$ parameterized by the regularization weight $λ\geq 0$. The trace of $\{\mathbf{c}(λ)\}_{λ\geq 0}$ suffers from high …
Nonmonotone Barzilai-Borwein Gradient Algorithm for $\ell_1$-Regularized Nonsmooth Minimization in Compressive Sensing
Institute of Applied Mathematics, College of Mathematics and Information Science, Henan University（河南大学数学与信息科学学院应用数学研究所）；National Center for Theoretical Sciences (South), National Cheng Kung University（台湾成功大学南方理论科学中心）；Department of Applied Mathematics, Hong Kong Polytechnic University（香港理工大学应用数学系）
We study the dependence of the continuity constants for the regularized Poincaré and Bogovski\uı integral operators acting on differential forms defined on a domain $Ω$ of $\mathbb{R}^n$. We, in particular, study the dependence of such constants on certain geometric characteristics of the domain when these operators are considered as mappings from (a subset of) $L^2(Ω,Λ^\ell)$ to $H^1(Ω,Λ^{\ell-1})$, $\ell \in \{1, \ldots, n\}$. For domains $Ω$ that are star shaped with respect to a ball $B$ we study the dependence of the constants on the ratio $diam(Ω)/diam(B)$. A program on how to develop es…
The choice of the parameter value for regularized inverse problems is critical to the results and remains a topic of interest. This article explores a criterion for selecting a good parameter value by maximizing the probability of the data, {with no prior knowledge of the noise variance}. These concepts are developed for $\ell_2$ and consequently $\ell_1$ regularization models by way of their Bayesian interpretations. Based on these concepts, an iterative scheme is proposed and demonstrated to converge accurately, and analytical convergence results are provided that substantiate these empirica…
We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the $\ell_1$ norm composed with a linear transform. Empirical results show that the regularization with the $\ell_1$ norm can promote sparsity of a regularized solution. It is the goal of this paper to understand theoretically the effect of the regularization parameter on the sparsity of the regularized solutions. We establish a characterization of the sparsity under the transform matrix of the solution. When the fidelity term has a special structure and the …
In this paper we present an active-set method for the solution of $\ell_1$-regularized convex quadratic optimization problems. It is derived by combining a proximal method of multipliers (PMM) strategy with a standard semismooth Newton method (SSN). The resulting linear systems are solved using a Krylov-subspace method, accelerated by certain general-purpose preconditioners which are shown to be optimal with respect to the proximal parameters. Practical efficiency is further improved by warm-starting the algorithm using a proximal alternating direction method of multipliers. We show that the o…
