Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements
Mark Rudelson, Roman Vershynin
备注：
References to related work added
发表信息：
CISS'06 (40th Annual Conference on Information Sciences and Systems), 2006, 207--212. Journal version in: Communications on Pure and Applied Mathematics 61 (2008), 1025-1045
我们想要从少量的线性测量（与某些固定向量的内积）中精确重建一个稀疏信号 f（R^n 中具有小支撑的向量）。自 80 年代以来，Dave Donoho 及其合作者一直倡导一种通过线性规划实现的直观且优雅的重建方法。也就是说，可以将高度非凸的重建问题松弛为凸问题——而且进一步松弛为线性规划。然而，重建问题何时等价于其凸松弛仍是一个开放问题。最近许多作者的工作表明，使用凸松弛方法从线性测量中精确重建任何长度为 n、支撑为 r 的 r-稀疏信号 f（R^n 中支撑为 r 的向量）所需的测量数 k(r,n) 通常为 O(r polylog(n))。然而，尽管算法在实践中表现非常好，但已知的测量数 k(r,n) 估计包含巨大的常数。在本文中，我们考虑随机高斯测量和随机傅里叶测量（f 的频率样本）。对于高斯测量，我们首次证明了具有合理常数的保证：k(r,n) < 12 r (2 + log(n/r))，这在常数意义上是最优的。对于傅里叶测量，我们证明了已知的最佳界 k(r,n) = O(r log(n) . log^2(r) log(r log n))，这在 log log n 和 log^3 r 因子内是最优的。我们的论证基于巴拿赫空间中的几何泛函分析和概率技术。
英文摘要：
We want to exactly reconstruct a sparse signal f (a vector in R^n of small support) from few linear measurements of f (inner products with some fixed vectors). A nice and intuitive reconstruction by Linear Programming has been advocated since 80-ies by Dave Donoho and his collaborators. Namely, one can relax the reconstruction problem, which is highly nonconvex, to a convex problem -- and, moreover, to a linear program. However, when is exactly the reconstruction problem equivalent to its convex relaxation is an open question. Recent work of many authors shows that the number of measurements k(r,n) needed to exactly reconstruct any r-sparse signal f of length n (a vector in R^n of support r) from its linear measurements with the convex relaxation method is usually O(r polylog(n)). However, known estimates of the number of measurements k(r,n) involve huge constants, in spite of very good performance of the algorithms in practice. In this paper, we consider random Gaussian measurements and random Fourier measurements (a frequency sample of f). For Gaussian measurements, we prove the first guarantees with reasonable constants: k(r,n) < 12 r (2 + log(n/r)), which is optimal up to constants. For Fourier measurements, we prove the best known bound k(r,n) = O(r log(n) . log^2(r) log(r log n)), which is optimal within the log log n and log^3 r factors. Our arguments are based on the technique of Geometric Functional Analysis and Probability in Banach spaces.
We develop an efficient and robust high-dimensional sparse Fourier algorithm for noisy samples. Earlier in the paper ``Multi-dimensional sublinear sparse Fourier algorithm" (2016), an efficient sparse Fourier algorithm with $Θ(ds \log s)$ average-case runtime and $Θ(ds)$ sampling complexity under certain assumptions was developed for signals that are $s$-sparse and bandlimited in the $d$-dimensional Fourier domain, i.e. there are at most $s$ energetic frequencies and they are in $ \left[-N/2, N/2\right)^d\cap \mathbb{Z}^d$. However, in practice the measurements of signals often contain noise, …
Multigrid methods are popular solution algorithms for many discretized PDEs, either as standalone iterative solvers or as preconditioners, due to their high efficiency. However, the choice and optimization of multigrid components such as relaxation schemes and grid-transfer operators is crucial to the design of optimally efficient algorithms. It is well--known that local Fourier analysis (LFA) is a useful tool to predict and analyze the performance of these components. In this paper, we develop a local Fourier analysis of monolithic multigrid methods based on additive Vanka relaxation schemes …
In this paper a deterministic sparse Fourier transform algorithm is presented which breaks the quadratic-in-sparsity runtime bottleneck for a large class of periodic functions exhibiting structured frequency support. These functions include, e.g., the oft-considered set of block frequency sparse functions of the form $$f(x) = \sum^{n}_{j=1} \sum^{B-1}_{k=0} c_{ω_j + k} e^{i(ω_j + k)x},~~\{ ω_1, \dots, ω_n \} \subset \left(-\left\lceil \frac{N}{2}\right\rceil, \left\lfloor \frac{N}{2}\right\rfloor\right]\cap\mathbb{Z}$$ as a simple subclass. Theoretical error bounds in combination with numerica…
In this paper we propose a general strategy for rapidly computing sparse Legendre expansions. The resulting methods yield a new class of fast algorithms capable of approximating a given function $f:[-1,1] \rightarrow \mathbb{R}$ with a near-optimal linear combination of $s$ Legendre polynomials of degree $\leq N$ in just $(s \log N)^{\mathcal{O}(1)}$-time. When $s \ll N$ these algorithms exhibit sublinear runtime complexities in $N$, as opposed to traditional $Ω(N \log N)$-time methods for computing all of the first $N$ Legendre coefficients of $f$. Theoretical as well as numerical results dem…
In this paper, we discuss the development of a sublinear sparse Fourier algorithm for high-dimensional data. In ``Adaptive Sublinear Time Fourier Algorithm" by D. Lawlor, Y. Wang and A. Christlieb (2013), an efficient algorithm with $Θ(k\log k)$ average-case runtime and $Θ(k)$ average-case sampling complexity for the one-dimensional sparse FFT was developed for signals of bandwidth $N$, where $k$ is the number of significant modes such that $k\ll N$. In this work we develop an efficient algorithm for sparse FFT for higher dimensional signals, extending some of the ideas in the paper mentioned …
In this paper, we derive a new reconstruction method for real non-harmonic Fourier sums, i.e., real signals which can be represented as sparse exponential sums of the form $f(t) = \sum_{j=1}^{K} γ_{j} \, \cos(2πa_{j} t + b_{j})$, where the frequency parameters $a_{j} \in {\mathbb R}$ (or $a_{j} \in {\mathrm i} {\mathbb R}$) are pairwise different. Our method is based on the recently proposed stable iterative rational approximation algorithm in \cite{NST18}. For signal reconstruction we use a set of classical Fourier coefficients of $f$ with regard to a fixed interval $(0, P)$ with $P>0$. Even …
In this paper, we theoretically propose a new hashing scheme to establish the sparse Fourier transform in high-dimensional space. The estimation of the algorithm complexity shows that this sparse Fourier transform can overcome the curse of dimensionality. To the best of our knowledge, this is the first polynomial-time algorithm to recover the high-dimensional continuous frequencies.
Large sparse linear algebraic systems can be found in a variety of scientific and engineering fields, and many scientists strive to solve them in an efficient and robust manner. In this paper, we propose an interpretable neural solver, the Fourier Neural Solver (FNS), to address them. FNS is based on deep learning and Fast Fourier transform. Because the error between the iterative solution and the ground truth involves a wide range of frequency modes, FNS combines a stationary iterative method and frequency space correction to eliminate different components of the error. Local Fourier analysis…
We present a general class of compressed sensing matrices which are then demonstrated to have associated sublinear-time sparse approximation algorithms. We then develop methods for constructing specialized matrices from this class which are sparse when multiplied with a discrete Fourier transform matrix. Ultimately, these considerations improve previous sampling requirements for deterministic sparse Fourier transform methods.
In this paper we propose a new fast Fourier transform to recover a real nonnegative signal ${\bf x}$ from its discrete Fourier transform. If the signal ${\mathbf x}$ appears to have a short support, i.e., vanishes outside a support interval of length $m < N$, then the algorithm has an arithmetical complexity of only ${\cal O}(m \log m \log (N/m)) $ and requires ${\cal O}(m \log (N/m))$ Fourier samples for this computation. In contrast to other approaches there is no a priori knowledge needed about sparsity or support bounds for the vector ${\bf x}$. The algorithm automatically recognizes and e…
We develop the uniform sparse Fast Fourier Transform (usFFT), an efficient, non-intrusive, adaptive algorithm for the solution of elliptic partial differential equations with random coefficients. The algorithm is an adaption of the sparse Fast Fourier Transform (sFFT), a dimension-incremental algorithm, which tries to detect the most important frequencies in a given search domain and therefore adaptively generates a suitable Fourier basis corresponding to the approximately largest Fourier coefficients of the function. The usFFT does this w.r.t. the stochastic domain of the PDE simultaneously f…
In this paper modified variants of the sparse Fourier transform algorithms from [14] are presented which improve on the approximation error bounds of the original algorithms. In addition, simple methods for extending the improved sparse Fourier transforms to higher dimensional settings are developed. As a consequence, approximate Fourier transforms are obtained which will identify a near-optimal k-term Fourier series for any given input function, $f : [0, 2 pi] -> C, in O(k^2 \cdot D^4)$ time (neglecting logarithmic factors). Faster randomized Fourier algorithm variants with runtime complexiti…
In his monograph Chebyshev and Fourier Spectral Methods, John Boyd claimed that, regarding Fourier spectral methods for solving differential equations, ``[t]he virtues of the Fast Fourier Transform will continue to improve as the relentless march to larger and larger [bandwidths] continues''. This paper attempts to further the virtue of the Fast Fourier Transform (FFT) as not only bandwidth is pushed to its limits, but also the dimension of the problem. Instead of using the traditional FFT however, we make a key substitution: a high-dimensional, sparse Fourier transform (SFT) paired with rando…
In this paper a sublinear time algorithm is presented for the reconstruction of functions that can be represented by just few out of a potentially large candidate set of Fourier basis functions in high spatial dimensions, a so-called high-dimensional sparse fast Fourier transform. In contrast to many other such algorithms, our method works for arbitrary candidate sets and does not make additional structural assumptions on the candidate set. Our transform significantly improves upon the other approaches available for such a general framework in terms of the scaling of the sample complexity. Our…
We consider fast, provably accurate algorithms for approximating functions on the $d$-dimensional torus, $f: \mathbb{ T }^d \rightarrow \mathbb{C}$, that are sparse (or compressible) in the Fourier basis. In particular, suppose that the Fourier coefficients of $f$, $\{c_{\bf k} (f) \}_{{\bf k} \in \mathbb{Z}^d}$, are concentrated in a finite set $I \subset \mathbb{Z}^d$ so that $$\min_{Ω\subset I s.t. |Ω| =s } \left\| f - \sum_{{\bf k} \in Ω} c_{\bf k} (f) e^{ -2 πi {\bf k} \cdot \circ} \right\|_2 < ε\|f \|_2$$ holds for $s \ll |I|$ and $ε\in (0,1)$. We aim to identify a near-minimizing subset…
Defense Innovation Institute, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Perception of the full state is an essential technology to support the monitoring, analysis, and design of physical systems, one of whose challenges is to recover global field from sparse observations. Well-known for brilliant approximation ability, deep neural networks have been attractive to data-driven flow and heat field reconstruction studies. However, limited by network structure, existing researches mostly learn the reconstruction mapping in finite-dimensional space and has poor transferability to variable resolution of outputs. In this paper, we extend the new paradigm of neural operat…
We consider the numerical solution of the discrete multi-marginal optimal transport (MOT) by means of the Sinkhorn algorithm. In general, the Sinkhorn algorithm suffers from the curse of dimensionality with respect to the number of marginals. If the MOT cost function decouples according to a tree or circle, its complexity is linear in the number of marginal measures. In this case, we speed up the convolution with the radial kernel required in the Sinkhorn algorithm by non-uniform fast Fourier methods. Each step of the proposed accelerated Sinkhorn algorithm with a tree-structured cost function…
In this paper we consider the special case where a discrete signal ${\bf x}$ of length N is known to vanish outside a support interval of length $m < N$. If the support length $m$ of ${\bf x}$ or a good bound of it is a-priori known we derive a sublinear deterministic algorithm to compute ${\bf x}$ from its discrete Fourier transform. In case of exact Fourier measurements we require only ${\cal O}(m \log m)$ arithmetical operations. For noisy measurements, we propose a stable ${\cal O}(m \log N)$ algorithm.
We propose a sparse regularization model for inversion of incomplete Fourier transforms and apply it to seismic wavefield modeling. The objective function of the proposed model employs the Moreau envelope of the $\ell_0$ norm under a tight framelet system as a regularization to promote sparsity. This model leads to a non-smooth, non-convex optimization problem for which traditional iteration schemes are inefficient or even divergent. By exploiting special structures of the $\ell_0$ norm, we identify a local minimizer of the proposed non-convex optimization problem with a global minimizer of a …
Phase retrieval consists in the recovery of an unknown signal from phaseless measurements of its usually complex-valued Fourier transform. Without further assumptions, this problem is notorious to be severe ill posed such that the recovery of the true signal is nearly impossible. In certain applications like crystallography, speckle imaging in astronomy, or blind channel estimation in communications, the unknown signal has a specific, sparse structure. In this paper, we exploit these sparse structure to recover the unknown signal uniquely up to inevitable ambiguities as global phase shifts, tr…
