In this paper, we present explicit expressions for the mixed and componentwise condition numbers of the truncated total least squares (TTLS) solution of $A\boldsymbol{x} \approx \boldsymbol{b} $ under the genericity condition, where $A$ is a $m\times n$ real data matrix and $\boldsymbol{b}$ is a real $m$-vector. Moreover, we reveal that normwise, componentwise and mixed condition numbers for the TTLS problem can recover the previous corresponding counterparts for the total least squares (TLS) problem when the truncated level of for the TTLS problem is $n$. When $A$ is a structured matrix, the structured perturbations for the structured truncated TLS (STTLS) problem are investigated and the corresponding explicit expressions for the structured normwise, componentwise and mixed condition numbers for the STTLS problem are obtained. Furthermore, the relationships between the structured and unstructured normwise, componentwise and mixed condition numbers for the STTLS problem are studied. Based on small sample statistical condition estimation (SCE), reliable condition estimation algorithms for both unstructured and structured normwise, mixed and componentwise are devised, which utilize the SVD of the augmented matrix $[A~\boldsymbol{b} ]$. The efficient proposed condition estimation algorithms can be integrated into the SVD-based direct solver for the small and medium size TTLS problem to give the error estimation for the numerical TTLS solution. Numerical experiments are reported to illustrate the reliability of the proposed estimation algorithms, which coincide with our theoretical results.
Total least squares (TLS) methods have been widely used in data fitting. Compared with the least squares method, for TLS problem we takes into account not only the observation errors, but also the errors in the measurement matrix. This is more realistic in practical applications. For the large-scale discrete ill-posed problem $Ax \approx b$, we introduce the quantum-inspired techniques to approximate the truncated total least squares (TTLS) solution. We analyze the accuracy of the quantum-inspired truncated total least squares algorithm and perform numerical experiments to demonstrate the effi…
A new closed formula for the first order perturbation estimate of the mixed least squares-total least squares (MTLS) solution is presented. It is mathematically equivalent to the one by Zheng and Yang(Numer. Linear Algebra Appl. 2019; 26(4):e2239). With this formula, general and structured normwise, mixed and componentwise condition numbers of the MTLS problem are derived. Perturbation bounds based on the normwise condition number, and compact forms for the upper bounds of mixed and componentwise condition numbers are also given in order for economic storage and efficient computation. It is sh…
This paper studies adaptive least-squares finite element methods for convection-dominated diffusion-reaction problems. The least-squares methods are based on the first-order system of the primal and dual variables with various ways of imposing outflow boundary conditions. The coercivity of the homogeneous least-squares functionals are established, and the a priori error estimates of the least-squares methods are obtained in a norm that incorporates the streamline derivative. All methods have the same convergence rate provided that meshes in the layer regions are fine enough. To increase comput…
Least-Squares versus Partial Least-Squares Finite Element Methods: Robust A Priori and A Posteriori Error Estimates of Augmented Mixed Finite Element Methods
Given a sequence of Marcinkiewicz-Zygmund inequalities in $L_2$ on a usual compact space $\mathcal M$, Gröchenig introduced the weighted least squares polynomials and the least squares quadrature from pointwise samples of a function, and obtained approximation theorems and quadrature errors. In this paper we obtain approximation theorems and quadrature errors on the sphere which are optimal. We also give upper bounds of the operator norms of the weighted least squares operators.
This work presents study on regularized and non-regularized versions of perceptron learning and least squares algorithms for classification problems. Fr'echet derivatives for regularized least squares and perceptron learning algorithms are derived. Different Tikhonov's regularization techniques for choosing the regularization parameter are discussed. Decision boundaries obtained by non-regularized algorithms to classify simulated and experimental data sets are analyzed.
This paper mainly investigates the strong convergence and stability of the truncated Euler-Maruyama (EM) method for stochastic differential delay equations with variable delay whose coefficients can be growing super-linearly. By constructing appropriate truncated functions to control the super-linear growth of the original coefficients, we present a type of the truncated EM method for such SDDEs with variable delay, which is proposed to be approximated by the value taken at the nearest grid points on the left of the delayed argument. The strong convergence result (without order) of the method …
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
We define and analyse a least-squares finite element method for a first-order reformulation of a scaled Brinkman model of fluid flow through porous media. We introduce a pseudostress variable that allows to eliminate the pressure variable from the system. It can be recovered by a simple post-processing. It is shown that the least-squares functional is uniformly equivalent, i.e., independent of the singular perturbation parameter, to a parameter dependent norm. This norm equivalence implies that the least-squares functional evaluated in the discrete solution provides an efficient and reliable a…
In this article, we address the solution of advection-dominated flow problems by stabilised methods, by means of least-squares computed stabilised coefficients. As main methodological tool, we introduce a data-driven off-line/on-line strategy to compute them with low computational cost. We compare the errors provided by the least-squares stabilised coefficients to those provided by several previously established stabilised coefficients within the solution of advection-diffusion and Navier-Stokes flows, on structured and un-structured grids, with and Lagrange Finite Elements up to third degree …
The alternating least squares algorithm for CP and Tucker decomposition is dominated in cost by the tensor contractions necessary to set up the quadratic optimization subproblems. We introduce a novel family of algorithms that uses perturbative corrections to the subproblems rather than recomputing the tensor contractions. This approximation is accurate when the factor matrices are changing little across iterations, which occurs when alternating least squares approaches convergence. We provide a theoretical analysis to bound the approximation error. Our numerical experiments demonstrate that t…
In this paper we discuss spectral properties of operators associated with the least-squares finite element approximation of elliptic partial differential equations. The convergence of the discrete eigenvalues and eigenfunctions towards the corresponding continuous eigenmodes is studied and analyzed with the help of appropriate $L^2$ error estimates. A priori and a posteriori estimates are proved.
We present a two-stage least-squares method to inverse medium problems of reconstructing multiple unknown coefficients simultaneously from noisy data. A direct sampling method is applied to detect the location of the inhomogeneity in the first stage, while a total least-squares method with mixed regularization is used to recover the medium profile in the second stage. The total least-squares method is designed to minimize the residual of the model equation and the data fitting, along with an appropriate regularization, in an attempt to significantly improve the accuracy of the approximation ob…
We propose a novel stochastic gradient descent method for solving linear least squares problems with partially observed data. Our method uses submatrices indexed by a randomly selected pair of row and column index sets to update the iterate at each step. Theoretical convergence guarantees in the mean square sense are provided. Numerical experiments are reported to demonstrate the theoretical findings.
We apply the recently developed least squares stabilized symmetric Nitsche method for enforcement of Dirichlet boundary conditions to the finite cell method. The least squares stabilized Nitsche method in combination with finite cell stabilization leads to a symmetric positive definite stiffness matrix and relies only on elementwise stabilization, which does not lead to additional fill in. We prove a priori error estimates and bounds on the condition numbers.
We study the approximation of the spectrum of least-squares operators arising from linear elasticity. We consider a two-field (stress/displacement) and a three-field (stress/displacement/vorticity) formulation; other formulations might be analyzed with similar techniques. We prove a priori estimates and we confirm the theoretical results with simple two-dimensional numerical experiments.
In this paper, a well-posed simultaneous space-time First Order System Least Squares formulation is constructed of the instationary incompressible Stokes equations with slip boundary conditions. As a consequence of this well-posedness, the minimization over any conforming triple of finite element spaces for velocities, pressure and stress tensor gives a quasi-best approximation from that triple. The formulation is practical in the sense that all norms in the least squares functional can be efficiently evaluated. Being of least squares type, the formulation comes with an efficient and reliable …
In this paper, we present a progressive and iterative approximation method with memory for least square fitting(MLSPIA). It adjusts the control points and the weighted sums iteratively to construct a series of fitting curves (surfaces) with three weights. For any normalized totally positive basis even when the collocation matrix is of deficient column rank, we obtain a condition to guarantee that these curves (surfaces) converge to the least square fitting curve (surface) to the given data points. It is proved that the theoretical convergence rate of the method is faster than the one of the pr…
We propose a new least squares finite element method to solve the Stokes problem with two sequential steps. The approximation spaces are constructed by patch reconstruction with one unknown per element. For the first step, we reconstruct an approximation space consisting of piecewise curl-free polynomials with zero trace. By this space, we minimize a least squaresfunctional toobtain thenumericalapproximationstothe gradientof thevelocityand the pressure. In the second step, we minimize another least squares functional to give the solution to the velocity in the reconstructed piecewise divergenc…
Stretched Gaussian distribution is the fundamental solution of the Hausdorff derivative diffusion equation and its corresponding stretched Gaussian noise is a widely encountered non-Gaussian noise in science and engineering. The least square method is a standard regression approach to fit Gaussian noisy data, but has distinct limits for non-Gaussian noise. Based on the Hausdorff calculus, this study develops a stretched least square method to fit stretched Gaussian noise by using the Hausdorff fractal distance as the horizontal coordinate. To better compare with the least square method, differ…
In this paper we formulate and solve a robust least squares problem for a system of linear equations subject to quantization error in the data matrix. Ordinary least squares fails to consider uncertainty in the operator, modeling all noise in the observed signal. Total least squares accounts for uncertainty in the data matrix, but necessarily increases the condition number of the operator compared to ordinary least squares. Tikhonov regularization or ridge regression is frequently employed to combat ill-conditioning, but requires parameter tuning which presents a host of challenges and places …
The simulation of complex biological systems such as the description of blood flow in organs requires a lot of computational power as well as a detailed description of the organ physiology. We present a novel Least-Squares discretization method for the simulation of sinusoidal blood flow in liver lobules using a porous medium approach for the liver tissue. The scaling of the different Least-Squares terms leads to a robust algorithm and the inherent error estimator provides an efficient refinement strategy.
A few iterations of alternating least squares with a random starting point provably suffice to produce nearly optimal spectral- and Frobenius-norm accuracies of low-rank approximations to a matrix; iterating to convergence is unnecessary. Thus, software implementing alternating least squares can be retrofitted via appropriate setting of parameters to calculate nearly optimally accurate low-rank approximations highly efficiently, with no need for convergence.
We introduce and analyze a space-time least-squares method associated to the unsteady Navier-Stokes system. Weak solution in the two dimensional case and regular solution in the three dimensional case are considered. From any initial guess, we construct a minimizing sequence for the least-squares functional which converges strongly to a solution of the Navier-Stokes system. After a finite number of iterates related to the value of the viscosity constant, the convergence is quadratic. Numerical experiments within the two dimensional case support our analysis. This globally convergent least-squa…
