This work presents study on regularized and non-regularized versions of perceptron learning and least squares algorithms for classification problems. Fr'echet derivatives for regularized least squares and perceptron learning algorithms are derived. Different Tikhonov's regularization techniques for choosing the regularization parameter are discussed. Decision boundaries obtained by non-regularized algorithms to classify simulated and experimental data sets are analyzed.
A new closed formula for the first order perturbation estimate of the mixed least squares-total least squares (MTLS) solution is presented. It is mathematically equivalent to the one by Zheng and Yang(Numer. Linear Algebra Appl. 2019; 26(4):e2239). With this formula, general and structured normwise, mixed and componentwise condition numbers of the MTLS problem are derived. Perturbation bounds based on the normwise condition number, and compact forms for the upper bounds of mixed and componentwise condition numbers are also given in order for economic storage and efficient computation. It is sh…
This paper studies adaptive least-squares finite element methods for convection-dominated diffusion-reaction problems. The least-squares methods are based on the first-order system of the primal and dual variables with various ways of imposing outflow boundary conditions. The coercivity of the homogeneous least-squares functionals are established, and the a priori error estimates of the least-squares methods are obtained in a norm that incorporates the streamline derivative. All methods have the same convergence rate provided that meshes in the layer regions are fine enough. To increase comput…
Least-Squares versus Partial Least-Squares Finite Element Methods: Robust A Priori and A Posteriori Error Estimates of Augmented Mixed Finite Element Methods
Total least squares (TLS) methods have been widely used in data fitting. Compared with the least squares method, for TLS problem we takes into account not only the observation errors, but also the errors in the measurement matrix. This is more realistic in practical applications. For the large-scale discrete ill-posed problem $Ax \approx b$, we introduce the quantum-inspired techniques to approximate the truncated total least squares (TTLS) solution. We analyze the accuracy of the quantum-inspired truncated total least squares algorithm and perform numerical experiments to demonstrate the effi…
Given a sequence of Marcinkiewicz-Zygmund inequalities in $L_2$ on a usual compact space $\mathcal M$, Gröchenig introduced the weighted least squares polynomials and the least squares quadrature from pointwise samples of a function, and obtained approximation theorems and quadrature errors. In this paper we obtain approximation theorems and quadrature errors on the sphere which are optimal. We also give upper bounds of the operator norms of the weighted least squares operators.
We define and analyse a least-squares finite element method for a first-order reformulation of a scaled Brinkman model of fluid flow through porous media. We introduce a pseudostress variable that allows to eliminate the pressure variable from the system. It can be recovered by a simple post-processing. It is shown that the least-squares functional is uniformly equivalent, i.e., independent of the singular perturbation parameter, to a parameter dependent norm. This norm equivalence implies that the least-squares functional evaluated in the discrete solution provides an efficient and reliable a…
In this article, we address the solution of advection-dominated flow problems by stabilised methods, by means of least-squares computed stabilised coefficients. As main methodological tool, we introduce a data-driven off-line/on-line strategy to compute them with low computational cost. We compare the errors provided by the least-squares stabilised coefficients to those provided by several previously established stabilised coefficients within the solution of advection-diffusion and Navier-Stokes flows, on structured and un-structured grids, with and Lagrange Finite Elements up to third degree …
The alternating least squares algorithm for CP and Tucker decomposition is dominated in cost by the tensor contractions necessary to set up the quadratic optimization subproblems. We introduce a novel family of algorithms that uses perturbative corrections to the subproblems rather than recomputing the tensor contractions. This approximation is accurate when the factor matrices are changing little across iterations, which occurs when alternating least squares approaches convergence. We provide a theoretical analysis to bound the approximation error. Our numerical experiments demonstrate that t…
In this paper we discuss spectral properties of operators associated with the least-squares finite element approximation of elliptic partial differential equations. The convergence of the discrete eigenvalues and eigenfunctions towards the corresponding continuous eigenmodes is studied and analyzed with the help of appropriate $L^2$ error estimates. A priori and a posteriori estimates are proved.
We present a two-stage least-squares method to inverse medium problems of reconstructing multiple unknown coefficients simultaneously from noisy data. A direct sampling method is applied to detect the location of the inhomogeneity in the first stage, while a total least-squares method with mixed regularization is used to recover the medium profile in the second stage. The total least-squares method is designed to minimize the residual of the model equation and the data fitting, along with an appropriate regularization, in an attempt to significantly improve the accuracy of the approximation ob…
We propose a novel stochastic gradient descent method for solving linear least squares problems with partially observed data. Our method uses submatrices indexed by a randomly selected pair of row and column index sets to update the iterate at each step. Theoretical convergence guarantees in the mean square sense are provided. Numerical experiments are reported to demonstrate the theoretical findings.
We apply the recently developed least squares stabilized symmetric Nitsche method for enforcement of Dirichlet boundary conditions to the finite cell method. The least squares stabilized Nitsche method in combination with finite cell stabilization leads to a symmetric positive definite stiffness matrix and relies only on elementwise stabilization, which does not lead to additional fill in. We prove a priori error estimates and bounds on the condition numbers.
We study the approximation of the spectrum of least-squares operators arising from linear elasticity. We consider a two-field (stress/displacement) and a three-field (stress/displacement/vorticity) formulation; other formulations might be analyzed with similar techniques. We prove a priori estimates and we confirm the theoretical results with simple two-dimensional numerical experiments.
In this paper, a well-posed simultaneous space-time First Order System Least Squares formulation is constructed of the instationary incompressible Stokes equations with slip boundary conditions. As a consequence of this well-posedness, the minimization over any conforming triple of finite element spaces for velocities, pressure and stress tensor gives a quasi-best approximation from that triple. The formulation is practical in the sense that all norms in the least squares functional can be efficiently evaluated. Being of least squares type, the formulation comes with an efficient and reliable …
In this paper, we present a progressive and iterative approximation method with memory for least square fitting(MLSPIA). It adjusts the control points and the weighted sums iteratively to construct a series of fitting curves (surfaces) with three weights. For any normalized totally positive basis even when the collocation matrix is of deficient column rank, we obtain a condition to guarantee that these curves (surfaces) converge to the least square fitting curve (surface) to the given data points. It is proved that the theoretical convergence rate of the method is faster than the one of the pr…
We propose a new least squares finite element method to solve the Stokes problem with two sequential steps. The approximation spaces are constructed by patch reconstruction with one unknown per element. For the first step, we reconstruct an approximation space consisting of piecewise curl-free polynomials with zero trace. By this space, we minimize a least squaresfunctional toobtain thenumericalapproximationstothe gradientof thevelocityand the pressure. In the second step, we minimize another least squares functional to give the solution to the velocity in the reconstructed piecewise divergenc…
We present a reduced basis (RB) method for parametrized linear elliptic partial differential equations (PDEs) in a least-squares finite element framework. A rigorous and reliable error estimate is developed, and is shown to bound the error with respect to the exact solution of the PDE, in contrast to estimates that measure error with respect to a finite-dimensional (high-fidelity) approximation. It is shown that the first-order formulation of the least-squares finite element is a key ingredient. The method is demonstrated using numerical examples.
In this paper, within a unified framework of the condition number theory we present the explicit expression of the projected condition number of the equality constrained indefinite least squares problem. By setting specific norms and parameters, some widely used condition numbers, like the normwise, mixed and componentwise condition numbers follow as its special cases. Considering practical applications and computation, some new compact forms or upper bounds of the projected condition numbers are given to improve the computational efficiency. The new compact forms are of particular interest in…
Localized collocation methods based on radial basis functions (RBFs) for elliptic problems appear to be non-robust in the presence of Neumann boundary conditions. In this paper we overcome this issue by formulating the RBF-generated finite difference method in a discrete least-squares setting instead. This allows us to prove high-order convergence under node refinement and to numerically verify that the least-squares formulation is more accurate and robust than the collocation formulation. The implementation effort for the modified algorithm is comparable to that for the collocation method.
This paper is devoted to condition numbers of the multidimensional total least squares problem with linear equality constraint (TLSE). Based on the perturbation theory of invariant subspace, the TLSE problem is proved to be equivalent to a multidimensional unconstrained weighed total least squares problem in the limit sense. With a limit technique, Kronecker-product-based formulae for normwise, mixed and componentwise condition numbers of the minimum Frobenius norm TLSE solution are given. Compact upper bounds of these condition numbers are provided to reduce the storage and computation cost. …
We propose and analyze a discontinuous least squares finite element method for solving the indefinite time-harmonic Maxwell equations. The scheme is based on the $L^2$ norm least squares functional with the weak imposition of the continuity across the interior faces. We minimize the functional over the piecewise polynomial spaces to seek numerical solutions. The method is shown to be stable without any constraint on the mesh size. We prove the convergence orders under both the energy norm and the $L^2$ norm. Numerical results in two and three dimensions are presented to verify the error estima…
Stretched Gaussian distribution is the fundamental solution of the Hausdorff derivative diffusion equation and its corresponding stretched Gaussian noise is a widely encountered non-Gaussian noise in science and engineering. The least square method is a standard regression approach to fit Gaussian noisy data, but has distinct limits for non-Gaussian noise. Based on the Hausdorff calculus, this study develops a stretched least square method to fit stretched Gaussian noise by using the Hausdorff fractal distance as the horizontal coordinate. To better compare with the least square method, differ…
In this paper we formulate and solve a robust least squares problem for a system of linear equations subject to quantization error in the data matrix. Ordinary least squares fails to consider uncertainty in the operator, modeling all noise in the observed signal. Total least squares accounts for uncertainty in the data matrix, but necessarily increases the condition number of the operator compared to ordinary least squares. Tikhonov regularization or ridge regression is frequently employed to combat ill-conditioning, but requires parameter tuning which presents a host of challenges and places …
The simulation of complex biological systems such as the description of blood flow in organs requires a lot of computational power as well as a detailed description of the organ physiology. We present a novel Least-Squares discretization method for the simulation of sinusoidal blood flow in liver lobules using a porous medium approach for the liver tissue. The scaling of the different Least-Squares terms leads to a robust algorithm and the inherent error estimator provides an efficient refinement strategy.
We introduce and analyze a space-time least-squares method associated to the unsteady Navier-Stokes system. Weak solution in the two dimensional case and regular solution in the three dimensional case are considered. From any initial guess, we construct a minimizing sequence for the least-squares functional which converges strongly to a solution of the Navier-Stokes system. After a finite number of iterates related to the value of the viscosity constant, the convergence is quadratic. Numerical experiments within the two dimensional case support our analysis. This globally convergent least-squa…
This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine parameters of the deep neural network. There are various least-squares functionals for a partial differential equation. This paper focuses on the so-called first-order system least-squares (FOSLS) functional studied in [3], which is based on a first-order system of scalar second-orde…
