Combining the classical theory of optimal transport with modern operator splitting techniques, we develop a new numerical method for nonlinear, nonlocal partial differential equations, arising in models of porous media, materials science, and biological swarming. Our method proceeds as follows: First, we discretize in time, either via the classical JKO scheme or via a novel Crank-Nicolson type method we introduce. Next, we use the Benamou-Brenier dynamical characterization of the Wasserstein distance to reduce computing the solution of the discrete time equations to solving fully discrete minimization problems, with strictly convex objective functions and linear constraints. Third, we compute the minimizers by applying a recently introduced, provably convergent primal dual splitting scheme for three operators [Yan 2018].
By leveraging the PDEs' underlying variational structure, our method overcomes stability issues present in previous numerical work built on explicit time discretizations, which suffer due to the equations' strong nonlinearities and degeneracies. Our method is also naturally positivity and mass preserving and, in the case of the JKO scheme, energy decreasing. We prove that minimizers of the fully discrete problem converge to minimizers of the spatially continuous, discrete time problem as the spatial discretization is refined. We conclude with simulations of nonlinear PDEs and Wasserstein geodesics in one and two dimensions that illustrate the key properties of our approach, including higher order convergence our novel Crank-Nicolson type method, when compared to the classical JKO method.
This article devises a new primal-dual weak Galerkin finite element method for the convection-diffusion equation. Optimal order error estimates are established for the primal-dual weak Galerkin approximations in various discrete norms and the standard $L^2$ norms. A series of numerical experiments are conducted and reported to verify the theoretical findings.
A new primal-dual weak Galerkin (PDWG) finite element method is introduced and analyzed for the ill-posed elliptic Cauchy problems with ultra-low regularity assumptions on the exact solution. The Euler-Lagrange formulation resulting from the PDWG scheme yields a system of equations involving both the primal equation and the adjoint (dual) equation. The optimal order error estimate for the primal variable in a low regularity assumption is established. A series of numerical experiments are illustrated to validate effectiveness of the developed theory.
In this paper, with the parametric symmetric coercive elliptic boundary value problem as an example of the primal-dual variational problems satisfying the strong duality, we develop primal-dual reduced basis methods (PD-RBM) with robust true error certifications and discuss three versions of greedy algorithms to balance the finite element error, the exact reduced basis error, and the adaptive mesh refinements. For a class of convex minimization variational problems which has corresponding dual problems satisfying the strong duality, the primal-dual gap between the primal and dual functionals c…
In this article, the authors present a new $L^p$- primal-dual weak Galerkin method ($L^p$-PDWG) for convection-diffusion equations with $p>1$. The existence and uniqueness of the numerical solution is discussed, and an optimal-order error estimate is derived in the $L^q$-norm for the primal variable, where $\frac 1p+\frac 1q=1$. Furthermore, error estimates are established for the numerical approximation of the dual variable in the standard $W^{m,p}$ norm, $0\le m\le 2$. Numerical results are presented to demonstrate the efficiency and accuracy of the proposed $L^p$-PDWG method.
A new primal-dual weak Galerkin (PD-WG) finite element method was developed and analyzed in this article for first-order linear convection equations in non-divergence form. The PD-WG method results in a symmetric discrete system involving not only the original equation for the primal variable, but also the dual/adjoint equation for the dual variable (also known as Lagrangian multiplier). Optimal order of error estimates in various discrete Sobolev norms are derived for the numerical solutions arising from the PD-WG scheme. Numerical results are produced and reported to illustrate the accuracy …
A new $L^p$-primal-dual weak Galerkin method ($L^p$-PDWG) with $p>1$ is proposed for the first-order transport problems. The existence and uniqueness of the $L^p$-PDWG numerical solutions is established. In addition, the $L^p$-PDWG method offers a numerical solution which retains mass conservation locally on each element. An optimal order error estimate is established for the primal variable. A series of numerical results are presented to verify the efficiency and accuracy of the proposed $L^p$-PDWG scheme.
Focus of this work is solving a non-smooth constraint minimization problem by a primal-dual splitting algorithm involving proximity operators. The problem is penalized by the Bregman divergence associated with the non-smooth total variation (TV) functional. We analyse two aspects: Firstly, the convergence of the regularized solution of the minimization problem to the minimum norm solution. Second, the convergence of the iteratively regularized minimizer to the minimum norm solution by a primal-dual algorithm. For both aspects, we use the assumption of a variational source condition (VSC). This…
A mixed finite element method for nearly incompressible elasticity and Stokes equations using primal and dual meshes with quadrilateral and hexahedral grids
We present a method to efficiently compute Wasserstein gradient flows. Our approach is based on a generalization of the back-and-forth method (BFM) introduced by Jacobs and Léger to solve optimal transport problems. We evolve the gradient flow by solving the dual problem to the JKO scheme. In general, the dual problem is much better behaved than the primal problem. This allows us to efficiently run large-scale simulations for a large class of internal energies including singular and non-convex energies.
We consider a primal-dual algorithm for minimizing $f(x)+h\square l(Ax)$ with Fréchet differentiable $f$ and $l^*$. This primal-dual algorithm has two names in literature: Primal-Dual Fixed-Point algorithm based on the Proximity Operator (PDFP$^2$O) and Proximal Alternating Predictor-Corrector (PAPC). In this paper, we prove its convergence under a weaker condition on the stepsizes than existing ones. With additional assumptions, we show its linear convergence. In addition, we show that this condition (the upper bound of the stepsize) is tight and can not be weakened. This result also recovers…
Dual first-order methods are essential techniques for large-scale constrained convex optimization. However, when recovering the primal solutions, we need $T(ε^{-2})$ iterations to achieve an $ε$-optimal primal solution when we apply an algorithm to the non-strongly convex dual problem with $T(ε^{-1})$ iterations to achieve an $ε$-optimal dual solution, where $T(x)$ can be $x$ or $\sqrt{x}$. In this paper, we prove that the iteration complexity of the primal solutions and dual solutions have the same $O\left(\frac{1}{\sqrtε}\right)$ order of magnitude for the accelerated randomized dual coordin…
The primal-dual method of Chambolle and Pock is a widely used algorithm to solve various optimization problems written as convex-concave saddle point problems. Each update step involves the application of both the forward linear operator and its adjoint. However, in practical applications like computerized tomography, it is often computationally favourable to replace the adjoint operator by a computationally more efficient approximation. This leads to an adjoint mismatch in the algorithm. In this paper, we analyze the convergence of Chambolle-Pock's primal-dual method under the presence of a m…
We develop block structure adapted primal-dual algorithms for non-convex non-smooth optimisation problems whose objectives can be written as compositions $G(x)+F(K(x))$ of non-smooth block-separable convex functions $G$ and $F$ with a non-linear Lipschitz-differentiable operator $K$. Our methods are refinements of the non-linear primal-dual proximal splitting method for such problems without the block structure, which itself is based on the primal-dual proximal splitting method of Chambolle and Pock for convex problems. We propose individual step length parameters and acceleration rules for ea…
The Stochastic Primal-Dual Hybrid Gradient (SPDHG) was proposed by Chambolle et al. (2018) and is an efficient algorithm to solve some nonsmooth large-scale optimization problems. In this paper we prove its almost sure convergence for convex but not necessarily strongly convex functionals. We also look into its application to parallel Magnetic Resonance Imaging reconstruction in order to test performance of SPDHG. Our numerical results show that for a range of settings SPDHG converges significantly faster than its deterministic counterpart.
We study structured nonsmooth convex finite-sum optimization that appears widely in machine learning applications, including support vector machines and least absolute deviation. For the primal-dual formulation of this problem, we propose a novel algorithm called \emph{Variance Reduction via Primal-Dual Accelerated Dual Averaging (\vrpda)}. In the nonsmooth and general convex setting, \vrpda~has the overall complexity $O(nd\log\min \{1/ε, n\} + d/ε)$ in terms of the primal-dual gap, where $n$ denotes the number of samples, $d$ the dimension of the primal variables, and $ε$ the desired accuracy…
