Convex Optimization on Functionals of Probability Densities

非光滑弱凸函数约束优化中单循环切换次梯度方法的预言复杂度

Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimization

本文首次分析了切换次梯度方法在非光滑弱凸函数约束非凸优化中的预言复杂度，证明该方法能以单循环达到与双循环方法相同的复杂度，且更易实现。

Comparing different subgradient methods for solving convex optimization problems with functional constraints

We consider the problem of minimizing a convex, nonsmooth function subject to a closed convex constraint domain. The methods that we propose are reforms of subgradient methods based on Metel--Takeda's paper [Optimization Letters 15.4 (2021): 1491-1504] and Boyd's works [Lecture notes of EE364b, Stanford University, Spring 2013-14, pp. 1-39]. While the former has complexity $\mathcal{O}(\varepsilon^{-2r})$ for all $r> 1$, the complexity of the latter is $\mathcal{O}(\varepsilon^{-2})$. We perform some comparisons between these two methods using several test examples.

Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks

Gradient flows are a powerful tool for optimizing functionals in general metric spaces, including the space of probabilities endowed with the Wasserstein metric. A typical approach to solving this optimization problem relies on its connection to the dynamic formulation of optimal transport and the celebrated Jordan-Kinderlehrer-Otto (JKO) scheme. However, this formulation involves optimization over convex functions, which is challenging, especially in high dimensions. In this work, we propose an approach that relies on the recently introduced input-convex neural networks (ICNN) to parametrize …

Constrained Functional Value under General Convexity Conditions with Applications to Distributed Simulation

We show a general phenomenon of the constrained functional value for densities satisfying general convexity conditions, which generalizes the observation in Bobkov and Madiman (2011) that the entropy per coordinate in a log-concave random vector in any dimension with given density at the mode has a range of just 1. Specifically, for general functions $ϕ$ and $ψ$, we derive upper and lower bounds of density functionals taking the form $I_ϕ(f) = \int_{\mathbb{R}^n} ϕ(f(x))dx$ assuming the convexity of $ψ^{-1}(f(x))$ for the density, and establish the tightness of these bounds under mild conditio…

How type of Convexity of the Core function affects the Csiszár $f$-divergence functional

We investigate how the type of Convexity of the Core function affects the Csiszár $f$-divergence functional. A general treatment for the type of convexity has been considered and the associated perspective functions have been studied. In particular, it has been shown that when the core function is \rm{MN}-convex, then the associated perspective function is jointly \rm{MN}-convex if the two scalar mean \rm{M} and \rm{N} are the same. In the case where $\mathrm{M}\neq\mathrm{N}$, we study the type of convexity of the perspective function. As an application, we prove that the \textit{Hellinger di…

Estimating divergence functionals and the likelihood ratio by convex risk minimization

We develop and analyze $M$-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a non-asymptotic variational characterization of $f$-divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve …