On Suboptimality of Least Squares with Application to Estimation of Convex Bodies

Suboptimality of Penalized Empirical Risk Minimization in Classification

Let $\cF$ be a set of $M$ classification procedures with values in $[-1,1]$. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $\cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various types of convexity, we prove that optimal rates of aggregation can be either $((\log M)/n)^{1/2}$ or $(\log M)/n$. We prove that, if all the $M$ classifiers are binary, the (penalized) Empirical Risk Minimization procedures are suboptimal (even under the margin/low noise conditio…

随机凸优化中自适应的代价

The Price of Adaptivity in Stochastic Convex Optimization

本文证明了非光滑随机凸优化中自适应的不可能性结果，定义了“自适应代价”（PoA）来衡量参数不确定性导致的次优性乘性增加，并给出了在距离或梯度范数不确定时PoA的下界，几乎匹配现有上界。

On the Suboptimality of Negative Momentum for Minimax Optimization

Smooth game optimization has recently attracted great interest in machine learning as it generalizes the single-objective optimization paradigm. However, game dynamics is more complex due to the interaction between different players and is therefore fundamentally different from minimization, posing new challenges for algorithm design. Notably, it has been shown that negative momentum is preferred due to its ability to reduce oscillation in game dynamics. Nevertheless, the convergence rate of negative momentum was only established in simple bilinear games. In this paper, we extend the analysis …

多维凸回归：最小二乘估计量的次优性

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

本文证明当维数d≥5时，凸函数类上的最小二乘估计量在平方误差损失下达到n^{-2/d}速率，次优于n^{-4/(d+4)}的极小极大速率，并建立了无约束凸LSE在多面体域固定设计下的首个收敛速率结果。

Reconstruction of convex bodies from moments

We investigate how much information about a convex body can be retrieved from a finite number of its geometric moments. We give a sufficient condition for a convex body to be uniquely determined by a finite number of its geometric moments, and we show that among all convex bodies, those which are uniquely determined by a finite number of moments form a dense set. Further, we derive a stability result for convex bodies based on geometric moments. It turns out that the stability result is improved considerably by using another set of moments, namely Legendre moments. We present a reconstruction …