We propose a unified framework for establishing existence of nonparametric M-estimators, computing the corresponding estimates, and proving their strong consistency when the class of functions is exceptionally rich. In particular, the framework addresses situations where the class of functions is complex involving information and assumptions about shape, pointwise bounds, location of modes, height at modes, location of level-sets, values of moments, size of subgradients, continuity, distance to a "prior" function, multivariate total positivity, and any combination of the above. The class might be engineered to perform well in a specific setting even in the presence of little data. The framework views the class of functions as a subset of a particular metric space of upper semicontinuous functions under the Attouch-Wets distance. In addition to allowing a systematic treatment of numerous M-estimators, the framework yields consistency of plug-in estimators of modes of densities, maximizers of regression functions, level-sets of classifiers, and related quantities, and also enables computation by means of approximating parametric classes. We establish consistency through a one-sided law of large numbers, here extended to sieves, that relaxes assumptions of uniform laws, while ensuring global approximations even under model misspecification.
VarGrad: A Low-Variance Gradient Estimator for Variational Inference
Freie Universität Berlin(柏林自由大学);BTU Cottbus-Senftenberg(科特布斯-森夫滕贝格工业大学);dida Datenschmiede GmbH(迪达数据工坊有限公司);University of Warwick(华威大学);Universität Potsdam(波茨坦大学);DeepMind(深度思维);The Alan Turing Institute(艾伦·图灵研究所)
Consistency for constrained maximum likelihood estimation and clustering based on mixtures of elliptically-symmetric distributions under general data generating processes