On $w$-mixtures: Finite convex combinations of prescribed component distributions

A Bagging and Boosting Based Convexly Combined Optimum Mixture Probabilistic Model

Unlike previous studies on mixture distributions, a bagging and boosting based convexly combined mixture probabilistic model has been suggested. This model is a result of iteratively searching for obtaining the optimum probabilistic model that provides the maximum p value.

A Deterministic Analysis of an Online Convex Mixture of Expert Algorithms

We analyze an online learning algorithm that adaptively combines outputs of two constituent algorithms (or the experts) running in parallel to model an unknown desired signal. This online learning algorithm is shown to achieve (and in some cases outperform) the mean-square error (MSE) performance of the best constituent algorithm in the mixture in the steady-state. However, the MSE analysis of this algorithm in the literature uses approximations and relies on statistical models on the underlying signals and systems. Hence, such an analysis may not be useful or valid for signals generated by va…

A Tight Convex Upper Bound on the Likelihood of a Finite Mixture

The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we ask: is it possible to assess how far we are from the global maximum of the likelihood? Since the likelihood of a finite mixture model can grow unboundedly by centering a Gaussian on a single datapoint and shrinking the covariance, we constrain the problem by assuming that the parameters of the individual models are members of a large discrete set (e.g. estim…

MixMin: 通过凸优化寻找数据混合

MixMin: Finding Data Mixtures via Convex Minimization

MixMin通过凸优化方法改进数据混合，提升了语言模型和化学任务的性能，且在不同规模模型上均有效。

Estimating Mixture Models via Mixtures of Polynomials

Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of a mixture model is non-convex, so EM has no known global convergence guarantees. Recently, method of moments approaches offer global guarantees for some mixture models, but they do not extend easily to the range of mixture models that exist. In this work, we present Polymom, an …

凸集的比率覆盖与最优混合密度估计

Ratio Covers of Convex Sets and Optimal Mixture Density Estimation

本文研究了基于Kullback-Leibler散度的密度估计问题，提出在有限密度字典下，通过凸集覆盖理论和混合密度估计方法，获得最优的高概率保证，改进了传统密度比有界时的估计率。