Considerable efforts have been devoted to statistical modeling and the characterization of channels in a range of statistical models for fading channels. In this paper, we consider a unified approach to model wireless channels by the mixture of Gaussian (MoG) distribution. Simulations provided have shown the new probability density function to accurately characterize multipath fading as well as composite fading channels. We utilize the well known expectation-maximization algorithm to estimate the parameters of the MoG model and further utilize the Kullback-Leibler divergence and the mean square error criteria to demonstrate that our model provides both high accuracy and low computational complexity, in comparison with existing results. Additionally, we provide closed form expressions for several performance metrics used in wireless communication systems, including the moment generating function, the raw moments, the amount of fading, the outage probability, the average channel capacity, and the probability of energy detection for cognitive radio. Numerical Analysis and Monte-Carlo simulations are presented to corroborate the analytical results and to provide detailed performance comparisons with the other models in the literature.
A belief-propagation decoder for low-density lattice codes is given which represents messages explicitly as a mixture of Gaussians functions. The key component is an algorithm for approximating a mixture of several Gaussians with another mixture with a smaller number of Gaussians. This Gaussian mixture reduction algorithm iteratively reduces the number of Gaussians by minimizing the distance between the original mixture and an approximation with one fewer Gaussians. Error rates and noise thresholds of this decoder are compared with those for the previously-proposed decoder which discretely qua…
This paper presents a quantified assessment of the physical layer security capabilities of reconfigurable intelligent surface (RIS)-aided wireless systems under eavesdropping. Specifically, we derive a closed-form expression for the ergodic secrecy capacity (ESC) that is adaptable to different types of fading and RIS size. The channels between the transmitter (TX) and RIS, the RIS and legitimate receiver as well as the TX and eavesdropper are assumed to follow independent mixture Gamma (MG) distributions. Note that MG is capable of modeling a large variety of well-known distributions, includin…
Underwater wireless optical communication is one of the critical technologies for buoy-based high-speed cross-sea surface communication, where the communication nodes are vertically deployed. Due to the vertically inhomogeneous nature of the underwater environment, seawater is usually vertically divided into multiple layers with different parameters that reflect the real environment. In this work, we consider a generalized UWOC channel model that contains$N$ layers. To capture the effects of air bubbles and temperature gradients on channel statistics, we model each layer by a mixture Exponenti…
In this work, we present a unified framework for the performance analysis of dual-hop underwater wireless optical communication (UWOC) systems with amplify-and-forward fixed gain relays in the presence of air bubbles and temperature gradients. Operating under either heterodyne detection or intensity modulation with direct detection, the UWOC is modeled by the unified mixture Exponential-Generalized Gamma distribution that we have proposed based on an experiment conducted in an indoor laboratory setup and has been shown to provide an excellent fit with the measured data under the considered lab…
Consider the estimation of a signal ${\bf x}\in\mathbb{R}^N$ from noisy observations ${\bf r=x+z}$, where the input~${\bf x}$ is generated by an independent and identically distributed (i.i.d.) Gaussian mixture source, and ${\bf z}$ is additive white Gaussian noise (AWGN) in parallel Gaussian channels. Typically, the $\ell_2$-norm error (squared error) is used to quantify the performance of the estimation process. In contrast, we consider the $\ell_\infty$-norm error (worst case error). For this error metric, we prove that, in an asymptotic setting where the signal dimension $N\to\infty$, the …
This paper shows that the normalized maximum likelihood~(NML) code-length calculated in [1] is an upper bound on the NML code-length strictly calculated for the Gaussian Mixture Model. When we use this upper bound on the NML code-length, we must change the scale of the data sequence to satisfy the restricted domain. However, we also show that the algorithm for model selection is essentially universal, regardless of the scale conversion of the data in Gaussian Mixture Models, and that, consequently, the experimental results in [1] can be used as they are. In addition to this, we correct the NML…
Bayesian filtering is a general framework for recursively estimating the state of a dynamical system. Classical solutions such that Kalman filter and Particle filter are introduced in this report. Gaussian processes have been introduced as a non-parametric technique for system estimation from supervision learning. For the thesis project, we intend to propose a new, general methodology for inference and learning in non-linear state-space models probabilistically incorporating with the Gaussian process model estimation.
A lattice decoder which represents messages explicitly as a mixture of Gaussians functions is given. In order to prevent the number of functions in a mixture from growing as the decoder iterations progress, a method for replacing N Gaussian functions with M Gaussian functions, with M < N, is given. A squared distance metric is used to select functions for combining. A pair of selected Gaussians is replaced by a single Gaussian with the same first and second moments. The metric can be computed efficiently, and at the same time, the proposed algorithm empirically gives good results, for example,…
WDMoE: Wireless Distributed Large Language Models with Mixture of Experts
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学合作媒介网络创新中心）；Pengcheng Laboratory（鹏城实验室）；Future Network of Intelligent Institute (FNii), the Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）未来网络智能研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）
This paper investigates the problem of Gaussian approximation for the wireless multi-access interference distribution in large spatial wireless networks. First, a principled methodology is presented to establish rates of convergence of the multi-access interference distribution to a Gaussian distribution for general bounded and power-law decaying path-loss functions. The model is general enough to also include various random wireless channel dynamics such as fading and shadowing arising from multipath propagation and obstacles existing in the communication environment. It is shown that the wir…
WDMoE: Wireless Distributed Mixture of Experts for Large Language Models
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学合作中位数网络创新中心）；Pengcheng Laboratory（鹏城实验室）；Future Network of Intelligent Institute (FNii), the Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）未来网络智能研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）
In this paper, the statistical properties of the product of independent and non-identically distributed mixture Gamma (MG) random variables (RVs) are provided first. Specifically, simple exact closed-form expressions for the probability density function (PDF), cumulative distribution function (CDF), and moment generating function (MGF) are derived in terms of univariate Meijer's $G$-function. The statistical characterisations of the distribution of the ratio of products of MG variates are then derived. These statistics are used to analyse the outage probability (OP), the average error probabil…
We show, by an explicit construction, that a mixture of univariate Gaussian densities with variance $1$ and means in $[-A,A]$ can have $Ω(A^2)$ modes. This disproves a recent conjecture of Dytso, Yagli, Poor and Shamai \cite{DYPS20} who showed that such a mixture can have at most $O(A^{2})$ modes and surmised that the upper bound could be improved to $O(A)$. Our result holds even if an additional variance constraint is imposed on the mixing distribution. Extending the result to higher dimensions, we exhibit a mixture of Gaussians in $\mathbb{R}^{d}$, with identity covariances and means inside …
A symmetric random variable is called a Gaussian mixture if it has the same distribution as the product of two independent random variables, one being positive and the other a standard Gaussian random variable. Examples of Gaussian mixtures include random variables with densities proportional to $e^{-|t|^p}$ and symmetric $p$-stable random variables, where $p\in(0,2]$. We obtain various sharp moment and entropy comparison estimates for weighted sums of independent Gaussian mixtures and investigate extensions of the B-inequality and the Gaussian correlation inequality in the context of Gaussian…
Gauss--Hermite Quadrature for Gaussian-Mixture Entropy with an Action-Space Hermite Surrogate
Extreme Materials Research Center, Korea Institute of Science and Technology（韩国科学技术研究院极端材料研究中心）；Climate and Environmental Research Institute, Korea Institute of Science and Technology（韩国科学技术研究院气候与环境研究所）；Division of AI-Robotics, KIST Campus, University of Science and Technology（科学技术大学KIST校区AI机器人系）
Sony Computer Science Laboratories, Inc.（索尼计算机科学实验室有限公司）；Tokyo（东京）
The traditional Minkowski distances are induced by the corresponding Minkowski norms in real-valued vector spaces. In this work, we propose novel statistical symmetric distances based on the Minkowski's inequality for probability densities belonging to Lebesgue spaces. These statistical Minkowski distances admit closed-form formula for Gaussian mixture models when parameterized by integer exponents. This result extends to arbitrary mixtures of exponential families with natural parameter spaces being cones: This includes the binomial, the multinomial, the zero-centered Laplacian, the Gaussian a…
Northwestern University（西北大学）；Halicioǧlu Data Science Institute, UC San Diego（加州大学圣地亚哥分校哈利西奥卢数据科学研究所）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；UC San Diego（加州大学圣地亚哥分校）
Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory. While the total variation distance appears naturally in the sample complexity of distribution learning, it is analytically difficult to obtain tight lower bounds for mixtures. Exploiting a connection between total variation distance and the characteristic function of the mixture, we provide fairly tight functional approximations. This enables us to derive new lower bounds on the total variation distance between pairs of two-component Gaussian mixtures that have a shared covarian…
In this paper, we develop an unsupervised generative clustering framework that combines the Variational Information Bottleneck and the Gaussian Mixture Model. Specifically, in our approach, we use the Variational Information Bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) and provide a variational inference type algorithm that allows computing it. In the algorithm, the coders' mappings are parametrized using neural networks, and the bound is approximated by Monte Carlo s…
We determine the information-theoretic cutoff value on separation of cluster centers for exact recovery of cluster labels in a $K$-component Gaussian mixture model with equal cluster sizes. Moreover, we show that a semidefinite programming (SDP) relaxation of the $K$-means clustering method achieves such sharp threshold for exact recovery without assuming the symmetry of cluster centers.
University of Waterloo（滑铁卢大学）；Bar Ilan University（巴伊兰大学）；Northeastern University（东北大学）
Learning the parameters of Gaussian mixture models is a fundamental and widely studied problem with numerous applications. In this work, we give new algorithms for learning the parameters of a high-dimensional, well separated, Gaussian mixture model subject to the strong constraint of differential privacy. In particular, we give a differentially private analogue of the algorithm of Achlioptas and McSherry. Our algorithm has two key properties not achieved by prior work: (1) The algorithm's sample complexity matches that of the corresponding non-private algorithm up to lower order terms in a wi…
