arXivDaily arXiv每日学术速递 周一至周五更新
arXiv math/0110103math.PRcs.NAmath.NAmath.OCmath.RAmath.STq-biostat.TH
收藏

广义尖峰过程、稀疏性与统计独立性

The Generalized Spike Process, Sparsity, and Statistical Independence

Naoki Saito

AI总结:

本文针对广义尖峰过程,证明在正交基下最佳稀疏化基与最大化峰度基均为标准基,在更广变换下前者仍存在而后者不存在,支持BSB优于LSDB/KMB用于数据压缩。

摘要:

一个基,在该基下给定随机过程的一组实现可以被最稀疏地表示(所谓的“最佳稀疏化基”(BSB)),以及另一个基,在该基下这样一组实现变得尽可能统计独立(所谓的“最小统计依赖基”(LSDB)),对于数据压缩非常重要,并引起了计算神经科学家和应用数学家的兴趣。在这里,我们考虑这些基对于一个特别简单的随机过程,称为“广义尖峰过程”,该过程在每次实现中,在长度为$\ndim$的零向量的随机位置放置一个单一尖峰,其幅度从标准正态分布中采样。与我们在先前论文中处理的“简单尖峰过程”不同,其幅度是恒定的,由于评估广义尖峰过程的微分熵和互信息的困难,我们需要考虑最大化峰度的基(KMB)而不是LSDB。通过计算边际密度和矩,我们证明:1)如果我们将基搜索限制在${\mathbb R}^n$中的所有可能正交基内,则BSB和KMB选择标准基;2)如果我们将基搜索扩展到所有可能的保体积可逆线性变换,则BSB存在且再次是标准基,而KMB不存在。因此,KMB对所考虑的变换的正交性相当敏感,而BSB对此不敏感。我们的结果再次支持了我们在先前工作中对BSB相对于LSDB/KMB在数据压缩应用中的偏好。

英文摘要:

A basis under which a given set of realizations of a stochastic process can be represented most sparsely (the so-called best sparsifying basis (BSB)) and the one under which such a set becomes as less statistically dependent as possible (the so-called least statistically-dependent basis (LSDB)) are important for data compression and have generated interests among computational neuroscientists as well as applied mathematicians. Here we consider these bases for a particularly simple stochastic process called ``generalized spike process'', which puts a single spike--whose amplitude is sampled from the standard normal distribution--at a random location in the zero vector of length $\ndim$ for each realization. Unlike the ``simple spike process'' which we dealt with in our previous paper and whose amplitude is constant, we need to consider the kurtosis-maximizing basis (KMB) instead of the LSDB due to the difficulty of evaluating differential entropy and mutual information of the generalized spike process. By computing the marginal densities and moments, we prove that: 1) the BSB and the KMB selects the standard basis if we restrict our basis search within all possible orthonormal bases in ${\mathbb R}^n$; 2) if we extend our basis search to all possible volume-preserving invertible linear transformations, then the BSB exists and is again the standard basis whereas the KMB does not exist. Thus, the KMB is rather sensitive to the orthonormality of the transformations under consideration whereas the BSB is insensitive to that. Our results once again support the preference of the BSB over the LSDB/KMB for data compression applications as our previous work did.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑