arXivDaily arXiv每日学术速递 周一至周五更新
arXiv math/0104083math.NAcs.NAmath.STq-biostat.TH
收藏

自适应信号表示中的稀疏性与统计独立性:尖峰过程案例研究

Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process

Bertrand Benichou, Naoki Saito

AI总结:

本文通过尖峰过程案例研究,揭示在自适应信号表示中稀疏性与统计独立性最优基的差异及非唯一性,并证明高维下无法通过线性变换实现真正的统计独立性。

摘要:

在许多领域,包括数据压缩和计算神经科学中,通过将输入数据流视为随机过程的实现来寻找能够高效表示它们的基/坐标系具有极其重要的意义。衡量这种基效率的两个流行指标是稀疏性(通过期望的 $\ell^p$ 范数衡量,$0 < p \leq 1$)和统计独立性(通过互信息衡量)。然而,深入理解它们之间错综复杂的关系仍然难以实现。因此,我们选择研究一个简单的合成随机过程,称为尖峰过程,该过程在每次实现中于 $n$ 维向量的随机位置放置一个单位脉冲。对于该过程,我们获得了以下结果:1) 如果 $n \geq 5$ 且基的搜索限制在 $R^n$ 中所有可能的正交基内,则标准基在稀疏性和统计独立性方面都是最优的;2) 如果我们将基搜索扩展到 $R^n$ 中所有可能的可逆线性变换,则统计独立性最优的基与稀疏性最优的基不同;3) 在上述任何一种情况下,统计独立性最优的基都不是唯一的,甚至存在使输入完全稠密的基;4) 对于 $n > 2$,不存在能够实现真正统计独立性的线性可逆变换。

英文摘要:

Finding a basis/coordinate system that can efficiently represent an input data stream by viewing them as realizations of a stochastic process is of tremendous importance in many fields including data compression and computational neuroscience. Two popular measures of such efficiency of a basis are sparsity (measured by the expected $\ell^p$ norm, $0 < p \leq 1$) and statistical independence (measured by the mutual information). Gaining deeper understanding of their intricate relationship, however, remains elusive. Therefore, we chose to study a simple synthetic stochastic process called the spike process, which puts a unit impulse at a random location in an $n$-dimensional vector for each realization. For this process, we obtained the following results: 1) The standard basis is the best both in terms of sparsity and statistical independence if $n \geq 5$ and the search of basis is restricted within all possible orthonormal bases in $R^n$; 2) If we extend our basis search in all possible invertible linear transformations in $R^n$, then the best basis in statistical independence differs from the one in sparsity; 3) In either of the above, the best basis in statistical independence is not unique, and there even exist those which make the inputs completely dense; 4) There is no linear invertible transformation that achieves the true statistical independence for $n > 2$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑