超越$\ell_1$的V1稀疏编码
Beyond $\ell_1$ sparse coding in V1
摘要:
越来越多的证据表明,只有一个稀疏的子集从池的感觉神经元是活跃的编码的视觉刺激在任何时刻。传统上,为了复制这种生物稀疏性,生成模型一直使用$\ell_1$范数作为惩罚,因为它的凸性,这使得它适合快速和简单的算法求解器。在这项工作中,我们使用生物视觉作为试验台,并表明,软阈值操作与使用的$\ell_1$规范是非常次优的相比,其他功能适合近似$\ell_q$与$0 \leq q < 1 $(包括最近提出的连续精确松弛),无论是在性能和生产的功能,是类似的签名的初级视觉皮层。我们表明,$\ell_1$稀疏产生一个更密集的代码或采用一个池与更多的神经元,即具有更高程度的过完备性,以保持相同的重建误差考虑的其他方法。对于所有测试的惩罚函数,一个子集的神经元开发方向选择性类似于V1神经元。当它们的代码足够稀疏时,这些方法还开发出具有不同功能的感受野,这是V1的另一个特征。与其他方法相比,软阈值实现了这种程度的稀疏性,代价是重建性能大大降低,这在生物视觉中很可能是不可接受的。我们的研究结果表明,V1使用的稀疏诱导正则化更接近于$\ell_0$伪范数,而不是$\ell_1$范数。
英文摘要:
Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the $\ell_1$ norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the $\ell_1$ norm is highly suboptimal compared to other functions suited to approximating $\ell_q$ with $0 \leq q < 1 $ (including recently proposed Continuous Exact relaxations), both in terms of performance and in the production of features that are akin to signatures of the primary visual cortex. We show that $\ell_1$ sparsity produces a denser code or employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. For all the penalty functions tested, a subset of the neurons develop orientation selectivity similarly to V1 neurons. When their code is sparse enough, the methods also develop receptive fields with varying functionalities, another signature of V1. Compared to other methods, soft thresholding achieves this level of sparsity at the expense of much degraded reconstruction performance, that more likely than not is not acceptable in biological vision. Our results indicate that V1 uses a sparsity inducing regularization that is closer to the $\ell_0$ pseudo-norm rather than to the $\ell_1$ norm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。