In this paper, we consider a recursive estimation problem for linear regression where the signal to be estimated admits a sparse representation and measurement samples are only sequentially available. We propose a convergent parallel estimation scheme that consists in solving a sequence of $\ell_{1}$-regularized least-square problems approximately. The proposed scheme is novel in three aspects: i) all elements of the unknown vector variable are updated in parallel at each time instance, and convergence speed is much faster than state-of-the-art schemes which update the elements sequentially; ii) both the update direction and stepsize of each element have simple closed-form expressions, so the algorithm is suitable for online (real-time) implementation; and iii) the stepsize is designed to accelerate the convergence but it does not suffer from the common trouble of parameter tuning in literature. Both centralized and distributed implementation schemes are discussed. The attractive features of the proposed algorithm are also numerically consolidated.
Online Learning for Matrix Factorization and Sparse Coding
INRIA（法国国家信息与自动化研究所）；Laboratoire d’Informatique de l’Ecole Normale Supérieure（巴黎高等师范学院信息学实验室）；Ecole Normale Supérieure（巴黎高等师范学院）；University of Minnesota（明尼苏达大学）
Asymptotic Properties of Recursive Maximum Likelihood Estimation in Non-Linear State-Space Models
使用随机梯度搜索和最优滤波器导数，可以执行递归（即，在线）非线性状态空间模型中的极大似然估计。由于这种模型的最优滤波器及其导数在解析上是难以处理的，因此需要对它们进行数值近似。在[Poyiadjis，Doucet and Singh，Biometrika 2018]中，提出了一种基于粒子逼近最佳滤波器导数的递归最大似然算法，并通过数值模拟进行了研究。本文对该算法及其渐近行为进行了理论分析。我们表明，该算法准确地估计最大值的基本（平均）的对数似然粒子的数量是足够大时。我们还推导出（相对）紧界的估计误差。所得到的结果保持在（相对）温和的条件下，并涵盖了几类非线性状态空间模型在实践中遇到的。
