隐马尔可夫模型递推极大似然估计的解析性、收敛性和收敛速度

具有相关误差的递归线性模型的最大似然估计的计算

Computing maximum likelihood estimates in recursive linear models with correlated errors

在递归线性模型中，所有变量的多元正态联合分布表现出由线性结构方程的递归（或非循环）系统诱导的相关结构。这些线性模型有着悠久的传统，出现在看似无关的回归、结构方程建模和因果推理方法中。它们还通过称为路径图的经典表示与高斯图形模型相关。尽管这些模式有着悠久的历史，但仍存在一些问题。在本文中，我们解决的问题，计算最大似然估计的子类中的“弓自由”递归线性模型。术语“无弓形”是指如果变量$j$的结构方程中出现变量$i$，则变量$i$和$j$的误差不相关的条件。我们介绍了一种新的算法，称为残差迭代条件拟合（RICF），可以实现仅使用最小二乘计算。与现有算法相比，RICF具有清晰的收敛性，并尽可能以封闭形式找到参数估计。

AdaVol: An Adaptive Recursive Volatility Prediction Method

Quasi-Maximum Likelihood (QML) procedures are theoretically appealing and widely used for statistical inference. While there are extensive references on QML estimation in batch settings, it has attracted little attention in streaming settings until recently. An investigation of the convergence properties of the QML procedure in a general conditionally heteroscedastic time series model is conducted, and the classical batch optimization routines extended to the framework of streaming and large-scale problems. An adaptive recursive estimation routine for GARCH models named AdaVol is presented. Th…

有限状态嵌套马尔可夫模型中的递归头几何与无序高效推断

Recursive-Head Geometry and Order-Free Efficient Inference in Finite-State Nested Markov Models

本文通过递归头几何证明有限状态嵌套马尔可夫模型在任意ADMG上的无序高效推断，提出规范梯度公式与有效估计量，并区分了无序有效性与多重稳健性。

递归 Logit 选择模型的均衡约束估计

Equilibrium-Constrained Estimation of Recursive Logit Choice Models

针对递归 logit 模型依赖 NFXP 导致的高成本与数值不稳定问题，本文将最大似然估计重构为带均衡约束的指数锥优化，并在合成与真实数据上验证其精度相当且更稳定高效。

迭代合成数据增强下实体排序的最大似然估计

Maximum Likelihood Estimation for Entity Ranking under Iterative Synthetic Data Augmentation

本研究针对BTL模型下稀疏比较数据的实体排序问题，分析了迭代合成数据增强对最大似然估计的影响，推导了最优统计速率与渐近正态性，并刻画了避免模型崩溃的条件，通过实验验证了理论结果。

通过骨架分解因子化实现快速空间高斯过程最大似然估计

Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations

本文提出一种方法，通过骨架分解因子化和矩阵剥离算法，实现对空间高斯过程最大似然估计的高效计算，时间复杂度为~O(n^{3/2})。

Nearly second-order asymptotic optimality of sequential change-point detection with one-sample updates

Sequential change-point detection when the distribution parameters are unknown is a fundamental problem in statistics and machine learning. When the post-change parameters are unknown, we consider a set of detection procedures based on sequential likelihood ratios with non-anticipating estimators constructed using online convex optimization algorithms such as online mirror descent, which provides a more versatile approach to tackle complex situations where recursive maximum likelihood estimators cannot be found. When the underlying distributions belong to a exponential family and the estimator…

Particle-based, online estimation of tangent filters with application to parameter estimation in nonlinear state-space models

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished with a number of convergence results, including a central limit theorem with an asymptotic variance that can be shown to be uniformly bounded in time. Using the proposed filter derivative estimator we design a recursive maximum likelihood algorithm updating the parameters according the gradient of the one-step predictor log-likelihood. The efficiency of this …

Online Maximum Likelihood Estimation of the Parameters of Partially Observed Diffusion Processes

We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the parameter estimate should be updated recursively based on the observation filtration. We provide a theoretical analysis of the stochastic gradient ascent algorithm on the incomplete-data log-likelihood. The convergence of the algorithm is proved under suitable conditions regarding the ergodicity of the process consisting of state, filter, and tangent filter. Ad…

倒谱随机场的渐近理论

Asymptotic theory of cepstral random fields

随机场在空间相关数据的分析中起着核心作用，因此，对广泛的科学应用产生了重大影响。本文研究了倒谱随机场模型，给出了空间倒谱系数与等价的滑动平均随机场的递推公式，使自协方差矩阵的计算变得容易。我们还提供了一个全面的治疗的渐近理论的二维随机场模型：我们建立了渐近结果的贝叶斯，极大似然和拟极大似然估计的随机场参数和回归参数。一般的理论结果是独立的兴趣，属于广泛的一类随机场模型。倒谱模型的结果有利于模型的建立：因为倒谱系数在实践中是无约束的，数值优化大大简化，我们总是保证一个正定的协方差阵。我们表明，个别系数的推断是可能的，并且可以以有纪律的方式改进模型。通过模拟和田间试验的秸秆产量数据分析，说明了我们的研究结果。

可能错误指定的MA（1）模型的递归估计：一般算法的收敛性

Recursive estimation of possibly misspecified MA(1) models: Convergence of a general algorithm

本文给出了一种递推算法，它具有一般形式，可方便地估计一阶滑动平均模型的系数，并得到了正确和错误设定的MA（1）模型的收敛结果。该算法包括伪线性回归（PLR--也称为AML和$RML_1$）和递归最大似然法（$RML_2$），无需监视。受Hannan（1980）方法的启发，我们的收敛性结果是间接地通过证明递归序列可以用一个满足较简单（Robbins-Monro）形式的递归的序列来近似而得到的，最近已经得到了适用于我们情况的收敛性结果。

交互粒子系统中高效的在线学习

Efficient Online Learning in Interacting Particle Systems

本文提出了一种在交互粒子系统中进行高效在线学习的方法，通过连续观测少量粒子并递归更新模型参数，实现了对渐近对数似然的收敛性证明，并通过数值实验验证了其有效性。

傅里叶递归方法用于潜在马尔可夫模型的估计与推断

Estimation and Inference for Latent Markov Models by Fourier Recursions

本文提出一种理论上精确的傅里叶递归框架，用于非高斯非线性潜在马尔可夫模型的估计与推断，提供统一截断实现和首个一般渐近理论，模拟和破产数据应用验证了其有效性和稳定性。

Rescuing neural spike train models from bad MLE

The standard approach to fitting an autoregressive spike train model is to maximize the likelihood for one-step prediction. This maximum likelihood estimation (MLE) often leads to models that perform poorly when generating samples recursively for more than one time step. Moreover, the generated spike trains can fail to capture important features of the data and even show diverging firing rates. To alleviate this, we propose to directly minimize the divergence between neural recorded and model generated spike trains using spike train kernels. We develop a method that stochastically optimizes th…

迭代滤波

Iterated filtering

部分观测马尔可夫过程模型的推理一直是一个长期存在的方法挑战，许多科学和工程应用。迭代滤波算法通过求解滤波问题的递归序列来最大化部分观测马尔可夫过程模型的似然函数。我们提出了新的理论成果有关的迭代滤波算法的收敛性，通过顺序蒙特卡罗滤波器实现。这一理论补充了越来越多的经验证据，迭代过滤算法提供了一个有效的推理策略的非线性动态系统的科学模型。在我们的理论的第一步涉及研究一种新的递归方法，最大化的似然函数的潜在变量模型，当这种可能性是通过重要性抽样评估。这导致考虑迭代重要性采样算法，其用作迭代滤波的简单特殊情况，并且可能具有其自身的适用性。

基于粗糙、不规则、含噪样本的导数估计：一种MLE-样条方法

Derivative Estimation from Coarse, Irregular, Noisy Samples: An MLE-Spline Approach

针对粗糙非均匀采样且含高斯噪声的数值微分问题，提出带高阶导数$L_2$范数约束的MLE-样条方法，含两种可权衡平滑性与速度的形式，仿真验证其在低采样高噪场景下性能优于现有微分器。

Maximum Likelihood With a Time Varying Parameter

We consider the problem of tracking an unknown time varying parameter that characterizes the probabilistic evolution of a sequence of independent observations. To this aim, we propose a stochastic gradient descent-based recursive scheme in which the log-likelihood of the observations acts as time varying gain function. We prove convergence in mean-square error in a suitable neighbourhood of the unknown time varying parameter and illustrate the details of our findings in the case where data are generated from distributions belonging to the exponential family.

Efficient parameter inference in general hidden Markov models using the filter derivatives

Estimating online the parameters of general state-space hidden Markov models is a topic of importance in many scientific and engineering disciplines. In this paper we present an online parameter estimation algorithm obtained by casting our recently proposed particle-based, rapid incremental smoother (PaRIS) into the framework of recursive maximum likelihood estimation for general hidden Markov models. Previous such particle implementations suffer from either quadratic complexity in the number of particles or from the well-known degeneracy of the genealogical particle paths. By using the comput…

基于函数的非线性最小二乘法及其在Jelinski-Moranda软件可靠性模型中的应用

Function Based Nonlinear Least Squares and Application to Jelinski--Moranda Software Reliability Model

提出了一种基于函数的非线性最小二乘估计（FNLSE）方法，并将其应用于Jelinski-Moranda软件可靠性模型的参数估计。FNLSE扩展了传统最小二乘估计（LSE）的潜在拟合函数，并将对数变换非线性最小二乘估计（LogLSE）作为其特例。提出了一种基于幂变换函数的非线性最小二乘估计（powLSE），并将其应用于Jelinski-Moranda模型的参数估计。采用Newton-Raphson方法求解，将Jelinski-Moranda模型的LogLSE和powLSE分别应用于6个标准软件故障时间数据集的平均故障间隔时间（MTBF）预测。实验结果表明，与经典的最小二乘估计（LSE）、极大似然估计（MLE）和对数最小二乘估计（LogLSE）相比，具有最优幂指数的powLSE在递归相对误差（RE）指标和Braun统计量指标方面是有效的。

基于干扰平滑的线性混合效应状态空间模型估计

Estimating Linear Mixed-effects State Space Model Based on Disturbance Smoothing

本文对线性混合效应状态模型进行了扩展，使其包含相关个体，并研究了其参数和基于干扰平滑的状态估计。对于参数估计，考虑EM和基于分数的算法。首先研究了期望最大化算法的中间量，并对两个给定的模型导出了中间量最大化的显式递推公式。对于基于得分的算法，给出了得分向量的显式表达式，并由此证明了极大似然估计与矩估计的等价性。对于状态估计，我们主张在不假定随机效应事先已知的情况下进行估计，特别是当纵向观测稀疏时。为此，提出了一种核平滑混合卡尔曼滤波算法（MKF-KS）。数值研究进行了调查所提出的算法，验证了所提出的推理方法的有效性。