具有记忆过程的Fisher信息量的通用估计
Universal Estimation of the Fisher Information for Processes with Memory
- KTH Royal Institute of Technology（瑞典皇家理工学院）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
本文提出一种仅从模拟器输出估计具有记忆过程Fisher信息率的方法，利用KL散度曲率与CTW估计，实现强相合且误差阶为n^{-1/3}，实验验证其精度不随深度下降。
中文摘要 AI 辅助
Fisher信息矩阵（FIM）决定了估计问题中可达到的精度。对于具有记忆的参数过程，其FIM仅在少数几类模型中有闭式解，而当模型仅以模拟器形式存在时则无法计算。本文仅从模拟器输出出发，在不知道记忆长度的情况下，估计此类过程的Fisher信息率（即其FIM的每观测极限）。该构造利用了FIM与Kullback-Leibler（KL）散度的局部曲率之间的恒等关系。方向散度率通过上下文树加权（CTW）方法估计，矩阵则通过最小二乘法从这些方向散度率中恢复。我们建立了迭代极限下的强相合性以及一个有限样本界，该界对所有不小于真实记忆长度的工作深度一致成立，给出了阶为$n^{-1/3}$的平均误差。该分析还给出了基于CTW估计KL散度率的有限样本速率。实验证实了预测的指数，表明精度不会随着工作深度的增加而下降，并利用该估计来规划达到规定精度的采样预算。该估计器在无有限记忆长度的隐藏状态源上仍保持准确。
英文摘要
The Fisher information matrix (FIM) determines the accuracy attainable in an estimation problem. For a parametric process with memory, it has a closed form only for narrow classes of models, and it cannot be computed when the model is available only as a simulator. This paper estimates the Fisher information rate of such a process, the per-observation limit of its FIM, from simulator output alone and without knowledge of the memory length. The construction uses the identity between the FIM and the local curvature of the Kullback--Leibler (KL) divergence. Directional divergence rates are estimated by context-tree weighting (CTW), and the matrix is recovered from them by least squares. We establish strong consistency in an iterated limit and a finite-sample bound, uniform over every working depth at least as large as the true memory length, which gives a mean error of order $n^{-1/3}$. The analysis also delivers a finite-sample rate for CTW-based estimation of the KL divergence rate. Experiments confirm the predicted exponents, show that the accuracy does not degrade as the working depth grows, and use the estimate to plan a sampling budget that attains a prescribed precision. The estimator stays accurate on a hidden-state source that has no finite memory length.
