发表机构
Carnegie Mellon University（卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种基于预测状态的生成式序列建模方法，用于估计具有无限记忆的多元随机过程的条件分布，通过压缩历史到低维统计量实现快速收敛，并利用深度神经网络验证了理论结果。
AI 中文摘要
我们考虑估计多变量随机过程的一步超前条件分布。许多现有方法依赖于有限范围记忆、稀疏性或可加性等假设，这些假设可能不适用于具有长程非线性相互作用的过程。然而，在没有此类结构假设的情况下，非参数估计因维数灾难而具有挑战性。为应对这一挑战，我们引入了一种基于过程预测状态的新估计方法，该过程可能具有无限范围记忆。我们证明，当过去历史可被压缩为足以预测未来的低维统计量时，我们的估计器可实现快速收敛速度。具体而言，我们表明估计问题的统计复杂度由预测状态空间的内在维度决定。我们为基于深度神经网络估计器的方法实例建立了保证，并通过实验支持这些理论结果。
英文摘要
We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions such as finite-range memory, sparsity, or additivity, which can be poorly suited to processes with long-range nonlinear interactions. However, without such structural assumptions, nonparametric estimation is challenging due to the curse of dimensionality. To address this challenge, we introduce a new estimation approach based on the predictive states of a process, possibly with infinite-range memory. We show that our estimator achieves fast convergence rates when the past history can be compressed into a low-dimensional statistic that is sufficient for predicting the future. Specifically, we show that the statistical complexity of the estimation problem is determined by the intrinsic dimension of the predictive state space. We establish guarantees for an instantiation of our method based on deep neural network estimators, and we support these theoretical results with experiments.
