arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.09607stat.MEstat.ML
收藏

通用状态空间模型的变分超额风险界

Variational excess risk bound for general state space models

Élisabeth Gassiat, Sylvain Le Corff

AI总结:

该研究针对通用状态空间模型的VAE,通过变分分布反向分解推导了显式依赖样本量与序列长度的预言不等式,给出相关KL散度上界,并证明结果适用于神经网络构建的高斯反向核。

摘要:

本文研究适用于通用状态空间模型的变分自编码器(variational autoencoders, VAE)。我们采用变分分布的反向分解来分析与VAE相关的超额风险,这类反向分解是近期被提出用于实现在线变分学习、以及推导变分估计误差上界的方法。在可观测到序列的独立轨迹、且状态空间模型与变分分布满足强混合假设的条件下,我们给出了样本量与观测序列长度显式相关的预言不等式。随后我们推导了该理论结果的推论:当数据分布由某一状态空间模型生成时,我们给出了数据分布与其估计量之间、以及变分后验与估计的状态空间后验分布之间的Kullback-Leibler散度的上界。在经典假设下,我们证明该结果可应用于由全连接神经网络和循环神经网络构建的高斯反向核。

英文摘要:

In this paper, we consider variational autoencoders (VAE) for general state space models. We consider a backward factorization of the variational distributions to analyze the excess risk associated with VAE. Such backward factorizations were recently proposed to perform online variational learning and to obtain upper bounds on the variational estimation error. When independent trajectories of sequences are observed and under strong mixing assumptions on the state space model and on the variational distribution, we provide an oracle inequality explicit in the number of samples and in the length of the observation sequences. We then derive consequences of this theoretical result. In particular, when the data distribution is given by a state space model, we provide an upper bound for the Kullback-Leibler divergence between the data distribution and its estimator and between the variational posterior and the estimated state space posterior distributions.Under classical assumptions, we prove that our results can be applied to Gaussian backward kernels built with dense and recurrent neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑