用于动态系统辨识的稀疏贝叶斯深度学习
Sparse Bayesian Deep Learning for Dynamic System Identification
摘要：
本文提出了一种用于系统辨识的深度神经网络（DNN）的稀疏贝叶斯处理。虽然DNN在各个领域都表现出令人印象深刻的逼近能力，但系统辨识问题仍然存在一些挑战。首先，DNN太复杂，很容易过拟合训练数据。其次，系统辨识的输入回归量的选择是非平凡的。第三，模型参数和预测的不确定性量化是必要的。所提出的贝叶斯方法提供了一个原则性的方法，以减轻上述挑战的边际似然/模型证据近似和结构化组稀疏诱导先验构造。识别算法是作为一个迭代正则化优化过程推导出来的，可以像训练典型的DNN一样有效地解决。值得注意的是，为DNN的每一层开发了一种高效的递归Hessian计算方法，将棘手的训练/优化过程变成了一个易于处理的过程。此外，一个实用的计算方法的基础上的Monte-Carlo积分方法来量化的参数和预测的不确定性。所提出的贝叶斯方法的有效性证明了几个线性和非线性系统辨识基准，实现良好的和有竞争力的仿真精度。重现实验结果的代码是开源的，可以在线获得。
英文摘要：
This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. Although DNNs show impressive approximation ability in various fields, several challenges still exist for system identification problems. First, DNNs are known to be too complex that they can easily overfit the training data. Second, the selection of the input regressors for system identification is nontrivial. Third, uncertainty quantification of the model parameters and predictions are necessary. The proposed Bayesian approach offers a principled way to alleviate the above challenges by marginal likelihood/model evidence approximation and structured group sparsity-inducing priors construction. The identification algorithm is derived as an iterative regularised optimisation procedure that can be solved as efficiently as training typical DNNs. Remarkably, an efficient and recursive Hessian calculation method for each layer of DNNs is developed, turning the intractable training/optimisation process into a tractable one. Furthermore, a practical calculation approach based on the Monte-Carlo integration method is derived to quantify the uncertainty of the parameters and predictions. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear system identification benchmarks by achieving good and competitive simulation accuracy. The code to reproduce the experimental results is open-sourced and available online.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
