arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.12832cs.LGcs.SYeess.SY
收藏

用于系统辨识的深度网络:综述

Deep networks for system identification: a Survey

University of Padova(帕多瓦大学);University of Washington(华盛顿大学)

Gianluigi Pillonetto, Aleksandr Aravkin, Daniel Gedon, Lennart Ljung, Antônio H. Ribeiro, Thomas B. Schön

摘要:

深度学习是当前相当感兴趣的话题。大量的数据汇集和强大的软件资源的可用性,导致了令人印象深刻的结果在许多应用领域,揭示了基本的,但隐藏的属性的观察。系统辨识从输入输出数据中学习动态系统的数学描述,因此可以受益于深度神经网络的进步,以丰富可供选择的模型的可能范围。出于这个原因,我们从系统辨识的角度对深度学习进行了概述。我们涵盖了广泛的主题,使研究人员能够了解这些方法,为使用它们的好处和挑战提供严格的实践和理论见解。识别模型的主要目的是根据以前的观测结果预测新的数据。这可以通过不同的基于深度学习的建模技术来实现,我们讨论了文献中常用的架构,如前馈,卷积和递归网络。它们的参数必须根据过去的数据来估计,以优化预测性能。为了这个目的,我们讨论了一组特定的一阶优化工具,是作为有效的。然后,该调查与基于内核的方法的研究领域建立了联系。它们通过正则化项来控制数据拟合,这些正则化项会惩罚不符合先前假设的模型。我们说明了如何在深层架构中转换它们以获得基于内核的深层方法。深度学习的成功也带来了令人惊讶的经验观察,比如具有许多参数的模型的反直觉行为。我们讨论过参数化模型的作用,包括它们与内核的连接,以及影响泛化的隐式正则化机制,特别是良性过拟合的有趣现象。

英文摘要:

Deep learning is a topic of considerable current interest. The availability of massive data collections and powerful software resources has led to an impressive amount of results in many application areas that reveal essential but hidden properties of the observations. System identification learns mathematical descriptions of dynamic systems from input-output data and can thus benefit from the advances of deep neural networks to enrich the possible range of models to choose from. For this reason, we provide a survey of deep learning from a system identification perspective. We cover a wide spectrum of topics to enable researchers to understand the methods, providing rigorous practical and theoretical insights into the benefits and challenges of using them. The main aim of the identified model is to predict new data from previous observations. This can be achieved with different deep learning based modelling techniques and we discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks. Their parameters have to be estimated from past data trying to optimize the prediction performance. For this purpose, we discuss a specific set of first-order optimization tools that is emerged as efficient. The survey then draws connections to the well-studied area of kernel-based methods. They control the data fit by regularization terms that penalize models not in line with prior assumptions. We illustrate how to cast them in deep architectures to obtain deep kernel-based methods. The success of deep learning also resulted in surprising empirical observations, like the counter-intuitive behaviour of models with many parameters. We discuss the role of overparameterized models, including their connection to kernels, as well as implicit regularization mechanisms which affect generalization, specifically the interesting phenomena of benign overfitting ...

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑