arXiv 0801.18832008-01-12cs.NEcs.ITmath.IT

D-optimal Bayesian Interrogation for Parameter and Noise Identification of Recurrent Neural Networks

Barnabas Poczos, Andras Lorincz

英文摘要：

We introduce a novel online Bayesian method for the identification of a family of noisy recurrent neural networks (RNNs). We develop Bayesian active learning technique in order to optimize the interrogating stimuli given past experiences. In particular, we consider the unknown parameters as stochastic variables and use the D-optimality principle, also known as `\emph{infomax method}', to choose optimal stimuli. We apply a greedy technique to maximize the information gain concerning network parameters at each time step. We also derive the D-optimal estimation of the additive noise that perturbs the dynamical system of the RNN. Our analytical results are approximation-free. The analytic derivation gives rise to attractive quadratic update rules.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Author Identification using Multi-headed Recurrent Neural Networks

Recurrent neural networks (RNNs) are very good at modelling the flow of text, but typically need to be trained on a far larger corpus than is available for the PAN 2015 Author Identification task. This paper describes a novel approach where the output layer of a character-level RNN language model is split into several independent predictive sub-models, each representing an author, while the recurrent layer is shared by all. This allows the recurrent layer to model the language as a whole without over-fitting, while the outputs select aspects of the underlying model that reflect their author's …

2016-08-16cs.CLcs.LGcs.NE

Physics-Incorporated Convolutional Recurrent Neural Networks for Source Identification and Forecasting of Dynamical Systems

Georgia Institute of Technology（佐治亚理工学院）

Spatio-temporal dynamics of physical processes are generally modeled using partial differential equations (PDEs). Though the core dynamics follows some principles of physics, real-world physical processes are often driven by unknown external sources. In such cases, developing a purely analytical model becomes very difficult and data-driven modeling can be of assistance. In this paper, we present a hybrid framework combining physics-based numerical models with deep learning for source identification and forecasting of spatio-temporal dynamical systems with unobservable time-varying external sou…

2021-08-31cs.LGcs.NEstat.ML

De-identification of Patient Notes with Recurrent Neural Networks

MIT（麻省理工学院）；SUNY Albany（纽约州立大学奥尔巴尼分校）

Objective: Patient notes in electronic health records (EHRs) may contain critical information for medical investigations. However, the vast majority of medical investigators can only access de-identified notes, in order to protect the confidentiality of patients. In the United States, the Health Insurance Portability and Accountability Act (HIPAA) defines 18 types of protected health information (PHI) that needs to be removed to de-identify patient notes. Manual de-identification is impractical given the size of EHR databases, the limited number of researchers with access to the non-de-identif…

2016-06-10cs.AIcs.CLcs.NE

Driver Identification Based on Vehicle Telematics Data using LSTM-Recurrent Neural Network

Autonomous Control and Information Technology (ACIT) Institute（自主控制与信息技术（ACIT）研究院）

Despite advancements in vehicle security systems, over the last decade, auto-theft rates have increased, and cyber-security attacks on internet-connected and autonomous vehicles are becoming a new threat. In this paper, a deep learning model is proposed, which can identify drivers from their driving behaviors based on vehicle telematics data. The proposed Long-Short-Term-Memory (LSTM) model predicts the identity of the driver based on the individual's unique driving patterns learned from the vehicle telematics data. Given the telematics is time-series data, the problem is formulated as a time …

2019-11-19cs.LGcs.NEeess.SP

Temporal Overdrive Recurrent Neural Network

In this work we present a novel recurrent neural network architecture designed to model systems characterized by multiple characteristic timescales in their dynamics. The proposed network is composed by several recurrent groups of neurons that are trained to separately adapt to each timescale, in order to improve the system identification process. We test our framework on time series prediction tasks and we show some promising, preliminary results achieved on synthetic data. To evaluate the capabilities of our network, we compare the performance with several state-of-the-art recurrent architec…

2017-01-18cs.NEmath.DS

Deep Convolutional Networks in System Identification

Uppsala University（乌普萨拉大学）；Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）

Recent developments within deep learning are relevant for nonlinear system identification problems. In this paper, we establish connections between the deep learning and the system identification communities. It has recently been shown that convolutional architectures are at least as capable as recurrent architectures when it comes to sequence modeling tasks. Inspired by these results we explore the explicit relationships between the recently proposed temporal convolutional network (TCN) and two classic system identification model structures; Volterra series and block-oriented models. We end t…

2019-11-19cs.LGcs.NEcs.SY

更多相关论文收起更多论文

Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks

University of Oxford（牛津大学）

This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without requiring any feature engineering or system identification in the form of plant or sensor models. Specifically, our system accepts a stream of raw sensor data at one end and, in real-time, produces an estimate of the entire environment state at the output including even occluded objects. We achieve this by framing the problem as a deep learning task and exploit sequence models in the form of recurrent neural networks t…

2016-03-08cs.AIcs.CVcs.LG

Identification of Helicopter Dynamics based on Flight Data using Nature Inspired Techniques

The complexity of helicopter flight dynamics makes modeling and helicopter system identification a very difficult task. Most of the traditional techniques require a model structure to be defined apriori and in case of helicopter dynamics, this is difficult due to its complexity and the interplay between various subsystems.To overcome this difficulty, non-parametric approaches are commonly adopted for helicopter system identification. Artificial Neural Network are a widely used class of algorithms for non-parametric system identification, among them, the Nonlinear Auto Regressive eXogeneous inp…

2014-11-12cs.CEcs.NE