首页/论文/1808.00523

arXiv 1808.005232018-08cs.LGstat.ML

Mod-DeepESN: Modular Deep Echo State Network

Rochester Institute of Technology（罗切斯特理工学院）

Zachariah Carmichael, Humza Syed, Stuart Burtner, Dhireesha Kudithipudi

备注：: 4 pages, Cognitive Computational Neuroscience (CCN) 2018 Conference

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex tasks that are characterized by multi-scale structures. In this research, an intrinsic plasticity-infused modular deep echo state network architecture is proposed to solve complex and multiple timescale temporal tasks. It outperforms state-of-the-art for time series prediction tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Deep Echo State Network (DeepESN): A Brief Survey

Department of Computer Science, University of Pisa（比萨大学计算机科学系）

The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of DeepESNs allowed to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers, i.e. on the bias of depth in RNNs architectural design. In this paper, we summariz…

2020-09-25cs.AIcs.LGstat.ML

Reservoir Topology in Deep Echo State Networks

University of Pisa（比萨大学）

Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a parti…

2019-09-24cs.LGcs.NEstat.ML

Comparison between DeepESNs and gated RNNs on multivariate time-series prediction

University of Pisa（比萨大学）

We propose an experimental comparison between Deep Echo State Networks (DeepESNs) and gated Recurrent Neural Networks (RNNs) on multivariate time-series prediction tasks. In particular, we compare reservoir and fully-trained RNNs able to represent signals featured by multiple time-scales dynamics. The analysis is performed in terms of efficiency and prediction accuracy on 4 polyphonic music tasks. Our results show that DeepESN is able to outperform ESN in terms of prediction accuracy and efficiency. Whereas, between fully-trained approaches, Gated Recurrent Units (GRU) outperforms Long Short-T…

2019-11-20cs.LGcs.NEstat.ML

Richness of Deep Echo State Network Dynamics

Department of Computer Science, University of Pisa（比萨大学计算机科学系）

Reservoir Computing (RC) is a popular methodology for the efficient design of Recurrent Neural Networks (RNNs). Recently, the advantages of the RC approach have been extended to the context of multi-layered RNNs, with the introduction of the Deep Echo State Network (DeepESN) model. In this paper, we study the quality of state dynamics in progressively higher layers of DeepESNs, using tools from the areas of information theory and numerical analysis. Our experimental results on RC benchmark datasets reveal the fundamental role played by the strength of inter-reservoir connections to increasingl…

2019-09-24cs.AIcs.LGcs.NE

Deep Echo State Networks for Diagnosis of Parkinson's Disease

Department of Computer Science University of Pisa（比萨大学计算机科学系（更准确的常见译法））

In this paper, we introduce a novel approach for diagnosis of Parkinson's Disease (PD) based on deep Echo State Networks (ESNs). The identification of PD is performed by analyzing the whole time-series collected from a tablet device during the sketching of spiral tests, without the need for feature extraction and data preprocessing. We evaluated the proposed approach on a public dataset of spiral tests. The results of experimental analysis show that DeepESNs perform significantly better than shallow ESN model. Overall, the proposed approach obtains state-of-the-art results in the identificatio…

2018-02-19cs.LG

A Review of Designs and Applications of Echo State Networks

Peking University（北京大学）；National Institute of Health Data Science, Peking University（北京大学健康医疗大数据国家研究院）；Institute of Medical Technology, Health Science Center, Peking University（北京大学医学部医学技术研究院）

Recurrent Neural Networks (RNNs) have demonstrated their outstanding ability in sequence tasks and have achieved state-of-the-art in wide range of applications, such as industrial, medical, economic and linguistic. Echo State Network (ESN) is simple type of RNNs and has emerged in the last decade as an alternative to gradient descent training based RNNs. ESN, with a strong theoretical ground, is practical, conceptually simple, easy to implement. It avoids non-converging and computationally expensive in the gradient descent methods. Since ESN was put forward in 2002, abundant existing works hav…

2020-12-05cs.LGcs.NE

更多相关论文收起更多论文

基于局部场电位记录（LFP）的大鼠海马与伏隔核全局数据驱动模型

A Global Data-Driven Model for The Hippocampus and Nucleus Accumbens of Rat From The Local Field Potential Recordings (LFP)

K. N. Toosi University of Technology（柯尼托西大学）；Shahid Beheshti University of Medical Sciences（沙希德贝赫什提医科大学）；Neuroscience Research Center（神经科学研究中心）

本文利用大鼠伏隔核与海马在吗啡及天然食物奖励前后的LFP记录，比较五种机器学习方法，构建由LoLiMoT实现的单一全局预测模型，并发现两类奖励均不改变相关神经元动态特征。

2024-05-10cs.LGq-bio.NC

分层计数回声状态网络模型及其在研究生招生中的应用

Hierarchical Count Echo State Network Models with Application to Graduate Student Enrollments

本文提出计数回声状态网络作为泊松自回归的替代方法，并开发分层负二项回声状态网络，应用于美国1758所大学1972-2021年研究生招生数据，预测性能最优。

2025-01-24stat.APstat.MEstat.ML

Echo State Neural Machine Translation

Google Research（谷歌研究（Google Research））

We present neural machine translation (NMT) models inspired by echo state network (ESN), named Echo State NMT (ESNMT), in which the encoder and decoder layer weights are randomly generated then fixed throughout training. We show that even with this extremely simple model construction and training procedure, ESNMT can already reach 70-80% quality of fully trainable baselines. We examine how spectral radius of the reservoir, a key quantity that characterizes the model, determines the model behavior. Our findings indicate that randomized networks can work well even for complicated sequence-to-seq…

2020-02-27cs.CLcs.LGcs.NE

神经回声：理解神经网络的信号处理视角

The Neural Echo: A Signal Processing Perspective for Understanding Neural Networks

Saarland University（萨尔大学）；Zuse School ELIZA（楚塞学校ELIZA）

本文提出神经回声工具，将经典信号处理的脉冲响应等概念推广到神经网络，可解释各类网络行为，以DnCNN为例验证其能复现经典去噪器关键概念，搭建信号处理与可解释AI的桥梁。

2026-08-06cs.LGeess.SP

EchoGNN：使用图神经网络进行可解释的射血分数估计

EchoGNN: Explainable Ejection Fraction Estimation with Graph Neural Networks

University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）

本文提出EchoGNN，利用图神经网络从超声心动图视频中估计射血分数，并通过图权重识别关键帧以提供可解释性，在EchoNet-Dynamic上达到与最先进方法相当的性能。

2023-07-23cs.LGeess.IV

Echocardiographic Image Quality Assessment Using Deep Neural Networks

University of West London（西伦敦大学）；Imperial College（帝国理工学院）

Echocardiography image quality assessment is not a trivial issue in transthoracic examination. As the in vivo examination of heart structures gained prominence in cardiac diagnosis, it has been affirmed that accurate diagnosis of the left ventricle functions is hugely dependent on the quality of echo images. Up till now, visual assessment of echo images is highly subjective and requires specific definition under clinical pathologies. While poor-quality images impair quantifications and diagnosis, the inherent variations in echocardiographic image quality standards indicates the complexity face…

2022-09-02cs.CVcs.LGeess.IV