首页/论文/0705.2011

arXiv 0705.20112007-05-14cs.AIcs.CV

Multi-Dimensional Recurrent Neural Networks

IDSIA（达勒·莫尔人工智能研究所）

Alex Graves, Santiago Fernandez, Juergen Schmidhuber

备注：: 10 pages, 10 figures

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Recurrent neural networks (RNNs) have proved effective at one dimensional sequence learning tasks, such as speech and online handwriting recognition. Some of the properties that make RNNs suitable for such tasks, for example robustness to input warping, and the ability to access contextual information, are also desirable in multidimensional domains. However, there has so far been no direct way of applying RNNs to data with more than one spatio-temporal dimension. This paper introduces multi-dimensional recurrent neural networks (MDRNNs), thereby extending the potential applicability of RNNs to vision, video processing, medical imaging and many other areas, while avoiding the scaling problems that have plagued other multi-dimensional models. Experimental results are provided for two image segmentation tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

递归与循环神经网络综述

A Survey of Recursive and Recurrent Neural Networks

Department of Automation, College of Artificial Intelligence, China University of Petroleum, Beijing, Beijing, China（自动化系、人工智能学院、中国石油大学、北京、北京、中国）

本文系统梳理递归与循环神经网络，按结构、目标函数和学习算法将其分为三大类，阐述模型原理、变形、进展与应用，并展望未来方向。

2025-10-16cs.AIcs.NE

Stochastic Recurrent Neural Network for Multistep Time Series Forecasting

University College London（伦敦大学学院）

Time series forecasting based on deep architectures has been gaining popularity in recent years due to their ability to model complex non-linear temporal dynamics. The recurrent neural network is one such model capable of handling variable-length input and output. In this paper, we leverage recent advances in deep generative models and the concept of state space models to propose a stochastic adaptation of the recurrent neural network for multistep-ahead time series forecasting, which is trained with stochastic gradient variational Bayes. In our model design, the transition function of the rec…

2021-06-25cs.AIcs.LG

Verifying Recurrent Neural Networks using Invariant Inference

The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Stanford University（斯坦福大学）

Deep neural networks are revolutionizing the way complex systems are developed. However, these automatically-generated networks are opaque to humans, making it difficult to reason about them and guarantee their correctness. Here, we propose a novel approach for verifying properties of a widespread variant of neural networks, called recurrent neural networks. Recurrent neural networks play a key role in, e.g., natural language processing, and their verification is crucial for guaranteeing the reliability of many critical systems. Our approach is based on the inference of invariants, which allow…

2020-08-10cs.AIcs.LO

Bach in 2014: Music Composition with Recurrent Neural Network

Carnegie Mellon University（卡内基梅隆大学）

We propose a framework for computer music composition that uses resilient propagation (RProp) and long short term memory (LSTM) recurrent neural network. In this paper, we show that LSTM network learns the structure and characteristics of music pieces properly by demonstrating its ability to recreate music. We also show that predicting existing music using RProp outperforms Back propagation through time (BPTT).

2014-12-14cs.AIcs.NE

Quasi-Recurrent Neural Networks

Salesforce Research（Salesforce研究院）

Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism and makes RNNs unwieldy for very long sequences. We introduce quasi-recurrent neural networks (QRNNs), an approach to neural sequence modeling that alternates convolutional layers, which apply in parallel across timesteps, and a minimalist recurrent pooling function that applies in parallel across channels. Despite lacking trainable recurrent layers, stacked QRNNs have better predictive accuracy than stacked LSTMs of …

2016-11-21cs.AIcs.CLcs.LG

Applying Policy Iteration for Training Recurrent Neural Networks

Recurrent neural networks are often used for learning time-series data. Based on a few assumptions we model this learning task as a minimization problem of a nonlinear least-squares cost function. The special structure of the cost function allows us to build a connection to reinforcement learning. We exploit this connection and derive a convergent, policy iteration-based algorithm. Furthermore, we argue that RNN training can be fit naturally into the reinforcement learning framework.

2004-10-02cs.AIcs.LGcs.NE

更多相关论文收起更多论文

Network of Recurrent Neural Networks

Beijing Jiaotong University（北京交通大学）

We describe a class of systems theory based neural networks called "Network Of Recurrent neural networks" (NOR), which introduces a new structure level to RNN related models. In NOR, RNNs are viewed as the high-level neurons and are used to build the high-level layers. More specifically, we propose several methodologies to design different NOR topologies according to the theory of system evolution. Then we carry experiments on three different tasks to evaluate our implementations. Experimental results show our models outperform simple RNN remarkably under the same number of parameters, and som…

2017-10-10cs.AIcs.NE

Recurrent Neural Networks for video object detection

Technical University of Munich（慕尼黑工业大学）

There is lots of scientific work about object detection in images. For many applications like for example autonomous driving the actual data on which classification has to be done are videos. This work compares different methods, especially those which use Recurrent Neural Networks to detect objects in videos. We differ between feature-based methods, which feed feature maps of different frames into the recurrent units, box-level methods, which feed bounding boxes with class probabilities into the recurrent units and methods which use flow networks. This study indicates common outcomes of the c…

2020-10-29cs.CVcs.LGeess.IV

Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece

School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；Mila（米拉研究所）

We provide a brief review of the common assumptions about biological learning with findings from experimental neuroscience and contrast them with the efficiency of gradient-based learning in recurrent neural networks. The key issues discussed in this review include: synaptic plasticity, neural circuits, theory-experiment divide, and objective functions. We conclude with recommendations for both theoretical and experimental neuroscientists when designing new studies that could help bring clarity to these issues.

2022-01-05cs.AIq-bio.NC

Densely Connected Recurrent Residual (Dense R2UNet) Convolutional Neural Network for Segmentation of Lung CT Images

Washington University School of Medicine（华盛顿大学医学院）

Deep Learning networks have established themselves as providing state of art performance for semantic segmentation. These techniques are widely applied specifically to medical detection, segmentation and classification. The advent of the U-Net based architecture has become particularly popular for this application. In this paper we present the Dense Recurrent Residual Convolutional Neural Network(Dense R2U CNN) which is a synthesis of Recurrent CNN, Residual Network and Dense Convolutional Network based on the U-Net model architecture. The residual unit helps training deeper network, while the…

2021-02-01cs.CVcs.LGeess.IV

Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks

MIT（麻省理工学院）

Recent approaches based on artificial neural networks (ANNs) have shown promising results for short-text classification. However, many short texts occur in sequences (e.g., sentences in a document or utterances in a dialog), and most existing ANN-based systems do not leverage the preceding short texts when classifying a subsequent one. In this work, we present a model based on recurrent neural networks and convolutional neural networks that incorporates the preceding short texts. Our model achieves state-of-the-art results on three different datasets for dialog act prediction.

2016-03-12cs.AIcs.CLcs.LG

Using Recurrent Neural Network for Learning Expressive Ontologies

FBK-irst；University of Trento（特伦托大学）

Recently, Neural Networks have been proven extremely effective in many natural language processing tasks such as sentiment analysis, question answering, or machine translation. Aiming to exploit such advantages in the Ontology Learning process, in this technical report we present a detailed description of a Recurrent Neural Network based system to be used to pursue such goal.

2016-07-14cs.AIcs.CL

Learning Hierarchical Information Flow with Recurrent Neural Modules

Google Brain（谷歌大脑）；Google DeepMind（谷歌DeepMind）

We propose ThalNet, a deep learning model inspired by neocortical communication via the thalamus. Our model consists of recurrent neural modules that send features through a routing center, endowing the modules with the flexibility to share features over multiple time steps. We show that our model learns to route information hierarchically, processing input data by a chain of modules. We observe common architectures, such as feed forward neural networks and skip connections, emerging as special cases of our architecture, while novel connectivity patterns are learned for the text8 compression t…

2017-11-04cs.AIcs.LG

From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing

Universidad de Buenos Aires（布宜诺斯艾利斯大学）

In recent studies [1][13][12] Recurrent Neural Networks were used for generative processes and their surprising performance can be explained by their ability to create good predictions. In addition, data compression is also based on predictions. What the problem comes down to is whether a data compressor could be used to perform as well as recurrent neural networks in natural language processing tasks. If this is possible,then the problem comes down to determining if a compression algorithm is even more intelligent than a neural network in specific tasks related to human language. In our journ…

2017-05-01cs.AIcs.CLcs.IT