Fourier-Enhanced Recurrent Neural Networks for Electrical Load Time Series Downscaling
Department of Statistics University of Chicago Chicago, IL, USA（统计学系 芝加哥大学）；Computer Science Division Argonne National Laboratory（计算机科学 division 阿贡国家实验室）
We present a Fourier-enhanced recurrent neural network (RNN) for downscaling electrical loads. The model combines (i) a recurrent backbone driven by low-resolution inputs, (ii) explicit Fourier seasonal embeddings fused in latent space, and (iii) a self-attention layer that captures dependencies among high-resolution components within each period. Across four PJM territories, the approach yields RMSE lower and flatter horizon-wise than classical Prophet baselines (with and without seasonality/LAA) and than RNN ablations without attention or Fourier features.
École polytechnique fédérale de Lausanne（洛桑联邦理工学院）；Federal Office of Meteorology and Climatology MeteoSwiss（联邦气象和气候学办公室 MeteoSwiss）
Generative adversarial networks (GANs) have been recently adopted for super-resolution, an application closely related to what is referred to as "downscaling" in the atmospheric sciences: improving the spatial resolution of low-resolution images. The ability of conditional GANs to generate an ensemble of solutions for a given input lends itself naturally to stochastic downscaling, but the stochastic nature of GANs is not usually considered in super-resolution applications. Here, we introduce a recurrent, stochastic super-resolution GAN that can generate ensembles of time-evolving high-resoluti…
This paper compares recurrent neural networks (RNNs) with different types of gated cells for forecasting time series with multiple seasonality. The cells we compare include classical long short term memory (LSTM), gated recurrent unit (GRU), modified LSTM with dilation, and two new cells we proposed recently, which are equipped with dilation and attention mechanisms. To model the temporal dependencies of different scales, our RNN architecture has multiple dilated recurrent layers stacked with hierarchical dilations. The proposed RNN produces both point forecasts and predictive intervals (PIs) …
Université Paris-Saclay（巴黎萨克雷大学）；CentraleSupélec（巴黎中央理工-高等电力学院）；Inria（法国国家信息与自动化研究所）；Schneider Toshiba Inverter Europe（施耐德东芝逆变器欧洲公司）；Samovar（萨莫瓦尔实验室）；CNRS（法国国家科学研究中心）；Télécom SudParis（南巴黎电信学院）；Institut Polytechnique de Paris（巴黎理工学院）
Electrical motors are the most important source of mechanical energy in the industrial world. Their modeling traditionally relies on a physics-based approach, which aims at taking their complex internal dynamics into account. In this paper, we explore the feasibility of modeling the dynamics of an electrical motor by following a data-driven approach, which uses only its inputs and outputs and does not make any assumption on its internal behaviour. We propose a novel encoder-decoder architecture which benefits from recurrent skip connections. We also propose a novel loss function that takes int…
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；University Medical Center Schleswig-Holstein（石勒苏益格-荷尔斯泰因大学医学中心）
Electrical impedance tomography (EIT) is a noninvasive imaging modality that allows a continuous assessment of changes in regional bioimpedance of different organs. One of its most common biomedical applications is monitoring regional ventilation distribution in critically ill patients treated in intensive care units. In this work, we put forward a proof-of-principle study that demonstrates how one can reconstruct synchronously measured respiratory or circulatory parameters from the EIT image sequence using a deep learning model trained in an end-to-end fashion. We demonstrate that one can acc…
Surface Electromyography (sEMG/EMG) is to record muscles' electrical activity from a restricted area of the skin by using electrodes. The sEMG-based gesture recognition is extremely sensitive of inter-session and inter-subject variances. We propose a model and a deep-learning-based domain adaptation method to approximate the domain shift for recognition accuracy enhancement. Analysis performed on sparse and HighDensity (HD) sEMG public datasets validate that our approach outperforms state-of-the-art methods.
Discrete-time modeling of acoustic, mechanical and electrical systems is a prominent topic in the musical signal processing literature. Such models are mostly derived by discretizing a mathematical model, given in terms of ordinary or partial differential equations, using established techniques. Recent work has applied the techniques of machine-learning to construct such models automatically from data for the case of systems which have lumped states described by scalar values, such as electrical circuits. In this work, we examine how similar techniques are able to construct models of systems w…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Eaton Research Lab（伊顿研究实验室）；Eaton Corporation（伊顿公司）
Accurate load forecasting is critical for electricity market operations and other real-time decision-making tasks in power systems. This paper considers the short-term load forecasting (STLF) problem for residential customers within a community. Existing STLF work mainly focuses on forecasting the aggregated load for either a feeder system or a single customer, but few efforts have been made on forecasting the load at individual appliance level. In this work, we present an STLF algorithm for efficiently predicting the power consumption of individual electrical appliances. The proposed method b…
The brain electrical activity presents several short events during sleep that can be observed as distinctive micro-structures in the electroencephalogram (EEG), such as sleep spindles and K-complexes. These events have been associated with biological processes and neurological disorders, making them a research topic in sleep medicine. However, manual detection limits their study because it is time-consuming and affected by significant inter-expert variability, motivating automatic approaches. We propose a deep learning approach based on convolutional and recurrent neural networks for sleep EEG…
Fully Analog Resonant Recurrent Neural Network via Metacircuit
Institute of Humanoid Robots, CAS Key Laboratory of Mechanical Behavior and Design of Materials, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China（机器人研究院、材料力学行为与设计国家重点实验室、精密机械与精密仪器系、中国科学技术大学）；State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University（机械系统与振动国家重点实验室、上海交通大学）
We introduce a neural-network architecture, termed the constrained recurrent sparse autoencoder (CRsAE), that solves convolutional dictionary learning problems, thus establishing a link between dictionary learning and neural networks. Specifically, we leverage the interpretation of the alternating-minimization algorithm for dictionary learning as an approximate Expectation-Maximization algorithm to develop autoencoders that enable the simultaneous training of the dictionary and regularization parameter (ReLU bias). The forward pass of the encoder approximates the sufficient statistics of the E…
Surrogate models for nuclear fusion with parametric Shallow Recurrent Decoder Networks: applications to magnetohydrodynamics
Department of Energy, Politecnico di Milano（米兰理工大学能源部）；Autodesk Research, 6 Agar Street, London UK（伦敦Autodesk研究部，6 Agar Street，英国）；Department of Mechanical and Nuclear Engineering & Emirates Nuclear Technology Center, Khalifa University（卡利法大学机械与核工程系及阿联酋核技术中心）
University of Hildesheim（希尔德斯海姆大学）；Volkswagen AG（大众汽车公司）
Recent work has shown the efficiency of deep learning models such as Fully Convolutional Networks (FCN) or Recurrent Neural Networks (RNN) to deal with Time Series Regression (TSR) problems. These models sometimes need a lot of data to be able to generalize, yet the time series are sometimes not long enough to be able to learn patterns. Therefore, it is important to make use of information across time series to improve learning. In this paper, we will explore the idea of using meta-learning for quickly adapting model parameters to new short-history time series by modifying the original idea of…
Epilepsy Center Frankfurt（法兰克福癫痫中心）；Rhine-Main Neurocenter（莱茵-美因神经中心）
Epilepsy is a common neurological disorder characterized by recurrent seizures accompanied by excessive synchronous brain activity. The process of structural and functional brain alterations leading to increased seizure susceptibility and eventually spontaneous seizures is called epileptogenesis (EPG) and can span months or even years. Detecting and monitoring the progression of EPG could allow for targeted early interventions that could slow down disease progression or even halt its development. Here, we propose an approach for staging EPG using deep neural networks and identify potential ele…
Deep Learning has attracted considerable attention across multiple application domains, including computer vision, signal processing and natural language processing. Although quite a few single node deep learning frameworks exist, such as tensorflow, pytorch and keras, we still lack a complete processing structure that can accommodate large scale data processing, version control, and deployment, all while staying agnostic of any specific single node framework. To bridge this gap, this paper proposes a new, higher level framework, i.e. Nemesyst, which uses databases along with model sequentiali…
Single vs. Multiple Branches in DeepONet and S-DeepONet: Network Architecture Follows Coupling in Multiphysics Systems
National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign（国家超级计算应用中心，伊利诺伊大学厄巴纳-香槟分校）；The Grainger College of Engineering, Mechanical Science and Engineering, University of Illinois at Urbana-Champaign（格拉inger工程学院，机械科学与工程系，伊利诺伊大学厄巴纳-香槟分校）；The Grainger College of Engineering, Nuclear, Plasma & Radiological Engineering, University of Illinois at Urbana-Champaign（格拉inger工程学院，核物理与辐射工程系，伊利诺伊大学厄巴纳-香槟分校）；Department of Industrial and Manufacturing Systems Engineering, Kansas State University（工业与制造系统工程系，堪萨斯州立大学）；Civil and Urban Engineering Department, New York University Abu Dhabi, UAE（土木与城市工程系，纽约大学阿布扎比分校，阿联酋）
Epileptic Seizure Detection in Separate Frequency Bands Using Feature Analysis and Graph Convolutional Neural Network (GCN) from Electroencephalogram (EEG) Signals
Energy disaggregation or nonintrusive load monitoring (NILM), is a single-input blind source discrimination problem, aims to interpret the mains user electricity consumption into appliance level measurement. This article presents a new approach for power disaggregation by using a deep recurrent long short term memory (LSTM) network combined with convolutional neural networks (CNN). Deep neural networks have been shown to be a significant way for these types of problems because of their complexity and huge number of trainable paramters. Hybrid method that proposed in the article could significa…
