Photonic reservoir computing is an emergent technology toward beyond-Neumann computing. Although photonic reservoir computing provides superior performance in environments whose characteristics are coincident with the training datasets for the reservoir, the performance is significantly degraded if these characteristics deviate from the original knowledge used in the training phase. Here, we propose a scheme of adaptive model selection in photonic reservoir computing using reinforcement learning. In this scheme, a temporal waveform is generated by different dynamic source models that change over time. The system autonomously identifies the best source model for the task of time series prediction using photonic reservoir computing and reinforcement learning. We prepare two types of output weights for the source models, and the system adaptively selected the correct model using reinforcement learning, where the prediction errors are associated with rewards. We succeed in adaptive model selection when the source signal is temporally mixed, having originally been generated by two different dynamic system models, as well as when the signal is a mixture from the same model but with different parameter values. This study paves the way for autonomous behavior in photonic artificial intelligence and could lead to new applications in load forecasting and multi-objective control, where frequent environment changes are expected.
To perform temporal and sequential machine learning tasks, the use of conventional Recurrent Neural Networks (RNNs) has been dwindling due to the training complexities of RNNs. To this end, accelerators for delayed feedback reservoir computing (DFRC) have attracted attention in lieu of RNNs, due to their simple hardware implementations. A typical implementation of a DFRC accelerator consists of a delay loop and a single nonlinear neuron, together acting as multiple virtual nodes for computing. In prior work, photonic DFRC accelerators have shown an undisputed advantage of fast computation over…
As Moore's law comes to an end, neuromorphic approaches to computing are on the rise. One of these, passive photonic reservoir computing, is a strong candidate for computing at high bitrates (> 10 Gbps) and with low energy consumption. Currently though, both benefits are limited by the necessity to perform training and readout operations in the electrical domain. Thus, efforts are currently underway in the photonic community to design an integrated optical readout, which allows to perform all operations in the optical domain. In addition to the technological challenge of designing such a reado…
Photonic reservoir computing is a promising candidate for low-energy computing at high bandwidths. Despite recent successes, there are bounds to what one can achieve simply by making photonic reservoirs larger. Therefore, a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that backpropagation can not be used directly to train multi-reservoir systems in our targeted setting, we propose an alternative approach that still uses its power to derive intermediate targets. In this work we report our findings on a conducted experim…
The implementation of artificial neural networks in hardware substrates is a major interdisciplinary enterprise. Well suited candidates for physical implementations must combine nonlinear neurons with dedicated and efficient hardware solutions for both connectivity and training. Reservoir computing addresses the problems related with the network connectivity and training in an elegant and efficient way. However, important questions regarding impact of reservoir size and learning routines on the convergence-speed during learning remain unaddressed. Here, we study in detail the learning process …
Graduate school of Systems and Information Engineering（系统与信息工程研究生院）；University of Tsukuba（筑波大学）
Recently, reinforcement learning models have achieved great success, mastering complex tasks such as Go and other games with higher scores than human players. Many of these models store considerable data on the tasks and achieve high performance by extracting visual and time-series features using convolutional neural networks (CNNs) and recurrent neural networks, respectively. However, these networks have very high computational costs because they need to be trained by repeatedly using the stored data. In this study, we propose a novel practical approach called reinforcement learning with conv…
Instituto de Física Interdisciplinar y Sistemas Complejos, IFISC (UIB-CSIC)（跨学科物理与复杂系统研究所（IFISC，巴利阿里群岛大学-西班牙国家研究委员会））；Ghent University（根特大学）
Nonlinear photonic delay systems present interesting implementation platforms for machine learning models. They can be extremely fast, offer great degrees of parallelism and potentially consume far less power than digital processors. So far they have been successfully employed for signal processing using the Reservoir Computing paradigm. In this paper we show that their range of applicability can be greatly extended if we use gradient descent with backpropagation through time on a model of the system to optimize the input encoding of such systems. We perform physical experiments that demonstra…
Recently, reinforcement learning models have achieved great success, completing complex tasks such as mastering Go and other games with higher scores than human players. Many of these models collect considerable data on the tasks and improve accuracy by extracting visual and time-series features using convolutional neural networks (CNNs) and recurrent neural networks, respectively. However, these networks have very high computational costs because they need to be trained by repeatedly using a large volume of past playing data. In this study, we propose a novel practical approach called reinfor…
In this paper we give a profound insight into the computation capability of delay-based reservoir computing via an eigenvalue analysis. We concentrate on the task-independent memory capacity to quantify the reservoir performance and compare these with the eigenvalue spectrum of the dynamical system. We show that these two quantities are deeply connected, and thus the reservoir computing performance is predictable by analyzing the small signal response of the reservoir. Our results suggest that any dynamical system used as a reservoir can be analyzed in this way. We apply our method exemplarily…
Reservoir computing is a recent bio-inspired approach for processing time-dependent signals. It has enabled a breakthrough in analog information processing, with several experiments, both electronic and optical, demonstrating state-of-the-art performances for hard tasks such as speech recognition, time series prediction and nonlinear channel equalization. A proof-of-principle experiment using a linear optical circuit on a photonic chip to process digital signals was recently reported. Here we present a photonic implementation of a reservoir computer based on a coherently driven passive fiber c…
Reinforcement learning has been intensively investigated and developed in artificial intelligence in the absence of training data, such as autonomous driving vehicles, robot control, internet advertising, and elastic optical networks. However, the computational cost of reinforcement learning with deep neural networks is extremely high and reducing the learning cost is a challenging issue. We propose a photonic on-line implementation of reinforcement learning using optoelectronic delay-based reservoir computing, both experimentally and numerically. In the proposed scheme, we accelerate reinforc…
Nowadays, Information Photonics is extensively studied and sees applications in many fields. The interest in this breakthrough technology is mainly stimulated by the possibility of achieving real-time data processing for high-bandwidth applications, still implemented through small-footprint devices that would allow for breaking the limit imposed by Moore's law. One potential breakthrough implementation of information photonics is via integrated photonic circuits. Within this approach, the most suitable computational scheme is achieved by integrated photonic neural networks. In this chapter, we…
Today's unrelenting increase in demand for information processing creates the need for novel computing concepts. Reservoir computing is such a concept that lends itself particularly well to photonic hardware implementations. Over recent years, these hardware implementations have gained maturity and now achieve state-of-the-art performance on several benchmark tasks. However, implementations so far are essentially all based on sequential data processing, leaving the inherent parallelism of photonics unexploited. Parallel implementations process all neurons simultaneously, and therefore have the…
Driven by the remarkable breakthroughs during the past decade, photonics neural networks have experienced a revival. Here, we provide a general overview of progress over the past decade, and sketch a roadmap of important future developments. We focus on photonic implementations of the reservoir computing machine learning paradigm, which offers a conceptually simple approach that is amenable to hardware implementations. In particular, we provide an overview of photonic reservoir computing implemented via either spatio temporal or delay dynamical systems. Going beyond reservoir computing, we dis…
Information processing has reached the era of big data. Big data challenges are difficult to address with traditional Von Neumann or Turing approach. Hence implementation of new computational techniques is highly essential. Nanophotonics with its remarkable speed and multiplexing capability is a promising candidate for such implementations. This paper proposes a novel photonic computing system made-up of Mach-Zehnder interferometer and an optical fiber spool to emulate a powerful machine learning technique called reservoir computing. The proposed system is also integrated with a time-division-…
Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However,there is still an misconception about the relationship between the delay-time and the input clock-cycle which has noticeable consequences for the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that there is no predefined relationship between these two times-scales. Further, we discuss ways to improve the computing performance of a res…
High-dimensional nonlinear dynamical systems including neural networks can be utilized as a computational resource for information processing. In this sense, nonlinear wave systems are good candidate for such a computational resource. Here, we propose and numerically demonstrate information processing based on nonlinear wave dynamics in microcavity lasers, i.e., optical spatiotemporal systems at a microscale. One of the remarkable features is the ability of high-dimensional and nonlinear mapping of input information into the wave states, enabling efficient and fast information processing at a …
Photonic delay systems have revolutionized the hardware implementation of Recurrent Neural Networks and Reservoir Computing in particular. The fundamental principles of Reservoir Computing strongly benefit a realization in such complex analog systems. Especially delay systems, potentially providing large numbers of degrees of freedom even in simple architectures, can efficiently be exploited for information processing. The numerous demonstrations of their performance led to a revival of photonic Artificial Neural Network. Today, an astonishing variety of physical substrates, implementation tec…
Reservoir computing is a relatively recent computational paradigm that originates from a recurrent neural network and is known for its wide range of implementations using different physical technologies. Large reservoirs are very hard to obtain in conventional computers, as both the computation complexity and memory usage grow quadratically. We propose an optical scheme performing reservoir computing over very large networks potentially being able to host several millions of fully connected photonic nodes thanks to its intrinsic properties of parallelism and scalability. Our experimental studi…
The incorporation of high-performance optoelectronic devices into photonic neuromorphic processors can substantially accelerate computationally intensive operations in machine learning (ML) algorithms. However, the conventional device design wisdom is disconnected with system optimization. We report a device-system co-design methodology to optimize a free-space optical general matrix multiplication (GEMM) hardware accelerator by engineering a spatially reconfigurable array made from chalcogenide phase change materials. With a highly-parallelized hardware emulator constructed based on experimen…
The recent rapid increase in demand for data processing has resulted in the need for novel machine learning concepts and hardware. Physical reservoir computing and an extreme learning machine are novel computing paradigms based on physical systems themselves, where the high dimensionality and nonlinearity play a crucial role in the information processing. Herein, we propose the use of multidimensional speckle dynamics in multimode fibers for information processing, where input information is mapped into the space, frequency, and time domains by an optical phase modulation technique. The speckl…
Photonic neural networks have significant potential for high-speed neural processing with low latency and ultralow energy consumption. However, the on-chip implementation of a large-scale neural network is still challenging owing to its low scalability. Herein, we propose the concept of a photonic neural field and implement it experimentally on a silicon chip to realize highly scalable neuro-inspired computing. In contrast to existing photonic neural networks, the photonic neural field is a spatially continuous field that nonlinearly responds to optical inputs, and its high spatial degrees of …
Reservoir Computing is a relatively recent computational framework based on a large Recurrent Neural Network with fixed weights. Many physical implementations of Reservoir Computing have been proposed to improve speed and energy efficiency. In this study, we report new advances in Optical Reservoir Computing using multiple light scattering to accelerate the recursive computation of the reservoir states. Two different spatial light modulation technologies, namely, phase or binary amplitude modulations, are compared. Phase modulation is a promising direction already employed in other photonic im…
The University of Tokyo（东京大学）；Kanazawa University（金泽大学）；Saitama University（埼玉大学）
Accelerating artificial intelligence by photonics is an active field of study aiming to exploit the unique properties of photons. Reinforcement learning is an important branch of machine learning, and photonic decision-making principles have been demonstrated with respect to the multi-armed bandit problems. However, reinforcement learning could involve a massive number of states, unlike previously demonstrated bandit problems where the number of states is only one. Q-learning is a well-known approach in reinforcement learning that can deal with many states. The architecture of Q-learning, howe…
National Institute of Information and Communications Technology（国立信息通信技术研究所）；Saitama University（埼玉大学）；National Institute for Materials Science（国立材料科学研究所）
Reinforcement learning involves decision making in dynamic and uncertain environments, and constitutes one important element of artificial intelligence (AI). In this paper, we experimentally demonstrate that the ultrafast chaotic oscillatory dynamics of lasers efficiently solve the multi-armed bandit problem (MAB), which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. To solve the MAB, a certain degree of randomness is required for exploration purposes. However, pseudo-random numbers generated using conventional electronic circui…
National Institute of Information and Communications Technology（信息通信研究机构）；Saitama University（埼玉大学）；University of Yamanashi（山梨大学）；Nagahama Institute of Bio-Science and Technology（长滨生物科学与技术学院）；Nagoya University（名古屋大学）；Tokyo University of Science（东京理科大学）
Reinforcement learning involves decision making in dynamic and uncertain environments and constitutes a crucial element of artificial intelligence. In our previous work, we experimentally demonstrated that the ultrafast chaotic oscillatory dynamics of lasers can be used to solve the two-armed bandit problem efficiently, which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. However, only two selections were employed in that research; thus, the scalability of the laser-chaos-based reinforcement learning should be clarified. In this…
Nonlinear waves have played a historical role in laying the foundations of the science of complexity. Recently, they have also allowed the development of a new reservoir computing paradigm: neuromorphic computing by waves. In these systems, the information transmission acts as the excitation of wave dynamics, whose evolution processes the information to perform complex tasks at low energy consumption. To enable nonlinear hydrodynamic waves to do computing, we designed the Aqua-Photonic-Advantaged Computing Machine by Artificial Neural Networks (Aqua-PACMANN), a system where wave propagation in…
