To tackle the heterogeneous requirements of beyond 5G (B5G) and future 6G wireless networks, conventional medium access control (MAC) procedures need to evolve to enable base stations (BSs) and user equipments (UEs) to automatically learn innovative MAC protocols catering to extremely diverse services. This topic has received significant attention, and several reinforcement learning (RL) algorithms, in which BSs and UEs are cast as agents, are available with the aim of learning a communication policy based on agents' local observations. However, current approaches are typically overfitted to the environment they are trained in, and lack robustness against unseen conditions, failing to generalize in different environments. To overcome this problem, in this work, instead of learning a policy in the high dimensional and redundant observation space, we leverage the concept of observation abstraction (OA) rooted in extracting useful information from the environment. This in turn allows learning communication protocols that are more robust and with much better generalization capabilities than current baselines. To learn the abstracted information from observations, we propose an architecture based on autoencoder (AE) and imbue it into a multi-agent proximal policy optimization (MAPPO) framework. Simulation results corroborate the effectiveness of leveraging abstraction when learning protocols by generalizing across environments, in terms of number of UEs, number of data packets to transmit, and channel conditions.
DeepMind（深度思维）；Imperial College London（伦敦帝国理工学院）；Santa Fe Institute（圣塔菲研究所）
We characterise the problem of abstraction in the context of deep reinforcement learning. Various well established approaches to analogical reasoning and associative memory might be brought to bear on this issue, but they present difficulties because of the need for end-to-end differentiability. We review developments in AI and machine learning that could facilitate their adoption.
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）；Pontificia Universidad Católica de Chile（智利天主教大学）；Centro Nacional de Inteligencia Artificial（国家人工智能中心）
Natural and formal languages provide an effective mechanism for humans to specify instructions and reward functions. We investigate how to generate policies via RL when reward functions are specified in a symbolic language captured by Reward Machines, an increasingly popular automaton-inspired structure. We are interested in the case where the mapping of environment state to a symbolic (here, Reward Machine) vocabulary -- commonly known as the labelling function -- is uncertain from the perspective of the agent. We formulate the problem of policy learning in Reward Machines with noisy symbolic…
The ability to acquire abstract knowledge is a hallmark of human intelligence and is believed by many to be one of the core differences between humans and neural network models. Agents can be endowed with an inductive bias towards abstraction through meta-learning, where they are trained on a distribution of tasks that share some abstract structure that can be learned and applied. However, because neural networks are hard to interpret, it can be difficult to tell whether agents have learned the underlying abstraction, or alternatively statistical patterns that are characteristic of that abstra…
Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive biases, agents trained by meta-learning may sometimes acquire very different strategies from humans. We show that co-training these agents on predicting representations from natural language task descriptions and programs induced to generate such tasks guides them toward more human-like inductive biases. Human-generated language descriptions and program induction models that add new learned primitives both contain abst…
Tsinghua University（清华大学）；University of Science and Technology Beijing（北京科技大学）；University of South Florida（南佛罗里达大学）；University of Southampton（南安普顿大学）
Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and intera…
In this paper we take the first steps in studying a new approach to synthesis of efficient communication schemes in multi-agent systems, trained via reinforcement learning. We combine symbolic methods with machine learning, in what is referred to as a neuro-symbolic system. The agents are not restricted to only use initial primitives: reinforcement learning is interleaved with steps to extend the current language with novel higher-level concepts, allowing generalisation and more informative communication via shorter messages. We demonstrate that this approach allow agents to converge more quic…
Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON N6A 5B9, Canada（加拿大安大略省伦敦市西安大略大学电气与计算机工程系）
The fifth generation (5G) and beyond wireless networks are critical to support diverse vertical applications by connecting heterogeneous devices and machines, which directly increase vulnerability for various spoofing attacks. Conventional cryptographic and physical layer authentication techniques are facing some challenges in complex dynamic wireless environments, including significant security overhead, low reliability, as well as difficulty in pre-designing authentication model, providing continuous protections, and learning time-varying attributes. In this article, we envision new authenti…
Fudan University（复旦大学）；Shanghai University（上海大学）；Commonwealth Scientific and Industrial Research Organization (CSIRO)（联邦科学与工业研究组织（CSIRO））；University of Manitoba（曼尼托巴大学）
Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications. This is due to improved capabilities of terminal devices, explosively growing data volume, congestion in the radio interfaces, and increasing concern of data privacy. The unique features of wireless systems, such as large scale, geographically dispersed deployment, user mobility, and massive amount of data, give rise to new challenges in the design of DML techniques. There is a clear gap in the existi…
The Internet of Things (IoT) is expected to require more effective and efficient wireless communications than ever before. For this reason, techniques such as spectrum sharing, dynamic spectrum access, extraction of signal intelligence and optimized routing will soon become essential components of the IoT wireless communication paradigm. Given that the majority of the IoT will be composed of tiny, mobile, and energy-constrained devices, traditional techniques based on a priori network optimization may not be suitable, since (i) an accurate model of the environment may not be readily available …
Ghent University（根特大学）；Delft University of Technology（代尔夫特理工大学）
This paper provides a systematic and comprehensive survey that reviews the latest research efforts focused on machine learning (ML) based performance improvement of wireless networks, while considering all layers of the protocol stack (PHY, MAC and network). First, the related work and paper contributions are discussed, followed by providing the necessary background on data-driven approaches and machine learning for non-machine learning experts to understand all discussed techniques. Then, a comprehensive review is presented on works employing ML-based approaches to optimize the wireless commu…
One of the major challenges for the engineering of wireless sensing systems is to improve the software abstractions and frameworks that are available to programmers while ensuring system reliability and efficiency. The distributed systems community have developed a rich set of such abstractions for building dependable distributed systems connected using wired networks, however after 20 years research many of these elude wireless sensor systems. In this paper we present X Process Commit (XPC) an atomic commit protocol framework that utilizes Synchronous Transmission (ST). We also introduce Hybr…
Energy Efficiency in wireless sensor networks is an important topic in which the nodes rely on battery power, and efficient energy usage is a key issue for sensitive applications that require long working times. This stimulates many scientists at all levels of communication protocols Medium Access Control (MAC) who control the use of the wireless transmitter and receiver unit to create new protocols. Many protocols were suggested that primarily take energy efficiency as the primary objective of sustaining the function of the network for as long as possible into account with different objective…
Multi-beam antenna technologies have provided lots of promising solutions to many current challenges faced in wireless mesh networks. The antenna can establish several beamformings simultaneously and initiate concurrent transmissions or receptions using multiple beams, thereby increasing the overall throughput of the network transmission. Multi-beam antenna has the ability to increase the spatial reuse, extend the transmission range, improve the transmission reliability, as well as save the power consumption. Traditional Medium Access Control (MAC) protocols for wireless network largely relied…
This paper presents the design, implementation and performance evaluation of a location MAC protocol, called L-MAC, for wireless sensor networks. L-MAC is a combination of TDMA and CSMA while offsetting the high overhead of time slot assignment by allocating the time slots to sensor nodes based on their location information. This design avoids high computation complexity of time slot assignment incurred by node mobility and node failure. The area which the wireless sensor network occupies is divided into blocks and each block is associated with an inter-block time slot and an intra-block time …
Recently, significant research attention has been devoted to the study of reconfigurable intelligent surfaces (RISs), which are capable of reconfiguring the wireless propagation environment by exploiting the unique properties of metamaterials-based integrated large arrays of inexpensive antennas. Existing research demonstrates that RISs significantly improve the physical layer performance, including the wireless coverage, achievable data rate and energy efficiency. However, the medium access control (MAC) of multiple users accessing an RIS-enabled channel is still in its infancy, while many op…
In this paper we present new optimization formulations for maximizing the network lifetime in wireless mesh networks performing data aggregation and dissemination for machine-to-machine communication in the Internet of Things. We focus on heterogeneous networks in which multiple applications co-exist and nodes may take on different roles for different applications. Moreover, we address network reconfiguration as a means to increase the network lifetime, in keeping with the current trend towards software defined networks and network function virtualization. To test our optimization formulations…
Ensuring a reliable communication in wireless networks strictly depends on the effective estimation of the link quality, which is particularly challenging when propagation environment for radio signals significantly varies. In such environments, intelligent algorithms that can provide robust, resilient and adaptive links are being investigated to complement traditional algorithms in maintaining a reliable communication. In this respect, the data-driven link quality estimation (LQE) using machine learning (ML) algorithms is one of the most promising approaches. In this paper, we provide a quant…
A node equipped with a multi-beam antenna can achieve a throughput of up to m times as compared to a single-beam antenna, by simultaneously communicating on its m non-interfering beams. However, the existing multi-beam medium access control (MAC) schemes can achieve concurrent data communication only when the transmitter nodes are locally synchronized. Asynchronous packet arrival at a multi-beam receiver node would increase the node deafness and MAC layer capture problems, and thereby limit the data throughput. This paper presents an asynchronous multi-beam MAC protocol for multi-hop wireless …
We describe a new synchronous and distributed medium access control (MAC) protocol for multihop wireless networks that provides bandwidth guarantees to unicast connections. Our MAC protocol is based on a slotted time division multiple access (TDMA) architecture, with a multi-mini-slotted signaling phase scheduling data transmissions over slots in the following data phase. Resolving contentions at the beginning of a frame allows for effective utilization of bandwidth. Our protocol essentially combines the benefits of TDMA architecture with the distributed reservation mechanism of IEEE 802.11 MA…
Comparison Based Analysis of Different Cryptographic and Encryption Techniques Using Message Authentication Code (MAC) in Wireless Sensor Networks (WSN)
Evaluating Feasibility of Using Wireless Sensor Networks in a Coffee Crop Through Simulation of AODV, AOMDV, DSDV and Their Variants with 802.15.4 Mac Protocol
Recent advances in antenna technology have made the design of multi-beam antennas (MBA) feasible. Compared to an omni-directional or a single beam directional antenna, an MBA equipped node can achieve a throughput of up to m times, by simultaneously communicating on its m non-interfering beams. As a result, a few multi-beam directional medium access control (MAC) schemes have been proposed in the literature recently, which are implemented mostly on the in-house simulation setups in Matlab or C/C++. These implementations make many assumptions to simplify their design, without a thorough impleme…
This paper investigates a new class of carrier-sense multiple access (CSMA) protocols that employ deep reinforcement learning (DRL) techniques for heterogeneous wireless networking, referred to as carrier-sense deep-reinforcement learning multiple access (CS-DLMA). Existing CSMA protocols, such as the medium access control (MAC) of WiFi, are designed for a homogeneous network environment in which all nodes adopt the same protocol. Such protocols suffer from severe performance degradation in a heterogeneous environment where there are nodes adopting other MAC protocols. This paper shows that DR…
Critical Comparative Analysis and Recommendation in MAC Protocols for Wireless Mesh Networks Using Multi-objective Optimization and Statistical Testing
针对 IEEE 802.11s WMN 的关键 MAC 问题，本文综述 MAC 协议及 QoS 参数，并提出结合多目标优化与统计检验的比较分析及协议推荐框架。
We have witnessed an exponential growth in commercial data services, which has lead to the 'big data era'. Machine learning, as one of the most promising artificial intelligence tools of analyzing the deluge of data, has been invoked in many research areas both in academia and industry. The aim of this article is twin-fold. Firstly, we briefly review big data analysis and machine learning, along with their potential applications in next-generation wireless networks. The second goal is to invoke big data analysis to predict the requirements of mobile users and to exploit it for improving the pe…
Reconfigurable intelligent surface (RIS) is a promising reflective radio technology for improving the coverage and rate of future wireless systems by reconfiguring the wireless propagation environment. The current work mainly focuses on the physical layer design of RIS. However, enabling multiple devices to communicate with the assistance of RIS is a crucial challenging problem. Motivated by this, we explore RIS-assisted communications at the medium access control (MAC) layer and propose an RIS-assisted MAC framework. In particular, RISassisted transmissions are implemented by pre-negotiation …
