Vehicular crowdsensing is anticipated to become a key catalyst for data-driven optimization in the Intelligent Transportation System (ITS) domain. Yet, the expected growth in massive Machine-type Communication (mMTC) caused by vehicle-to-cloud transmissions will confront the cellular network infrastructure with great capacity-related challenges. A cognitive way for achieving relief without introducing additional physical infrastructure is the application of opportunistic data transfer for delay-tolerant applications. Hereby, the clients schedule their data transmissions in a channel-aware manner in order to avoid retransmissions and interference with other cell users. In this paper, we introduce a novel approach for this type of resourceaware data transfer which brings together supervised learning for network quality prediction with reinforcement learningbased decision making. The performance evaluation is carried out using data-driven network simulation and real world experiments in the public cellular networks of multiple Mobile Network Operators (MNOs) in different scenarios. The proposed transmission scheme significantly outperforms state-of-the-art probabilistic approaches in most scenarios and achieves data rate improvements of up to 181% in uplink and up to 270% in downlink transmission direction in comparison to conventional periodic data transfer.
In the Vehicular ad hoc networks (VANETs), due to the high mobility of vehicles, the network parameters change frequently and the information which the sender maintains may outdate when it wants to transmit data packet to the receiver, so for improving the routing effective, we propose the probability prediction based reliable (PRO) opportunistic routing for VANETs. The PRO routing algorithm can predict the variation of Signal to Interference plus Noise Ratio (SINR) and packet queue length (PQL) in the receiver. The prediction results are used to determine the utility of each relaying vehicle …
Both caching and interference alignment (IA) are promising techniques for future wireless networks. Nevertheless, most of existing works on cache-enabled IA wireless networks assume that the channel is invariant, which is unrealistic considering the time-varying nature of practical wireless environments. In this paper, we consider realistic time-varying channels. Specifically, the channel is formulated as a finite-state Markov channel (FSMC). The complexity of the system is very high when we consider realistic FSMC models. Therefore, we propose a novel big data reinforcement learning approach …
Routing packets opportunistically is an essential part of multihop ad hoc wireless sensor networks. The existing routing techniques are not adaptive opportunistic. In this paper we have proposed an adaptive opportunistic routing scheme that routes packets opportunistically in order to ensure that packet loss is avoided. Learning and routing are combined in the framework that explores the optimal routing possibilities. In this paper we implemented this Reinforced learning framework using a customer simulator. The experimental results revealed that the scheme is able to exploit the opportunistic…
The Internet of Things (IoT) revolution and the development of smart communities have resulted in increased demand for bandwidth due to the rise in network traffic. Instead of investing in expensive communications infrastructure, some researchers have proposed leveraging Vehicular Ad-Hoc Networks (VANETs) as the data communications infrastructure. However VANETs are not cheap since they require the deployment of expensive Road Side Units (RSU)s across smart communities. In this research, we propose an infrastructure-less system that opportunistically utilizes vehicles to serve as Local Communi…
With the increasing interest in connected vehicles, it is useful to evaluate the capability of delivering large content over a WiFi infrastructure to vehicles. The throughput achieved over WiFi channels can be highly variable and also rapidly degrades as the distance from the access point increases. While this behavior is well understood at the data link layer, the interactions across the various protocol layers (data link and up through the transport layer) and the effect of mobility may reduce the amount of content transferred to the vehicle, as it travels along the roadway. This paper exami…
Reinforcement Learning for Opportunistic Routing in Software-Defined LEO-Terrestrial Systems
School of Electrical and Mechanical Engineering, University of Adelaide（阿德莱德大学电气与机械工程学院）；Singapore University of Technology and Design（新加坡科技设计大学）；Electronics and Telecommunications Research Institute（电子电信研究院）
This paper investigates distributed processing in Vehicular Edge Cloud (VECs), where a group of vehicles in a car park, at a charging station or at a road traffic intersection, cluster and form a temporary vehicular cloud by combining their computational resources in the cluster. We investigated the problem of energy efficient processing task allocation in VEC by developing a Mixed Integer Linear Programming (MILP) model to minimize power consumption by optimizing the allocation of different processing tasks to the available network resources, cloud resources, fog resources and vehicular proce…
Institute for Informatics and Telematics, National Research Council（国家研究委员会信息与远程信息处理研究所）
The widespread diffusion of mobile phones is triggering an exponential growth of mobile data traffic that is likely to cause, in the near future, considerable traffic overload issues even in last-generation cellular networks. Offloading part of the traffic to other networks is considered a very promising approach and, in particular, in this paper, we consider offloading through opportunistic networks of users' devices. However, the performance of this solution strongly depends on the pattern of encounters between mobile nodes, which should therefore be taken into account when designing offload…
Shesha: Opportunistic In-network Acceleration of Asynchronous Distributed Reinforcement Learning
Leibniz University Hannover（汉诺威莱布尼茨大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；University of Waterloo（滑铁卢大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Evaluating Performance Characteristic of Opportunistic Routing Protocols: A Case Study of the 2016 Italian League Match Earthquake in the Stadio Adriatico
University of Nottingham（诺丁汉大学）
本文通过2016年意大利地震场景评估机会路由协议性能，采用Epidemic和Spray and Wait协议，在相同模拟条件下比较交付概率、延迟、开销比、跳数和丢包率，发现Spray and Wait在可靠性与效率间取得更好平衡。
The 5G Internet of Vehicles has become a new paradigm alongside the growing popularity and variety of computation-intensive applications with high requirements for computational resources and analysis capabilities. Existing network architectures and resource management mechanisms may not sufficiently guarantee satisfactory Quality of Experience and network efficiency, mainly suffering from coverage limitation of Road Side Units, insufficient resources, and unsatisfactory computational capabilities of onboard equipment, frequently changing network topology, and ineffective resource management s…
Major wireless operators are nowadays facing network capacity issues in striving to meet the growing demands of mobile users. At the same time, 3G-enabled devices increasingly benefit from ad hoc radio connectivity (e.g., Wi-Fi). In this context of hybrid connectivity, we propose Push-and-track, a content dissemination framework that harnesses ad hoc communication opportunities to minimize the load on the wireless infrastructure while guaranteeing tight delivery delays. It achieves this through a control loop that collects user-sent acknowledgements to determine if new copies need to be reinje…
Emergency communication is extremely important to aid rescue and search operation in the aftermath of any disaster. In such scenario, Unmanned Aerial Vehicle (UAV) networks may be used to complement the damaged cellular networks over large areas. However, in such UAV networks, routing is a challenge, owing to high UAV mobility, intermittent link quality between UAVs, dynamic three dimensional (3D) UAV topology and resource constraints. Though several UAV routing approaches have been proposed, none of them so far have addressed inter UAV coverage, collision and routing in an integrated manner. …
WiFi densification leads to the existence of multiple overlapping coverage areas, which allows user stations (STAs) to choose between different Access Points (APs). The standard WiFi association method makes the STAs select the AP with the strongest signal, which in many cases leads to underutilization of some APs while overcrowding others. To mitigate this situation, Reinforcement Learning techniques such as Multi-Armed Bandits can be used to dynamically learn the optimal mapping between APs and STAs, and so redistribute the STAs among the available APs accordingly. This is an especially chal…
A variety of mathematical tools have been developed for predicting the spreading patterns in a number of varied environments including infectious diseases, computer viruses, and urgent messages broadcast to mobile agent (e.g., humans, vehicles, and mobile devices). These tools have mainly focused on estimating the average time for the spread to reach a fraction (e.g., $α$) of the agents, i.e., the so-called average completion time $E(T_α)$. We claim that providing probabilistic guarantee on the time for the spread $T_α$ rather than only its average gives a much better understanding of the spre…
This letter proposes a neural network (NN) aided remote unmanned aerial vehicle (UAV) online control algorithm, coined oHJB. By downloading a UAV's state, a base station (BS) trains an HJB NN that solves the Hamilton-Jacobi-Bellman equation (HJB) in real time, yielding the optimal control action. Initially, the BS uploads this control action to the UAV. If the HJB NN is sufficiently trained and the UAV is far away, the BS uploads the HJB NN model, enabling to locally carry out control decisions even when the connection is lost. Simulations corroborate the effectiveness of oHJB in reducing the …
Vehicle platooning, one of the advanced services supported by 5G NR-V2X, improves traffic efficiency in the connected intelligent transportation systems (C-ITSs). However, the packet delivery ratio of platoon communication, especially in the out-of-coverage area, is significantly impacted by the random selection algorithms employed in the current resource allocation scheme. In this paper, we first analyze the collision probability via the random selection algorithm adopted in the current standard. Subsequently, we then investigate the deep reinforcement learning (DRL) algorithm that decreases …
In millimeter wave (mmWave) vehicular communications, multi-hop relay disconnection by line-of-sight (LOS) blockage is a critical problem, especially in the early diffusion phase of mmWave-available vehicles, where not all the vehicles have mmWave communication devices. This paper proposes a distributed position control method for autonomous vehicles to make long relays connecting to road side units (RSUs) by avoiding blockages to communicate with each other via LOS paths. Even though vehicles with the proposed method do not use the whole information of the environments and cooperate with each…
Major wireless operators are nowadays facing network capacity issues in striving to meet the growing demands of mobile users. At the same time, 3G-enabled devices increasingly benefit from ad hoc radio connectivity (e.g., Wi-Fi). In this context of hybrid connectivity, we propose Push-and-track, a content dissemina- tion framework that harnesses ad hoc communication opportunities to minimize the load on the wireless infrastructure while guaranteeing tight delivery delays. It achieves this through a control loop that collects user-sent acknowledgements to determine if new copies need to be rein…
The exploitation of vehicles as mobile sensors acts as a catalyst for novel crowdsensing-based applications such as intelligent traffic control and distributed weather forecast. However, the massive increases in Machine-type Communication (MTC) highly stress the capacities of the network infrastructure. With the system-immanent limitation of resources in cellular networks and the resource competition between human cell users and MTC, more resource-efficient channel access methods are required in order to improve the coexistence of the different communicating entities. In this paper, we present…
Traditional concept of cognitive radio is the coexistence of primary and secondary user in multiplexed manner. we consider the opportunistic channel access scheme in IEEE 802.11 based networks subject to the interference mitigation scenario. According to the protocol rule and due to the constraint of message passing, secondary user is unaware of the exact state of the primary user. In this paper, we have proposed an online algorithm for the secondary which assist determining a backoff counter or the decision of being idle for utilizing the time/frequency slot unoccupied by the primary user. Pr…
For in-vehicle application, the vehicles with different speeds have different delay requirements. However, vehicle speeds have not been extensively explored, which may cause mismatching between vehicle speed and its allocated computation and wireless resource. In this paper, we propose a vehicle speed aware task offloading and resource allocation strategy, to decrease the energy cost of executing tasks without exceeding the delay constraint. First, we establish the vehicle speed aware delay constraint model based on different speeds and task types. Then, the delay and energy cost of task execu…
The growth of 5G and edge computing has enabled the emergence of Internet of Vehicles. It supports different types of services with different resource and service requirements. However, limited resources at the edge, high mobility of vehicles, increasing demand, and dynamicity in service request-types have made service placement a challenging task. A typical static placement solution is not effective as it does not consider the traffic mobility and service dynamics. Handling dynamics in IoV for service placement is an important and challenging problem which is the primary focus of our work in …
The interconnection of vehicles in the future fifth generation (5G) wireless ecosystem forms the so-called Internet of vehicles (IoV). IoV offers new kinds of applications requiring delay-sensitive, compute-intensive and bandwidth-hungry services. Mobile edge computing (MEC) and network slicing (NS) are two of the key enabler technologies in 5G networks that can be used to optimize the allocation of the network resources and guarantee the diverse requirements of IoV applications. As traditional model-based optimization techniques generally end up with NP-hard and strongly non-convex and non-li…
as a novel client-based method for resource-efficient opportunistic transmission of delay-tolerant vehicular sensor data. BS-CB applies a hybrid approach which brings together all major machine learning disciplines - supervised, unsupervised, and reinforcement learning - in order to autonomously schedule vehicular sensor data transmissions with respect to the expected resource efficiency. Within a comprehensive real world performance evaluation in the public cellular networks of three Mobile Network Operators (MNOs), it is found that 1) The average uplink data rate is improved by 125%-195% 2) …
Huawei Technologies German Research Center（华为技术德国研究中心）
Radio resources in vehicle-to-vehicle (V2V) communication can be scheduled either by a centralized scheduler residing in the network (e.g., a base station in case of cellular systems) or a distributed scheduler, where the resources are autonomously selected by the vehicles. The former approach yields a considerably higher resource utilization in case the network coverage is uninterrupted. However, in case of intermittent or out-of-coverage, due to not having input from centralized scheduler, vehicles need to revert to distributed scheduling. Motivated by recent advances in reinforcement learni…
Huawei Technologies Duesseldorf GmbH（华为技术杜塞尔多夫有限公司）；Technische Universität Berlin（柏林工业大学）
Vehicle-to-vehicle (V2V) communications have distinct challenges that need to be taken into account when scheduling the radio resources. Although centralized schedulers (e.g., located on base stations) could be utilized to deliver high scheduling performance, they cannot be employed in case of coverage gaps. To address the issue of reliable scheduling of V2V transmissions out of coverage, we propose Vehicular Reinforcement Learning Scheduler (VRLS), a centralized scheduler that predictively assigns the resources for V2V communication while the vehicle is still in cellular network coverage. VRL…
