Applying Machine Learning (ML) techniques to design and optimize computer architectures is a promising research direction. Optimizing the runtime performance of a Network-on-Chip (NoC) necessitates a continuous learning framework. In this work, we demonstrate the promise of applying reinforcement learning (RL) to optimize NoC runtime performance. We present three RL-based methods for learning optimal routing algorithms. The experimental results show the algorithms can successfully learn a near-optimal solution across different environment states. Reproducible Code: github.com/huckiyang/interconnect-routing-gym
Semi-Gradient SARSA Routing with Theoretical Guarantee on Traffic Stability and Weight Convergence
UM Joint Institute（UM联合研究所）；Shanghai Jiao Tong University（上海交通大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Electronics, Peking University（北京大学电子学院）
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）；Carleton University（卡尔顿大学）
Deep Reinforcement Learning (DRL) uses diverse, unstructured data and makes RL capable of learning complex policies in high dimensional environments. Intelligent Transportation System (ITS) based on Autonomous Vehicles (AVs) offers an excellent playground for policy-based DRL. Deep learning architectures solve computational challenges of traditional algorithms while helping in real-world adoption and deployment of AVs. One of the main challenges in AVs implementation is that it can worsen traffic congestion on roads if not reliably and efficiently managed. Considering each vehicle's holistic e…
Value-Decomposed Reinforcement Learning Framework for Taxiway Routing with Hierarchical Conflict-Aware Observations
National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University（合成视觉基础科学国家重点实验室，四川大学）；College of Computer Science, Sichuan University（计算机学院，四川大学）
A COllective INtelligence (COIN) is a set of interacting reinforcement learning (RL) algorithms designed in an automated fashion so that their collective behavior optimizes a global utility function. We summarize the theory of COINs, then present experiments using that theory to design COINs to control internet traffic routing. These experiments indicate that COINs outperform all previously investigated RL-based, shortest path routing algorithms.
This paper aims to develop a paradigm that models the learning behavior of intelligent agents (including but not limited to autonomous vehicles, connected and automated vehicles, or human-driven vehicles with intelligent navigation systems where human drivers follow the navigation instructions completely) with a utility-optimizing goal and the system's equilibrating processes in a routing game among atomic selfish agents. Such a paradigm can assist policymakers in devising optimal operational and planning countermeasures under both normal and abnormal circumstances. To this end, we develop a M…
We introduced the load-balanced routing algorithms, for interconnection networks resulting from nesting, by considering the pressure of the data forwarding in each node. Benchmarks on a small cluster with various network topologies, and simulations for several larger clusters whose prototypes are too costly to construct, demonstrated substantial gains of communication performance with our routing on these networks over other mainstream routing algorithms.
We present a new, deadlock-free, routing scheme for toroidal interconnection networks, called OutFlank Routing (OFR). OFR is an adaptive strategy which exploits non-minimal links, both in the source and in the destination nodes. When minimal links are congested, OFR deroutes packets to carefully chosen intermediate destinations, in order to obtain travel paths which are only an additive constant longer than the shortest ones. Since routing performance is very sensitive to changes in the traffic model or in the router parameters, an accurate discrete-event simulator of the toroidal network has …
Next to the high performance, the essential feature of the multiprocessor systems is their fault-tolerant capability. In this regard, fault-tolerant interconnection networks and especially fault-tolerant routing methods are crucial parts of these systems. Hypercube is a popular interconnection network that is used in many multiprocessors. There are several suggested practices for fault tolerant routing in these systems. In this paper, a neural routing method is introduced which is named as Fault Avoidance Routing (FAR). This method keeps the message as far from the faulty nodes as possible. Th…
Multistage interconnection networks (MINs) provide critical communication resources between network components with an attractive cost/performance relation. In this paper, a novel architecture for a MIN is proposed. In contrast to other existing networks, this new architecture operates via wormhole routing using a multi-lane equivalent-weighted fair queuing system. Associating the lane storage with each physical channel for movement of the flits allows for high levels of packet flow and makes the system more robust. The resulting flit scheduling schema has been studied thoroughly, and the resu…
In the past years, Interconnection Networks have been used quite often and especially in applications where parallelization is critical. Message packets transmitted through such networks can be interrupted using buffers in order to maximize network usage and minimize the time required for all messages to reach their destination. However, preempting a packet will result in topology reconfiguration and consequently in time cost. The problem of scheduling message packets through such a network is referred to as PBS and is known to be NP-Hard. In this paper we have improved, critically, variations…
In this paper we have addressed a very important issue of Traffic Management.Our proposed approach provides assistance in traffic routing by integrating VANET and Grid Computing
Autonomous vehicles (AV) is an advanced technology that can bring convenience, improve the road-network throughput, and reduce traffic accidents. To enable higher levels of automation (LoA), massive amounts of sensory data need to be uploaded to the network for processing, and then, maneuvering decisions must be returned to the AV. Furthermore, passengers might have a higher transmission rate demands for various data-hungry and delay-sensitive applications.
Method for calculating routing cost of MPLS network is presented in the work. This method allows to minimize routing cost, taking into account traffic fractal properties, choice of traffic transmission path and quality of service requirements. The method uses values of the Hurst parameter and value of normalized spread of traffic values, which makes it possible to apply it to self-similar and multifractal data streams.
Network traffic classification using machine learning techniques has been widely studied. Most existing schemes classify entire traffic flows, but there are major limitations to their practicality. At a network router, the packets need to be processed with minimum delay, so the classifier cannot wait until the end of the flow to make a decision. Furthermore, a complicated machine learning algorithm can be too computationally expensive to implement inside the router. In this paper, we introduce flow-packet hybrid traffic classification (FPHTC), where the router makes a decision per packet based…
Traffic-Adaptive Per-Hop Multipath Routing in Multi-Hop UAV Networks
Beijing University of Posts and Telecommunications（北京邮电大学）；China Mobile Zijin (Jiangsu) Innovation Research Institute（中国移动紫金（江苏）创新研究院）；Queen Mary University of London（伦敦大学玛丽女王学院）；China Telecom Beijing Research Institute（中国电信北京研究院）；Chongqing University（重庆大学）
We present a simple yet effective routing strategy inspired by coverage control, which delays the onset of congestion on traffic networks, by introducing a control parameter. The routing algorithm allows a trade-off between the congestion level and the distance to the destination. Numerical verification of the strategy is provided on a number of representative examples in SUMO, a well known micro agent simulator used for the analysis of traffic networks. We find that it is crucial in many cases to tune the given control parameters to some optimal value in order to reduce congestion in the most…
Today in fast technology development in wireless mobile adhoc network there is vast scope for research. As it is known that wireless communication for mobile network has many application areas like routing services, security services etc. The mobile adhoc network is the wireless network for communication in which the mobile nodes are organized without any centralized administrator. There are many Manet routing protocols like reactive, proactive, hybrid etc. In this paper the reactive Manet routing protocol like DSR is simulated for traffic analysis for 50 mobile nodes for IP traffic flows. Als…
This paper proposes a Traffic-Differentiated Two-Hop Routing protocol for Quality of Service (QoS) in Wireless Sensor Networks (WSNs). It targets WSN applications having different types of data traffic with several priorities. The protocol achieves to increase Packet Reception Ratio (PRR) and reduce end-to-end delay while considering multi-queue priority policy, two-hop neighborhood information, link reliability and power efficiency. The protocol is modular and utilizes effective methods for estimating the link metrics. Numerical results show that the proposed protocol is a feasible solution t…
We design dynamic routing policies for an overlay network which meet delay requirements of real-time traffic being served on top of an underlying legacy network, where the overlay nodes do not know the underlay characteristics. We pose the problem as a constrained MDP, and show that when the underlay implements static policies such as FIFO with randomized routing, then a decentralized policy, that can be computed efficiently in a distributed fashion, is optimal. Our algorithm utilizes multi-timescale stochastic approximation techniques, and its convergence relies on the fact that the recursion…
Delay Tolerant Networking (DTN) has been proposed as a new architecture to provide efficient store-carry-and-forward data transport in satellite networks. Since these networks relay on scheduled contact plans, the Contact Graph Routing (CGR) algorithm can be used to optimize routing and data delivery performance. However, in spite of the various improvements that have been made to CGR, there have been no significant proposals to prioritize traffic with different quality of service requirements. In this work we propose adaptations to CGR that allow performance improvements when sending traffic …
Traffic Engineering (TE) in IP carrier networks is one of the functions that can benefit from the Software Defined Networking paradigm. By logically centralizing the control of the network, it is possible to "program" per-flow routing based on TE goals. Traditional per-flow routing requires a direct interaction between the SDN controller and each node that is involved in the traffic paths. Depending on the granularity and on the temporal properties of the flows, this can lead to scalability issues for the amount of routing state that needs to be maintained in core network nodes and for the req…
The advent of real-time multimedia services over the Internet has stimulated new technologies for expanding the information carrying capacity of optical network backbones. Multilayer wavelength division multiplexing (WDM) packet switching is an emerging technology for increasing the bandwidth of optical networks. Two algorithms for the routing of the multimedia traffic flows were applied: (i) Capacitated Shortest Path First (CSPF) routing, which minimizes the distance of each flow linking the given source and destination nodes and satisfying capacity constraints; and (ii) Flow Deviation Algori…
We consider a two-player zero-sum network routing game in which a router wants to maximize the amount of legitimate traffic that flows from a given source node to a destination node and an attacker wants to block as much legitimate traffic as possible by flooding the network with malicious traffic. We address scenarios with asymmetric information, in which the router must reveal its policy before the attacker decides how to distribute the malicious traffic among the network links, which is naturally modeled by the notion of Stackelberg equilibria. The paper focuses on parallel networks, and in…
System noise can negatively impact the performance of HPC systems, and the interconnection network is one of the main factors contributing to this problem. To mitigate this effect, adaptive routing sends packets on non-minimal paths if they are less congested. However, while this may mitigate interference caused by congestion, it also generates more traffic since packets traverse additional hops, causing in turn congestion on other applications and on the application itself. In this paper, we first describe how to estimate network noise. By following these guidelines, we show how noise can be …
Wide-area network traffic engineering enables network operators to reduce congestion and improve utilization by balancing load across multiple paths. Current approaches to traffic engineering can be modeled in terms of a routing component that computes forwarding paths, and a load balancing component that maps incoming flows onto those paths dynamically, adjusting sending rates to fit current conditions. Unfortunately, existing systems rely on simple strategies for one or both of these components, which leads to poor performance or requires making frequent updates to forwarding paths, signific…
The current specification of the Segment Routing (SR) architecture requires enhancements to the intra-domain routing protocols (e.g. OSPF and IS-IS) so that the nodes can advertise the Segment Identifiers (SIDs). We propose a simpler solution called PMSR (Poor Man's Segment Routing), that does not require any enhancement to routing protocol. We compare the procedures of PMSR with traditional SR, showing that PMSR can reduce the operation and management complexity. We analyze the set of use cases in the current SR drafts and we claim that PMSR can support the large majority of them. Thanks to t…
Burst contention is a well-known challenging problem in Optical Burst Switching (OBS) networks. Contention resolution approaches are always reactive and attempt to minimize the BLR based on local information available at the core node. On the other hand, a proactive approach that avoids burst losses before they occur is desirable. To reduce the probability of burst contention, a more robust routing algorithm than the shortest path is needed. This paper proposes a new routing mechanism for JET-based OBS networks, called Graphical Probabilistic Routing Model (GPRM) that selects less utilized lin…
Hebrew University of Jerusalem（希伯来大学）；Microsoft Research（微软研究院）；Technion（以色列理工学院）
Routing is, arguably, the most fundamental task in computer networking, and the most extensively studied one. A key challenge for routing in real-world environments is the need to contend with uncertainty about future traffic demands. We present a new approach to routing under demand uncertainty: tackling this challenge as stochastic optimization, and employing deep learning to learn complex patterns in traffic demands. We show that our method provably converges to the global optimum in well-studied theoretical models of multicommodity flow. We exemplify the practical usefulness of our approac…
We show that the capacity of a complex network that models a city street grid to support congested traffic can be optimized by using routes that collectively minimize the maximum ratio of betweenness to capacity in any link. Networks with a heterogeneous distribution of link capacities and with a heterogeneous transport load are considered. We find that overall traffic congestion and average travel times can be significantly reduced by a judicious use of slower, smaller capacity links.
Segment routing is an emerging technology to simplify traffic engineering implementation in WANs. It expresses an end-to-end logical path as a sequence of segments through a set of middlepoints. Traffic along each segment is routed along shortest paths. In this paper, we study practical traffic engineering (TE) with segment routing in SDN based WANs. We consider two common types of TE, and show that the TE problem can be solved in weakly polynomial time when the number of middlepoints is fixed and not part of the input. However, the corresponding linear program is of large scale and computatio…
In this paper, we evaluate and analyze the impact of different network loads and varying no. of nodes on distance vector and link state routing algorithms. We select three well known proactive protocols; Destination Sequenced Distance Vector (DSDV) operates on distance vector routing, while Fisheye State Routing (FSR) and Optimized Link State Routing (OLSR) protocols are based on link state routing. Further, we evaluate and compare the effects on the performance of protocols by changing the routing strategies of routing algorithms. We also enhance selected protocols to achieve high performance…
Adaptive routing algorithm has been employed in multichip interconnection networks in order to improve network performance. Does a algorithm use local or global network state? This is the key question in adaptive routing. In many traffic patterns, the ignorance of global network state, leading to routing selection based only on local congestion information, tends to violate global load balance. To attack the load balance issue in adapting routing, some global adaptive routing algorithms introduce a congestion propagation network to obtain global network status information, such as Regional Con…
The recent emergence of navigational tools has changed traffic patterns and has now enabled new types of congestion-aware routing control like dynamic road pricing. Using the fundamental diagram of traffic flows - applied in macroscopic and mesoscopic traffic modeling - the article introduces a new N-player dynamic routing game with explicit congestion dynamics. The model is well-posed and can reproduce heterogeneous departure times and congestion spill back phenomena. However, as Nash equilibrium computations are PPAD-complete, solving the game becomes intractable for large but realistic numb…
Filtering packet traffic and rules of permit/denial of data packets into network nodes are granted by facilitating Access Control Lists (ACL). This paper proposes a procedure of adding a link load threshold value to the access control list rules option, which acts on the basis of threshold value. The ultimate goal of this enhanced ACL is to avoid congestion in targeted subnetworks. The link load threshold value allows to decide that packet traffic is rerouted by the router to avoid congestion, or packet drop happens on the basis of packet priorities. The packet rerouting in case of high traffi…
Due to the enormous variety of application scenarios and ubiquity,Internet of Things (IoT) brought a new perspective of applications for the current and future Internet. The Wireless Sensor Networks provide key devices for developing the IoT communication paradigm, such as the sensors collecting various kind of information and the routing and MAC protocols. However, this type of network has strong power consumption and transmission capacity restrictions (low speed wireless links and subject to interference). In this context, it is necessary to develop solutions that enable a more efficient com…
Aerial networks, composed of Unmanned Aerial Vehicles (UAVs) acting as Wi-Fi access points or cellular base stations, are emerging as an interesting solution to provide on-demand wireless connectivity to users, when there is no network infrastructure available, or to enhance the network capacity. This article proposes a traffic-aware topology control solution for aerial networks that holistically combines the placement of UAVs with a predictive and centralized routing protocol. The synergy created by the combination of the UAV placement and routing solutions allows the aerial network to seamle…
Emerging applications such as the metaverse, telesurgery or cloud computing require increasingly complex operational demands on networks (e.g., ultra-reliable low latency). Likewise, the ever-faster traffic dynamics will demand network control mechanisms that can operate at short timescales (e.g., sub-minute). In this context, Traffic Engineering (TE) is a key component to efficiently control network traffic according to some performance goals (e.g., minimize network congestion). This paper presents Routing By Backprop (RBB), a novel TE method based on Graph Neural Networks (GNN) and different…
Software Defined Networking (SDN) achieves programmability of a network through separation of the control and data planes. It enables flexibility in network management and control. Energy efficiency is one of the challenging global problems which has both economic and environmental impact. A massive amount of information is generated in the controller of an SDN based network. Machine learning gives the ability to computers to progressively learn from data without having to write specific instructions. In this work, we propose MER-SDN: a machine learning framework for traffic-aware energy effic…
The present study proposes a novel collaborative traffic engineering scheme for networks of autonomous systems. Backpressure routing principles are used for deriving priority routing rules that optimally stabilize a network, while maximizing its throughput under latency considerations. The routing rules are deployed to the network following simple SDN principles. The proposed scheme requires minimal, infrequent interaction with a central controller, limiting its imposed workload. Furthermore, it respects the internal structure of the autonomous systems and their existing peering relations. In …
State-of-the-art Internet traffic engineering uses source-based explicit routing via MPLS or Segment Routing. Though widely adopted in practice, source routing can face certain inefficiencies and operational issues, caused by its use of bandwidth reservations. In this work, we make the case for Hop-by-Hop (HBH) Traffic Engineering: splitting traffic among nexthops at every router, rather than splitting traffic among paths only at edge routers. We show that HBH traffic engineering can achieve the original goals of MPLS (i.e., efficient use of network resources), with a much simpler design that …
Elastic Optical Network (EON) has been considered as a promising optical networking technology to architect the next-generation backbone networks. Routing and Spectrum Assignment (RSA) is the fundamental problem in EONs to realize service provisioning. Generally, the RSA is solved by routing the requests with lightpaths first and then assigning spectrum resources to the lightpaths to optimize the spectrum usage. Thus, the spectrum assignment explicitly decide the final spectrum usage of EONs. However, besides the spectrum assignment, there are three other factors, the network topology, traffic…
A massive and growing part of Autonomous System (AS)-level traffic exchanges takes place at Internet Exchange Points (IXPs). This paper leverages PeeringDB, a database providing a partial but reasonable view of the global interconnection of ASes at IXPs, to model a complex graph enabling the characterization of the key Internet peering players and their interactions over time. We model a PeeringDB snapshot as a weighted directed bipartite graph, called the pDB c-graph, that captures the port size ASes possess at IXPs using available metadata. This novel model of the Internet is shown to pictur…
