CFR-RL: Traffic Engineering with Reinforcement Learning in SDN
Fortinet, Inc.（飞塔公司）；New York University（纽约大学）；Beijing Institute of Technology（北京理工大学）
Junjie Zhang, Minghao Ye, Zehua Guo, Chen-Yu Yen, H. Jonathan Chao
备注：
This article has been accepted for inclusion in a future issue of IEEE Journal on Selected Areas in Communications Special Issue on Advances in Artificial Intelligence and Machine Learning for Networking
Traditional Traffic Engineering (TE) solutions can achieve the optimal or near-optimal performance by rerouting as many flows as possible. However, they do not usually consider the negative impact, such as packet out of order, when frequently rerouting flows in the network. To mitigate the impact of network disturbance, one promising TE solution is forwarding the majority of traffic flows using Equal-Cost Multi-Path (ECMP) and selectively rerouting a few critical flows using Software-Defined Networking (SDN) to balance link utilization of the network. However, critical flow rerouting is not trivial because the solution space for critical flow selection is enormous. Moreover, it is impossible to design a heuristic algorithm for this problem based on fixed and simple rules, since rule-based heuristics are unable to adapt to the changes of the traffic matrix and network dynamics. In this paper, we propose CFR-RL (Critical Flow Rerouting-Reinforcement Learning), a Reinforcement Learning-based scheme that learns a policy to select critical flows for each given traffic matrix automatically. CFR-RL then reroutes these selected critical flows to balance link utilization of the network by formulating and solving a simple Linear Programming (LP) problem. Extensive evaluations show that CFR-RL achieves near-optimal performance by rerouting only 10%-21.3% of total traffic.
Technische Universität Chemnitz（开姆尼茨工业大学）；Duy Tan University（维新大学）
Software-Defined Networking (SDN) introduces a centralized network control and management by separating the data plane from the control plane which facilitates traffic flow monitoring, security analysis and policy formulation. However, it is challenging to choose a proper degree of traffic flow handling granularity while proactively protecting forwarding devices from getting overloaded. In this paper, we propose a novel traffic flow matching control framework called Q-DATA that applies reinforcement learning in order to enhance the traffic flow monitoring performance in SDN based networks and …
Western Michigan University（西密歇根大学）；United Arab Emirates University（阿联酋大学）；Information Technology University（信息技术大学）
Modern information technology services largely depend on cloud infrastructures to provide their services. These cloud infrastructures are built on top of datacenter networks (DCNs) constructed with high-speed links, fast switching gear, and redundancy to offer better flexibility and resiliency. In this environment, network traffic includes long-lived (elephant) and short-lived (mice) flows with partitioned and aggregated traffic patterns. Although SDN-based approaches can efficiently allocate networking resources for such flows, the overhead due to network reconfiguration can be significant. W…
In this paper, we propose a destination-aware adaptive traffic flow rule aggregation (DATA) mechanism for facilitating traffic flow monitoring in SDN-based networks. This method adapts the number of flow table entries in SDN switches according to the level of detail of traffic flow information that other mechanisms (e.g. for traffic engineering, traffic monitoring, intrusion detection) require. It also prevents performance degradation of the SDN switches by keeping the number of flow table entries well below a critical level. This level is not preset as a hard threshold but learned during oper…
School of Information Science and Technology, Southwest Jiaotong University（西南交通大学信息科学与技术学院）；Chengdu University of Information Technology（成都信息工程大学）
Dynamic routing in software-defined networking (SDN) can be viewed as a centralized decision-making problem. Most of the existing deep reinforcement learning (DRL) agents can address it, thanks to the deep neural network (DNN)incorporated. However, fully-connected feed-forward neural network (FFNN) is usually adopted, where spatial correlation and temporal variation of traffic flows are ignored. This drawback usually leads to significantly high computational complexity due to large number of training parameters. To overcome this problem, we propose a novel model-free framework for dynamic rout…
Software Defined Network (SDN) is the next generation network that decouples the control plane from the data plane of forwarding devices by utilizing the OpenFlow protocol as a communication link between the data plane and the control plane. However, there are some security issues might be in actions on SDN that the attackers can take control over the SDN control plane. Thus, traffic measurement is a fundamental technique of protecting SDN against the high-security threats such as DDoS, heavy hitter, superspreader as well as live video calling, QoS control, high bandwidth requirement, resource…
Software Defined Networking (SDN) achieves programmability of a network through separation of the control and data planes. It enables flexibility in network management and control. Energy efficiency is one of the challenging global problems which has both economic and environmental impact. A massive amount of information is generated in the controller of an SDN based network. Machine learning gives the ability to computers to progressively learn from data without having to write specific instructions. In this work, we propose MER-SDN: a machine learning framework for traffic-aware energy effic…
Software Defined Networking (SDN) can effectively improve the performance of traffic engineering and has promising application foreground in backbone networks. Therefore, new energy saving schemes must take SDN into account, which is extremely important considering the rapidly increasing energy consumption from Telecom and ISP networks. At the same time, the introduction of SDN in a current network must be incremental in most cases, for both technical and economic reasons. During this period, operators have to manage hybrid networks, where SDN and traditional protocols coexist. In this paper, …
IP networks with a hybrid control plane deploy a distributed routing protocol like OSPF and the centralized paradigm of SDN in parallel. The advantages and disadvantages of a hybrid control plane have already been extensively discussed in the literature. This paper examines to what extent hybrid SDN/OSPF can solve the IP traffic matrix and related monitoring problems, inherent to the IP layer. The problem persists in hybrid networks, as the required SDN byte counters for a complete traffic matrix may not be sufficiently implemented (or even not at all), or the deployed SDN nodes may be too few…
Recent standardization efforts in IEEE 802.15.4-2015 Time Scheduled Channel Hopping (TSCH) and the IETF 6TiSCH Working Group (WG), aim to provide deterministic communications and efficient allocation of resources across constrained Internet of Things (IoT) networks, particularly in Industrial IoT (IIoT) scenarios. Within 6TiSCH, Software Defined Networking (SDN) has been identified as means of providing centralized control in a number of key situations. However, implementing a centralized SDN architecture in a Low Power and Lossy Network (LLN) faces considerable challenges: not only is control…
The Internet of Things (IoT) is advancing and the adoption of internet-connected devices in everyday use is constantly growing. This increase not only affects the traffic from other sources in the network, but also the communication quality requirements, like Quality of Service (QoS), for the IoT devices and applications. With the rise of dynamic network management and dynamic network programming technologies like Software-Defined Networking (SDN), traffic management and communication quality requirements can be tailored to fit niche use cases and characteristics. We propose a publish/subscrib…
The successful OpenFlow approach to Software Defined Networking (SDN) allows network programmability through a central controller able to orchestrate a set of dumb switches. However, the simple match/action abstraction of OpenFlow switches constrains the evolution of the forwarding rules to be fully managed by the controller. This can be particularly limiting for a number of applications that are affected by the delay of the slow control path, like traffic management applications. Some recent proposals are pushing toward an evolution of the OpenFlow abstraction to enable the evolution of forwa…
Software Defined Networking (SDN) is an emerging network control paradigm focused on logical centralization and programmability. At the same time, distributed routing protocols, most notably OSPF and IS-IS, are still prevalent in IP networks, as they provide shortest path routing, fast topological convergence after network failures, and, perhaps most importantly, the confidence based on decades of reliable operation. Therefore, a hybrid SDN/OSPF operation remains a desirable proposition. In this paper, we propose a new method of hybrid SDN/OSPF operation. Our method is different from other hyb…
We describe two applications of machine learning in the context of IP/Optical networks. The first one allows agile management of resources at a core IP/Optical network by using machine learning for short-term and long-term prediction of traffic flows and joint global optimization of IP and optical layers using colorless/directionless (CD) flexible ROADMs. Multilayer coordination allows for significant cost savings, flexible new services to meet dynamic capacity needs, and improved robustness by being able to proactively adapt to new traffic patterns and network conditions. The second applicati…
Software Defined Networking (SDN) is an emerging technology of efficiently controlling and managing computer networks, such as in data centres, Wide Area Networks (WANs), as well as in ubiquitous communication. In this paper, we explore the idea of embedding the SDN components, represented by SDN controller and virtual switch, in end-hosts to improve network performance. In particular, we consider load balancing across multiple network interfaces on end-hosts with different link capacity scenarios. We have explored and implemented different SDN-based load balancing approaches based on OpenFlow…
Traffic Engineering (TE) in IP carrier networks is one of the functions that can benefit from the Software Defined Networking paradigm. By logically centralizing the control of the network, it is possible to "program" per-flow routing based on TE goals. Traditional per-flow routing requires a direct interaction between the SDN controller and each node that is involved in the traffic paths. Depending on the granularity and on the temporal properties of the flows, this can lead to scalability issues for the amount of routing state that needs to be maintained in core network nodes and for the req…
With the passage of time and indulgence in Information Technology, network management has proved its significance and has become one of the most important and challenging task in today's era of information flow. Communication networks implement a high level of sophistication in managing and flowing the data through secure channels, to make it almost impossible for data loss. That is why there are many proposed solution that are currently implemented in wide range of network-based applications like social networks and finance applications. The objective of this research paper is to propose a ve…
We consider a distributed Software Defined Networking (SDN) architecture adopting a cluster of multiple controllers to improve network performance and reliability. Besides the Openflow control traffic exchanged between controllers and switches, we focus on the control traffic exchanged among the controllers in the cluster, needed to run coordination and consensus algorithms to keep the controllers synchronized. We estimate the effect of the inter-controller communications on the reaction time perceived by the switches depending on the data-ownership model adopted in the cluster. The model is a…
In this work, we propose online traffic engineering as a novel approach to detect and mitigate an emerging class of stealthy Denial of Service (DoS) link-flooding attacks. Our approach exploits the Software Defined Networking (SDN) paradigm, which renders the management of network traffic more flexible through centralised flow-level control and monitoring. We implement a full prototype of our solution on an emulated SDN environment using OpenFlow to interface with the network devices. We further discuss useful insights gained from our preliminary experiments as well as a number of open researc…
This paper presents NeuTM, a framework for network Traffic Matrix (TM) prediction based on Long Short-Term Memory Recurrent Neural Networks (LSTM RNNs). TM prediction is defined as the problem of estimating future network traffic matrix from the previous and achieved network traffic data. It is widely used in network planning, resource management and network security. Long Short-Term Memory (LSTM) is a specific recurrent neural network (RNN) architecture that is well-suited to learn from data and classify or predict time series with time lags of unknown size. LSTMs have been shown to model lon…
Hybrid IP networks that use both control paradigms - distributed and centralized - promise the best of two worlds: programmability and agility of SDN, and reliability and fault tolerance of distributed routing protocols like OSPF. The common approaches follow a division of labor concept, where SDN controls prioritized traffic and OSPF assures care-free operation of best effort traffic. We propose SDN Partitioning, which establishes centralized control over the distributed routing protocol by partitioning the topology into sub-domains with SDN-enabled border nodes, such that OSPF's routing upda…
Presence of a logically centralized controller in software-defined networks enables smart and fine-grained management of network traffic. Generally, traffic management includes measurement, analysis and control of traffic in order to improve resource utilization. This is done by inspecting corresponding performance requirements using metrics such as packet delay, jitter, loss rate and bandwidth utilization from global network view. There has been many works regarding traffic management of software-defined networks and how it could help to efficiently allocate resources. However, the vast major…
Network performance monitoring holds a pivotal role in improving the overall network performance. It is essential to monitor the traffic statistics at Internet eXchange Points (IXPs) to optimize the traffic flows. The existing monitoring system either lacks usability or programmability. We present a Software Defined Networking (SDN) enabled and OpenFlow (OF) compatible network performance monitoring system (SDX-Manager) to improve the usability or programmability of IXP simultaneously. The proposed system integrates traditional IXP-Manager with the SDN controller and Grafana to better visualiz…
The highly dynamic nature of the current network traffics, makes the network managers to exploit the flexibility of the state-of-the-art paradigm called SDN. In this way, there has been an increasing interest in hybrid networks of SDN-MPLS. In this paper, a new traffic engineering architecture for SDN-MPLS network is proposed. To this end, OpenFlow-enabled switches are applied over the edge of the network to improve flow-level management flexibility while MPLS routers are considered as the core of the network to make the scheme applicable for existing MPLS networks. The proposed scheme re-assi…
Distributed Denial of Service (DDoS) is one of the most prevalent attacks that an organizational network infrastructure comes across nowadays. We propose a deep learning based multi-vector DDoS detection system in a software-defined network (SDN) environment. SDN provides flexibility to program network devices for different objectives and eliminates the need for third-party vendor-specific hardware. We implement our system as a network application on top of an SDN controller. We use deep learning for feature reduction of a large set of features derived from network traffic headers. We evaluate…
Link capacity dimensioning is the periodic task where ISPs have to make provisions for sudden traffic bursts and network failures to assure uninterrupted operations. This provision comes in the form of link working capacities with noticeable amounts of headroom, i.e., spare capacities that are used in case of congestions or network failures. Distributed routing protocols like OSPF provide convergence after network failures and have proven their reliable operation over decades, but require overprovisioning and headroom of over 50%. However, SDN has recently been proposed to either replace or wo…
The introduction of SDN in IP backbones requires the coexistence of regular IP forwarding and SDN based forwarding. The former is typically applied to best effort Internet traffic, the latter can be used for different types of advanced services (VPNs, Virtual Leased Lines, Traffic Engineering...). In this paper we first introduce the architecture and the services of an "hybrid" IP/SDN networking scenario. Then we describe the design and implementation of an Open Source Hybrid IP/SDN (OSHI) node. It combines Quagga for OSPF routing and Open vSwitch for OpenFlow based switching on Linux. The ava…
Network slicing and resource allocation play pivotal roles in software-defined network (SDN)/network function virtualization (NFV)-assisted 5G networks. In 5G communications, the traffic rate is high, necessitating high data rates and low latency. Deep learning is a potential solution for overcoming these constraints. Secure slicing avoids resource wastage; however, DDoS attackers can exploit the sliced network. Therefore, we focused on secure slicing with resource allocation under massive network traffic. Traffic-aware scheduling is proposed for secure slicing and resource allocation over SDN…
5G networks are expected to be highly energy efficient, with a 10 times lower consumption than today's systems. An effective way to achieve such a goal is to act on the backhaul network by controlling the nodes operational state and the allocation of traffic flows. To this end, in this paper we formulate energy-efficient flow routing on the backhaul network as an optimization problem. In light of its complexity, which impairs the solution in large-scale scenarios, we then propose a heuristic approach. Our scheme, named EMMA, aims to both turn off idle nodes and concentrate traffic on the small…
The centralized architecture in software-defined network (SDN) provides a global view of the underlying network, paving the way for enormous research in the area of SDN traffic engineering (SDN TE). This research focuses on the load balancing aspects of SDN TE, given that the existing reactive methods for data-plane load balancing eventually result in packet loss and proactive schemes for data plane load balancing do not address congestion propagation. In the proposed work, the SDN controller periodically monitors flow level statistics and utilization on each link in the network and over-utili…
Detecting cyber attacks in the network environments used by Internet-of-things (IoT) and preventing them from causing physical perturbations play an important role in delivering dependable services. To achieve this goal, we propose in-network Honeypot based on Software-Defined Networking (SDN) to disrupt and mislead adversaries into exposures while they are in an early stage of preparing an attack. Different from traditional Honeypot requiring dedicated hardware setup, the in-network Honeypot directly reroutes traffic from suspicious nodes and intelligently spoofs the network traffic to them b…
Efficient algorithms and techniques to detect and identify large flows in a high throughput traffic stream in the SDN match-and-action model are presented. This is in contrast to previous work that either deviated from the match and action model by requiring additional switch level capabilities or did not exploit the SDN data plane. Our construction has two parts; (a) how to sample in an SDN match and action model, (b) how to detect large flows efficiently and in a scalable way, in the SDN model. Our large flow detection methods provide high accuracy and present a good and practical tradeoff b…
The current paper addresses relevant network security vulnerabilities introduced by network devices within the emerging paradigm of Internet of Things (IoT) as well as the urgent need to mitigate the negative effects of some types of Distributed Denial of Service (DDoS) attacks that try to explore those security weaknesses. We design and implement a Software-Defined Intrusion Detection System (IDS) that reactively impairs the attacks at its origin, ensuring the normal operation of the network infrastructure. Our proposal includes an IDS that automatically detects several DDoS attacks, and then…
Ethernet Virtual Private Network (EVPN) is an emerging technology that addresses the networking challenges presented by geo-distributed Data Centers (DCs). One of the major advantages of EVPN over legacy layer 2 VPN solutions is providing All-Active (A-A) mode of operation so that the traffic can truly be multi-homed on Provider Edge (PE) routers. However, A-A mode of operation introduces new challenges. In the case where the Customer Edge (CE) router is multi-homed to one or more PE routers, it is necessary that only one of the PE routers should forward Broadcast, Unknown unicast, and Multica…
Software-Defined-eXchanges (SDXes) promise to tackle the timely quest of bringing improving the inter-domain routing ecosystem through SDN deployment. Yet, the naive deployment of SDN on the Internet raises concerns about the correctness of the inter-domain data-plane. By allowing operators to deflect traffic from the default BGP route, SDN policies are susceptible of creating permanent forwarding loops invisible to the control-plane. In this paper, we propose a system, called Prelude, for detecting SDN-induced forwarding loops between SDXes with high accuracy without leaking the private routi…
Software-defined networking (SDN) is an architecture that aims to make networks fast and flexible. SDN's goal is to improve network control by enabling service providers as well as enterprises to respond quickly to changing business needs. In SDN, the administrator can shape traffic from a centralized control console without having to modify any of the individual switches belonging to the network. The SDN controller which is centralized directs the switches to deliver network services wherever they are needed, irrespective of the specific connections between a server and devices. This methodol…
Today 4G mobile systems are evolving to provide IP connectivity for diverse applications and services up to 1Gbps. They are designed to optimize the network performance, improve cost efficiency and facilitate the uptake of mass market IP-based services. Nevertheless, the growing demand and the diverse patterns of mobile traffic place an increasing strain on cellular networks. To cater to the large volumes of traffic delivered by the new services and applications, the future 5G network will provide the fundamental infrastructure for billions of new devices with less predictable traffic patterns…
In critical infrastructures, communication networks are used to exchange vital data among elements of Industrial Control Systems (ICSes). Due to the criticality of such systems and the increase of the cybersecurity risks in these contexts, best practices recommend the adoption of Intrusion Detection Systems (IDSes) as monitoring facilities. The choice of the positions of IDSes is crucial to monitor as many streams of data traffic as possible. This is especially true for the traffic patterns of ICS networks, mostly confined in many subnetworks, which are geographically distributed and largely a…
Software Defined Networking (SDN) offers a flexible and scalable architecture that abstracts decision making away from individual devices and provides a programmable network platform. However, implementing a centralized SDN architecture within the constraints of a low-power wireless network faces considerable challenges. Not only is controller traffic subject to jitter due to unreliable links and network contention, but the overhead generated by SDN can severely affect the performance of other traffic. This paper addresses the challenge of bringing high-overhead SDN architecture to IEEE 802.15…
Wireless mesh networks have been presented as a robust, scalable and low cost solution to provide connectivity in long distance areas. However, given its nature, routing strategies must support seamless mobility of nodes while enabling operation with a good performance and fast self-recovery from links fault. The routing approaches that meet these requirements are based on protocols usually used in ad-hoc networks (e.g. OLSR and B.A.T.M.A.N.) and, more recently, in the SDN paradigm (e.g. OpenWiMesh), each one of them having its own pros and cons. Therefore, it is important to evaluate the bene…
The convergence of IT and OT technologies results in the need for efficient network management solutions for automotive and industrial automation environments. However, configuring real-time Ethernet networks while maintaining the desired QoS is challenging due to the dynamic nature of OT networks and the high configuration parameters. This paper introduces an SDN-based self-configuration framework for the fully automated configuration of TSN networks. Unlike standard TSN configuration, we remove end-host-related dependencies and put flows initially on default paths to extract traffic characte…
Existing distributed denial-of-service attack detection in software defined networks (SDNs) typically perform detection in a single domain. In reality, abnormal traffic usually affects multiple network domains. Thus, a cross-domain attack detection has been proposed to improve detection performance. However, when participating in detection, the domain of each SDN needs to provide a large amount of real traffic data, from which private information may be leaked. Existing multiparty privacy protection schemes often achieve privacy guarantees by sacrificing accuracy or increasing the time cost. A…
Packet-level traffic measurement is essential in applications like QoS, traffic engineering, or anomaly detection. Software-Defined Networking (SDN) enables efficient and dynamic network configuration that we can deploy for fine-grained network state measurement. As the state changes dynamically, the sampling frequency must cope with that for accurate measurement. At the same time, we must consider the measurement cost, e.g., nodes' resource utilization. Existing works fall short in offering an optimal model to balance both the measurement accuracy and cost. We fill that gap by proposing ACE, …
An approach to using the concept of Software-Defined Networking and Network Functions Virtualization (SDN/NFV) for the implementation of an information security monitoring and management system in 5G and 6G networks is proposed. SDN switches based on the OpenFlow protocol are offered as network sensors. In order to reduce the time for finding a subset of the right rules in the vast array of all rules on traffic filtering systems that are logically located on sensors, a method of processing and filtering traffic in 5G and 6G transport networks is proposed. This method is based on DPDK with the …
Destination IP prefix-based routing protocols are core to Internet routing today. Internet autonomous systems (AS) possess fixed IP prefixes, while packets carry the intended destination AS's prefix in their headers, in clear text. As a result, network communications can be easily identified using IP addresses and become targets of a wide variety of attacks, such as DNS/IP filtering, distributed Denial-of-Service (DDoS) attacks, man-in-the-middle (MITM) attacks, etc. In this work, we explore an alternative network architecture that fundamentally removes such vulnerabilities by disassociating t…
Although Software-Defined Networking (SDN) enables flexible network resource allocations for traffic engineering, current literature mostly focuses on unicast communications. Compared to traffic engineering for multiple unicast flows, multicast traffic engineering for multiple trees is very challenging not only because minimizing the bandwidth consumption of a single multicast tree by solving the Steiner tree problem is already NP-Hard, but the Steiner tree problem does not consider the link capacity constraint for multicast flows and node capacity constraint to store the forwarding entries in…
CNN+Transformer Based Anomaly Traffic Detection in UAV Networks for Emergency Rescue
The Key Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum Space, Ministry of Industry and Information Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学电磁频谱空间动态认知系统重点实验室（工业和信息化部））；Department of Electronic and Electrical Engineering, University College London（伦敦大学学院电子与电气工程系）
Future cellular networks will be dense and require key traffic management technologies for fine-grained network control. The problem gets more complicated in the presence of different network segments with bottleneck links limiting the desired quality of service (QoS) delivery to the last mile user. In this work, we first design a framework for software-defined cellular networks (SDCN) and then propose new mechanisms for management of QoS and non-QoS users traffic considering both access and backhaul networks, jointly. The overall SDN-LTE system and related approaches are developed and tested …
Intelligent Detection and Mitigation of Carpet-Bombing DDoS Attacks in SDN Using Retrieval-Augmented Generation and Large Language Models
College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；ICNLab, Shenzhen Graduate School, Peking University（北京大学深圳研究生院ICN实验室）
AGH University of Science and Technology（AGH科技大学）；Barcelona Neural Networking Center（巴塞罗那神经网络中心）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
Network modeling is a key enabler to achieve efficient network operation in future self-driving Software-Defined Networks. However, we still lack functional network models able to produce accurate predictions of Key Performance Indicators (KPI) such as delay, jitter or loss at limited cost. In this paper we propose RouteNet, a novel network model based on Graph Neural Network (GNN) that is able to understand the complex relationship between topology, routing, and input traffic to produce accurate estimates of the per-source/destination per-packet delay distribution and loss. RouteNet leverages…
Control planes for global carrier networks should be programmable (so that new functionality can be easily introduced) and scalable (so they can handle the numerical scale and geographic scope of these networks). Neither traditional control planes nor new SDN-based control planes meet both of these goals. In this paper, we propose a framework for recursive routing computations that combines the best of SDN (programmability) and traditional networks (scalability through hierarchy) to achieve these two desired properties. Through simulation on graphs of up to 10,000 nodes, we evaluate our design…
