One of the most critical tasks for network administrator is to ensure system uptime and availability. For the network security, anomaly detection systems, along with firewalls and intrusion prevention systems are the must-have tools. So far in the field of network anomaly detection, people are working on two different approaches. One is flow-based; usually rely on network elements to make so-called flow information available for analysis. The second approach is packet-based; which directly analyzes the data packet information for the detection of anomalies. This paper describes the main differences between the two approaches through an in-depth analysis. We try to answer the question of when and why an approach is better than the other. The answer is critical for network administrators to make their choices in deploying a defending system, securing the network and ensuring business continuity.
In this paper, we examine a basic smart meter network topology in mininet and address the issue of congestion over a commodity network, proposing an adaptive algorithm to cope with varying grid data delivery latencies.
With the widespread adoption of cloud services, especially the extensive deployment of plenty of Web applications, it is important and challenging to detect anomalies from the packet payload. For example, the anomalies in the packet payload can be expressed as a number of specific strings which may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since they are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the paylo…
This paper proposes AFMT, a packet scheduling algorithm to achieve adaptive flow-aware multipath tunnelling. AFMT has two unique properties. Firstly, it implements robust adaptive traffic splitting for the subtunnels. Secondly, it detects and schedules bursts of packets cohesively, a scheme that already enabled traffic splitting for load balancing with little to no packet reordering. Several NS-3 experiments over different network topologies show that AFMT successfully deals with changing path characteristics due to background traffic while increasing throughput and reliability.
Network administrators want to detect TCP-level packet reordering to diagnose performance problems and attacks. However, reordering is expensive to measure, because each packet must be processed relative to the TCP sequence number of its predecessor in the same flow. Due to the volume of traffic, detection should take place in the data plane as the packets fly by. However, restrictions on the memory size and the number of memory accesses per packet make it impossible to design an efficient algorithm for pinpointing flows with heavy packet reordering. In practice, packet reordering is typically…
Datacenter applications demand both low latency and high throughput; while interactive applications (e.g., Web Search) demand low tail latency for their short messages due to their partition-aggregate software architecture, many data-intensive applications (e.g., Map-Reduce) require high throughput for long flows as they move vast amounts of data across the network. Recent proposals improve latency of short flows and throughput of long flows by addressing the shortcomings of existing packet scheduling and congestion control algorithms, respectively. We make the key observation that long tails …
Flow scheduling is crucial in data centers, as it directly influences user experience of applications. According to different assumptions and design goals, there are four typical flow scheduling problems/solutions: SRPT, LAS, Fair Queueing, and Deadline-Aware scheduling. When implementing these solutions in commodity switches with limited number of queues, they need to set static parameters by measuring traffic in advance, while optimal parameters vary across time and space. This paper proposes a generic framework, namely QCluster, to adapt all scheduling problems for limited number of queues.…
We consider a token bucket mechanism serving a heterogeneous flow with a focus on backlog, delay and packet loss properties. Previous models have considered the case for fixed size packets, i.e. "one token per packet" with and M/D/1 view on queuing behavior. We partition the heterogeneous flow into several packet size classes with individual Poisson arrival intensities. The accompanying queuing model is a "full state" model, i.e. buffer content is not reduced to a single quantity but encompasses the detailed content in terms of packet size classes. This yields a high model cardinality for whic…
In today's data center, a diverse mix of throughput-sensitive long flows and delay-sensitive short flows are commonly presented in shallow-buffered switches. Long flows could potentially block the transmission of delay-sensitive short flows, leading to degraded performance. Congestion can also be caused by the synchronization of multiple TCP connections for short flows, as typically seen in the partition/aggregate traffic pattern. While multiple end-to-end transport-layer solutions have been proposed, none of them have tackled the real challenge: reliable transmission in the network. In this p…
For cloud service providers, fine-grained packet loss detection across data centers is crucial in improving their service level and increasing business income. However, the inability to obtain sufficient measurements makes it difficult owing to the fundamental limit that the wide-area network links responsible for communication are not under their management. Moreover, millisecond-level delay jitter and clock synchronization errors in the WAN disable many tools that perform well in data center networks on this issue. Therefore, there is an urgent need to develop a new tool or method. In this w…
Time-to-Live data in the IP header offers two interesting characteristics: First, different IP stacks pick different start TTL values. Second, each traversed router should decrement the TTL value. The combination of both offers host and route fingerprinting options. We present the first work to investigate Internet-wide TTL behavior at carrier scale and evaluate its fit to detect anomalies, predominantly spoofed source IP addresses. Using purpose-built software, we capture 2 weeks of raw TTL data at a 40 Gbit/s Internet uplink. For further insight, we actively measure observed hosts and conduc…
Distributed computing networks, tasked with both packet transmission and processing, require the joint optimization of communication and computation resources. We develop a dynamic control policy that determines both routes and processing locations for packets upon their arrival at a distributed computing network. The proposed policy, referred to as Universal Computing Network Control (UCNC), guarantees that packets i) are processed by a specified chain of service functions, ii) follow cycle-free routes between consecutive functions, and iii) are delivered to their corresponding set of destina…
Various middleboxes are ubiquitously deployed in networks to perform packet processing functions, such as firewalling, proxy, scheduling, etc., for the flows passing through them. With the explosion of network traffic and the demand for multiple types of network resources, it has never been more challenging for a middlebox to provide Quality-of-Service (QoS) guarantees to grouped flows. Unfortunately, all currently existing fair queueing algorithms fail in supporting hierarchical scheduling, which is necessary to provide QoS guarantee to the grouped flows of multiple service classes. In this p…
Segment Routing over IPv6 (SRv6 in short) is a networking solution for IP backbones and datacenters, which has been recently adopted in several of large scale network deployments. The SRv6 research, standardization and implementation activities are going on at a remarkable pace. In particular, a number of Internet Drafts have been submitted related to the Performance Monitoring (PM) of flows in an SRv6 network. In this paper we discuss the proposed PM approaches, considering both data plane and control plane aspects and focusing on loss monitoring. Then we describe the implementation of a per-…
In multi-radio wireless mesh networks, one node is eligible to transmit packets over multiple channels to different destination nodes simultaneously. This feature of multi-radio wireless mesh network makes high throughput for the network and increase the chance for multi path routing. This is because the multiple channel availability for transmission decreases the probability of the most elegant problem called as interference problem which is either of interflow and intraflow type. For avoiding the problem like interference and maintaining the constant network performance or increasing the per…
This paper proposes an application-aware multipath packet forwarding framework that integrates Machine Learning Techniques (MLT) and Software Defined Networks (SDN). As the Internet provides a variety of services and their performance requirement has become heterogeneous, it is common to come across the scenario of multiple flows competing for a constrained resource such as bandwidth, less jitter or low latency path. Such factors are application specific requirement that is beyond the knowledge of a simple combination of protocol type and port number. Better overall performance could be achiev…
Network measurement is necessary to obtain an understanding of the network traffic and keep the network healthy. Flow-level measurement is widely used because it provides rich enough information while being resource efficient, in contrast to the traffic-level solutions like SNMP and packet-level solutions like tcpdump. Flow-level measurement requires to maintain a TFR for each active flow, and update the corresponding TFR when a packet arrives. In the scenario of high-speed networks, the processing time for each packet is only several or tens of nanoseconds, and there may be up to millions of …
Federal University of Rio de Janeiro（里约热内卢联邦大学）；University of Massachusetts at Amherst（马萨诸塞大学阿默斯特分校）
The problem of detecting anomalies in time series from network measurements has been widely studied and is a topic of fundamental importance. Many anomaly detection methods are based on packet inspection collected at the network core routers, with consequent disadvantages in terms of computational cost and privacy. We propose an alternative method in which packet header inspection is not needed. The method is based on the extraction of a normal subspace obtained by the tensor decomposition technique considering the correlation between different metrics. We propose a new approach for online ten…
We define the minimal interleaved regulator, which generalizes the Urgency Based Shaper that was recently proposed by Specht and Samii as a simpler alternative to per-flow reshaping in deterministic networks with aggregate scheduling. With this regulator, packets of multiple flows are processed in one FIFO queue; the packet at the head of the queue is examined against the regulation constraints of its flow; it is released at the earliest time at which this is possible without violating the constraints. Packets that are not at the head of the queue are not examined until they reach the head of …
Network traffic classification using machine learning techniques has been widely studied. Most existing schemes classify entire traffic flows, but there are major limitations to their practicality. At a network router, the packets need to be processed with minimum delay, so the classifier cannot wait until the end of the flow to make a decision. Furthermore, a complicated machine learning algorithm can be too computationally expensive to implement inside the router. In this paper, we introduce flow-packet hybrid traffic classification (FPHTC), where the router makes a decision per packet based…
Software Defined Networking (SDN) promises greater flexibility for directing packet flows, and Network Function Virtualization promises to enable dynamic management of software-based network functions. However, the current divide between an intelligent control plane and an overly simple, stateless data plane results in the inability to exploit the flexibility of a software based network. In this paper we propose SDNFV, a framework that expands the capabilities of network processing-and-forwarding elements to flexibly manage packet flows, while retaining both a high performance data plane and a…
Most load balancing techniques implemented in current data centers tend to rely on a mapping from packets to server IP addresses through a hash value calculated from the flow five-tuple. The hash calculation allows extremely fast packet forwarding and provides flow `stickiness', meaning that all packets belonging to the same flow get dispatched to the same server. Unfortunately, such static hashing may not yield an optimal degree of load balancing, e.g., due to variations in server processing speeds or traffic patterns. On the other hand, dynamic schemes, such as the Join-the-Shortest-Queue (J…
This work addresses the problem of evaluating optimal link capacities of a packet-flow network for the objective of congestion minimization. We present a simple model of packet flow in networks and present a numerical approach to evaluate packet flow probability mass function at any arbitrary edge of the network for a given routing algorithm and traffic rate. We further discuss techniques of assigning optimal capacity at each edge for attaining desired minimized congestion and discuss related trade-offs. Our framework is built around the assumption of Poisson traffic, however the numerical app…
In this paper, we propose a novel approach, called SENATUS, for joint traffic anomaly detection and root-cause analysis. Inspired from the concept of a senate, the key idea of the proposed approach is divided into three stages: election, voting and decision. At the election stage, a small number of \nop{traffic flow sets (termed as senator flows)}senator flows are chosen\nop{, which are used} to represent approximately the total (usually huge) set of traffic flows. In the voting stage, anomaly detection is applied on the senator flows and the detected anomalies are correlated to identify the m…
This paper investigates traffic flow modeling issue in multi-services oriented unmanned aerial vehicle (UAV)-enabled wireless networks, which is critical for supporting future various applications of such networks. We propose a general traffic flow model for multi-services oriented UAV-enable wireless networks. Under this model, we first classify the network services into three subsets: telemetry, Internet of Things (IoT), and streaming data. Based on the Pareto distribution, we then partition all UAVs into three subgroups with different network usage. We further determine the number of packet…
In current Data Center Networks (DCNs), Equal- Cost MultiPath (ECMP) is used as the de-facto routing protocol. However, ECMP does not differentiate between short and long flows, the two main categories of flows depending on their duration (lifetime). This issue causes hot-spots in the network, affecting negatively the Flow Completion Time (FCT) and the throughput, the two key performance metrics in data center networks. Previous work on load balancing proposed solutions such as splitting long flows into short flows, using per-packet forwarding approaches, and isolating the paths of short and l…
HyGra: Accelerating Network-State Simulation for LLM Training in DCNs via Adaptive Packet-Flow Granularity
College of Computer, Nanjing University of Posts and Telecommunications（南京邮电大学计算机学院）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）
When IP-packet processing is unconditionally carried out on behalf of an operating system kernel thread, processing systems can experience overload in high incoming traffic scenarios. This is especially worrying for embedded real-time devices controlling their physical environment in industrial IoT scenarios and automotive systems. We propose an embedded real-time aware IP stack adaption with an early demultiplexing scheme for incoming packets and subsequent per-flow aperiodic scheduling. By instrumenting existing embedded IP stacks, rigid prioritization with minimal latency is deployed withou…
In this work, a new region-based, multipath-enabled packet routing is presented and called SmartPacket Routing. The proposed approach provides several opportunities to re-distribute the smartness and decision making among various elements of a network including the packets themselves toward providing a decentralized solution for SDNs. This would bring efficiency and scalability, and therefore also lower environmental footprint for the ever-growing networks. In particular, a region-based representation of the network topology is proposed which is then used to describe the routing actions along …
Resilience functionality, including failure resilience and flow migration, is of pivotal importance in practical network function virtualization (NFV) systems. However, existing failure recovery procedures incur high packet processing delay due to heavyweight process-checkpointing, while flow migration has poor performance due to centralized control. This paper proposes NFVactor, a novel NFV system that aims to provide lightweight failure resilience and high-performance flow migration. NFVactor enables these by using actor model to provide a per-flow execution environment, so that each flow ca…
As the number of heterogenous IP-connected devices and traffic volume increase, so does the potential for security breaches. The undetected exploitation of these breaches can bring severe cybersecurity and privacy risks. Anomaly-based \acp{IDS} play an essential role in network security. In this paper, we present a practical unsupervised anomaly-based deep learning detection system called ARCADE (Adversarially Regularized Convolutional Autoencoder for unsupervised network anomaly DEtection). With a convolutional \ac{AE}, ARCADE automatically builds a profile of the normal traffic using a subse…
Internet analysis is a major challenge due to the volume and rate of network traffic. In lieu of analyzing traffic as raw packets, network analysts often rely on compressed network flows (netflows) that contain the start time, stop time, source, destination, and number of packets in each direction. However, many traffic analyses benefit from temporal aggregation of multiple simultaneous netflows, which can be computationally challenging. To alleviate this concern, a novel netflow compression and resampling method has been developed leveraging GraphBLAS hyperspace traffic matrices that preserve…
High Speed Downlink Packet Access (HSDPA) was introduced to UMTS radio access segment to provide higher capacity for new packet switched services. As a result, packet switched sessions with multiple diverse traffic flows such as concurrentvoice and data, or video and data being transmitted to the same user are a likely commonplace cellular packet data scenario. In HSDPA, Radio Access Network (RAN) buffer management schemes are essential to support the end-to-end QoS of such sessions. Hence in this paper we present the end-to-end performance study of a proposed RAN buffer management scheme for …
