Queues allow network operators to control traffic: where queues build, they can enforce scheduling and shaping policies. In the Internet today, however, there is a mismatch between where queues build and where control is most effectively enforced; queues build at bottleneck links that are often not under the control of the data sender. To resolve this mismatch, we propose a new kind of middlebox, called Bundler. Bundler uses a novel inner control loop between a sendbox (in the sender's site) and a receivebox (in the receiver's site) to determine the aggregate rate for the bundle, leaving the end-to-end connections and their control loops intact. Enforcing this sending rate ensures that bottleneck queues that would have built up from the bundle's packets now shift from the bottleneck to the sendbox. The sendbox then exercises control over its traffic by scheduling packets to achieve higher-level objectives. We have implemented Bundler in Linux and evaluated it with real-world and emulation experiments. We find that Bundler allows the sender-chosen policy to be effective: when configured to implement Stochastic Fairness Queueing (SFQ), it improves median flow completion time (FCT) by between 28% and 97% across various scenarios.
Internet traffic displays many persistent periodicities (oscillations) on a large range of time scales. This paper describes the measurement methodology to detect Internet traffic periodicities and also describes the main periodicities in Internet traffic.
Introducing Internet traffic anomaly detection mechanism based on large deviations results for empirical measures. Using past traffic traces we characterize network traffic during various time-of-day intervals, assuming that it is anomaly-free. Throughout, we compare the two approaches presenting their advantages and disadvantages to identify and classify temporal network anomalies. We also demonstrate how our framework can be used to monitor traffic from multiple network elements in order to identify both spatial and temporal anomalies. We validate our techniques by analyzing real traffic tra…
Modeling Internet growth is important both for understanding the current network and to predict and improve its future. To date, Internet models have typically attempted to explain a subset of the following characteristics: network structure, traffic flow, geography, and economy. In this paper we present a discrete, agent-based model, that integrates all of them. We show that the model generates networks with topologies, dynamics, and (more speculatively) spatial distributions that are similar to the Internet.
Mobile network traffic prediction is an important input in to network capacity planning and optimization. Existing approaches may lack the speed and computational complexity to account for bursting, non-linear patterns or other important correlations in time series mobile network data. We compare the performance of two deep learning architectures - Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) - for predicting mobile Internet traffic using two months of Telecom Italia data for the metropolitan area of Milan. K-Means clustering was used a priori to group cells based on Internet a…
We present a simple model reproducing the long-range autocorrelations and the power spectrum of the web traffic. The model assumes the traffic as Poisson flow of files with size distributed according to the power-law. In this model the long-range autocorrelations are independent of the network properties as well as of inter-packet time distribution.
Internet of Things (IoT) defines a network of devices connected to the internet and sharing a massive amount of data between each other and a central location. These IoT devices are connected to a network therefore prone to attacks. Various management tasks and network operations such as security, intrusion detection, Quality-of-Service provisioning, performance monitoring, resource provisioning, and traffic engineering require traffic classification. Due to the ineffectiveness of traditional classification schemes, such as port-based and payload-based methods, researchers proposed machine lea…
Transport protocols use port numbers to allow connection multiplexing on Internet hosts. TCP as well as UDP, the two most widely used transport protocols, have limitations on what constitutes a valid and invalid port number. One example of an invalid port number for these protocols is port 0. In this work, we present preliminary results from analyzing port 0 traffic at a large European IXP. In one week of traffic we find 74GB port 0 traffic. The vast majority of this traffic has both source and destination ports set to 0, suggesting scanning or reconnaissance as its root cause. Our analysis al…
Trust management of Internet of connected vehicles has been a hot topic during the recent years with the rapid development of UGV technologies. However, existing resolutions based on trustworthiness verification among vehicles make the traffic event transmission quite inefficient. In this paper, we assume that the deployed RSUs can provide efficient communication between any pair of RSU and vehicle, and propose Vcash, a reputation framework for identifying denial of traffic service, to resolve the trustworthiness problem in the application level of the Internet of connected vehicles. In our re…
Link dimensioning is used by ISPs to properly provision the capacity of their network links. Operators have to make provisions for sudden traffic bursts and network failures to assure uninterrupted operations. In practice, traffic averages are used to roughly estimate required capacity. More accurate solutions often require traffic statistics easily obtained from packet captures, e.g. variance. Our investigations on real Internet traffic have emphasized that the traffic shows high variations at small aggregation times, which indicates that the traffic is self-similar and has a heavy-tailed cha…
Denial of Service (DoS) attacks are one of the most challenging threats to Internet security. An attacker typically compromises a large number of vulnerable hosts and uses them to flood the victim's site with malicious traffic, clogging its tail circuit and interfering with normal traffic. At present, the network operator of a site under attack has no other resolution but to respond manually by inserting filters in the appropriate edge routers to drop attack traffic. However, as DoS attacks become increasingly sophisticated, manual filter propagation becomes unacceptably slow or even infeasibl…
The Internet traffic matrix plays a significant roll in network operation and management, therefore, the structural analysis of traffic matrix, which decomposes different traffic components of this high-dimensional traffic dataset, is quite valuable to some network applications. In this study, based on the Robust Principal Component Analysis (RPCA) theory, a novel traffic matrix structural analysis approach named Multi-Resolution RPCA is created, which utilizes the wavelet multi-resolution analysis. Firstly, we build the Multi-Resolution Traffic Matrix Decomposition Model (MR-TMDM), which char…
A new method of estimating some statistical characteristics of TCP flows in the Internet is developed in this paper. For this purpose, a new set of random variables (referred to as observables) is defined. When dealing with sampled traffic, these observables can easily be computed from sampled data. By adopting a convenient mouse/elephant dichotomy also dependent on traffic, it is shown how these variables give a reliable statistical representation of the number of packets transmitted by large flows during successive time intervals with an appropriate duration. A mathematical framework is deve…
We propose flow-based analysis to estimate quality of an Internet connection. Using results from the queuing theory we compare two expressions for backbone traffic that have different scopes of applicability. A curve that shows dependence of utilization of a link on a number of active flows in it describes different states of the network. We propose a methodology for plotting such a curve using data received from a Cisco router by NetFlow protocol, determining the working area and the overloading point of the network. Our test is an easy way to find a moment for upgrading the backbone.
The rapid deployment of new Internet protocols over the last few years and the COVID-19 pandemic more recently (2020) has resulted in a change in the Internet traffic composition. Consequently, an updated microscopic view of traffic shares is needed to understand how the Internet is evolving to capture both such shorter- and longer-term events. Toward this end, we observe traffic composition at a research network in Japan and a Tier-1 ISP in the USA. We analyze the traffic traces passively captured at two inter-domain links: MAWI (Japan) and CAIDA (New York-Sao Paulo), which cover 100GB of dat…
We advocate to create a \emph{spot} Internet transit market, where transit is sold using the under-utilized backbone capacity at a lower price. The providers can improve profit by capitalizing the perishable capacity, and customers can buy transit on-demand without a minimum commitment level for elastic traffic, and as a result improve its surplus (i.e. utility gains). We conduct a systematic study of the economical benefits of spot transit both theoretically and empirically. We propose a simple analytical framework with a general demand function, and solve the pricing problem of maximizing th…
The emergence of video applications and video capable devices have contributed substantially to the increase of video traffic on Internet. New mechanisms recommending video rate adaptation towards delivering enhanced Quality of Experience (QoE) at the same time making room for more sessions. This paper introduces a cross-layer QoE-aware architecture for video traffic over the Internet. It proposes that video sources at the application layer adapt their rate to the network environment by controlling their transmitted bit rate dynamically; and the edge of network at the network layer protects th…
In the mid-90's, it was shown that the statistics of aggregated time series from Internet traffic departed from those of traditional short range dependent models, and were instead characterized by asymptotic self-similarity. Following this seminal contribution, over the years, many studies have investigated the existence and form of scaling in Internet traffic. This contribution aims first at presenting a methodology, combining multiscale analysis (wavelet and wavelet leaders) and random projections (or sketches), permitting a precise, efficient and robust characterization of scaling which is …
An extensive body of research deals with estimating the correlation and the Hurst parameter of Internet traffic traces. The significance of these statistics is due to their fundamental impact on network performance. The coverage of Internet traffic traces is, however, limited since acquiring such traces is challenging with respect to, e.g., confidentiality, logging speed, and storage capacity. In this work, we investigate how the correlation of Internet traffic can be reliably estimated from random traffic samples. These samples are observed either by passive monitoring within the network, or …
The efficiency of flow-based networking mechanisms strongly depends on traffic characteristics and should thus be assessed using accurate flow models. For example, in the case of algorithms based on the distinction between elephant and mice flows, it is extremely important to ensure realistic flows' length and size distributions. Credible models or data are not available in literature. Numerous works contain only plots roughly presenting empirical distribution of selected flow parameters, without providing distribution mixture models or any reusable numerical data. This paper aims to fill that…
With the widespread adoption of Internet of Things (IoT), billions of everyday objects are being connected to the Internet. Effective management of these devices to support reliable, secure and high quality applications becomes challenging due to the scale. As one of the key cornerstones of IoT device management, automatic cross-device classification aims to identify the semantic type of a device by analyzing its network traffic. It has the potential to underpin a broad range of novel features such as enhanced security (by imposing the appropriate rules for constraining the communications of c…
This paper introduces Nimbus, a robust technique to detect whether the cross traffic competing with a flow is "elastic", and shows that this elasticity detector improves congestion control. If cross traffic is inelastic, then a sender can control queueing delays while achieving high throughput, but in the presence of elastic traffic, it may lose throughput if it attempts to control packet delay. To estimate elasticity, Nimbus modulates the flow's sending rate with sinusoidal pulses that create small traffic fluctuations at the bottleneck link, and measures the frequency response of the rate of…
The raising level of traffic imposes a great demand in the growth of intelligent traffic systems. With increase in complexity of alleviation, finding solutions to traffic congestion problem have become one of the challenges. Various optimization techniques have been proposed in literature to meet these challenges. This paper surveys different optimization techniques based on heuristics for automated traffic congestion control. Moreover, an approach based on River Formation Dynamics scheme is introduced to analyze the optimization problem for traffic congestion control and a scheme to extract r…
Getting good statistical models of traffic on network links is a well-known, often-studied problem. A lot of attention has been given to correlation patterns and flow duration. The distribution of the amount of traffic per unit time is an equally important but less studied problem. We study a large number of traffic traces from many different networks including academic, commercial and residential networks using state-of-the-art statistical techniques. We show that the log-normal distribution is a better fit than the Gaussian distribution commonly claimed in the literature. We also investigate…
Industrial control systems (ICS) are managed remotely with the help of dedicated protocols that were originally designed to work in walled gardens. Many of these protocols have been adapted to Internet transport and support wide-area communication. ICS now exchange insecure traffic on an inter-domain level, putting at risk not only common critical infrastructure but also the Internet ecosystem (e.g., DRDoS~attacks). In this paper, we uncover unprotected inter-domain ICS traffic at two central Internet vantage points, an IXP and an ISP. This traffic analysis is correlated with data from honeypo…
The different level of interest in deploying the new Internet address space across network operators has kept IPv6 tardy in its deployment. However, since the last block of IPv4 addresses has been assigned, Internet communities took the concern of the address space scarcity seriously and started to move forward actively. After the successful IPv6 test on 8 June, 2011 (World IPv6 Day [1]), network operators and service/content providers were brought together for preparing the next step of the IPv6 global deployment (World IPv6 Launch on 6 June, 2012 [2]). The main purpose of the event was to pe…
The performance of networks that use the Internet Protocol is sensitive to precise configuration of many low-level parameters on each network device. These settings govern the action of dynamic routing protocols, which direct the flow of traffic; in order to ensure that these dynamic protocols all converge to produce some 'optimal' flow, each parameter must be set correctly. Multiple conflicting optimization objectives, nondeterminism, and the need to reason about different failure scenarios make the task particularly complicated. We present a fast and flexible approach for the analysis of a n…
In recent years, there are some major changes in the way content is being distributed over the network. The content distribution techniques have recently started to embrace peer-to-peer (P2P) systems as an alternative to the traditional client-server architecture. P2P systemsthat are based on the BitTorrent protocol uses end-users' resources to provide a cost effective distribution of bandwidth intensive content to thousands of users. The BitTorrent protocol system offers a scalable mechanism for distributing a large volume of data to a set of peers over the Internet. With the growing demand f…
Domain name system communication may provide sensitive information on users' Internet activity. DNS-over-TLS and DNS-over-HTTPS are proposals aiming at increasing the privacy of Internet end users. In this paper we present an overview of the current state in the deployment of DNS-over-HTTPS (DoH) implementations complemented by measurements of DoH traffic in terms of the incurred overhead and the possibility of the DoH detection based on the inherent characteristics of the communication patterns.
The first decade of the 21st century has seen tremendous improvements in mobile internet and its technologies. The high traffic volume of services such as video conference and other real-time traffic applications are imposing a great challenge on networks. In the meantime, demand for the use of mobile devices in computation and communication such as smart phones, personal digital assistants, and mobile-enabled laptops has grown rapidly. These services have driven the demand for increasing and guaranteing bandwidth requirements in the network. A direction of this paper is in the case of resourc…
The Internet is the most complex system ever created in human history. Therefore, its dynamics and traffic unsurprisingly take on a rich variety of complex dynamics, self-organization, and other phenomena that have been researched for years. This paper is a review of the complex dynamics of Internet traffic. Departing from normal treatises, we will take a view from both the network engineering and physics perspectives showing the strengths and weaknesses as well as insights of both. In addition, many less covered phenomena such as traffic oscillations, large-scale effects of worm traffic, and …
Internet censors seek ways to identify and block internet access to information they deem objectionable. Increasingly, censors deploy advanced networking tools such as deep-packet inspection (DPI) to identify such connections. In response, activists and academic researchers have developed and deployed network traffic obfuscation mechanisms. These apply specialized cryptographic tools to attempt to hide from DPI the true nature and content of connections. In this survey, we give an overview of network traffic obfuscation and its role in circumventing Internet censorship. We provide historical a…
The lack of security of the Internet routing protocol (BGP) has allowed attackers to divert Internet traffic and consequently perpetrate service disruptions, monetary frauds, and even citizen surveillance for decades. State-of-the-art defenses rely on geo-distributed BGP monitors to detect rogue BGP announcements. As we show, though, attackers can easily evade detection by engineering their announcements. This paper presents Oscilloscope, an approach to accurately detect BGP hijacks by relying on real-time traffic analysis. As hijacks inevitably change the characteristics of the diverted traff…
Since the end of 2019, the SARS-CoV-2 virus known as COVID-19 has spread rapidly around the world, forcing many governments to impose restrictive blocking or lockdown to combat the pandemic. With locomotion restriction of people in almost of countries of the world, workers and students needed to keep their activities at home. As a result, people's behavior, habits, and the way they started using the Internet changed significantly. Like professionals of offices, the younger played an important role in this behavior, especially in the type of resources used by them. As result, the characterizati…
Nowadays, Machine-to-Machine (M2M) and Internet of Things (IoT) traffic rate is increasing at a fast pace. The use of satellites is expected to play a large role in delivering such a traffic. In this work, we investigate the use of two of the most common M2M/IoT protocols stacks on a satellite Random Access (RA) channel, based on DVB-RCS2 standard. The metric under consideration is the completion time, in order to identify the protocol stack that can provide the best performance level.
With the increasing usage of mobile devices to ubiquitously access heterogeneous applications in wireless Internet, the measurement and analysis of Internet traffic has become a key research area. In this paper, we present the results of our measurements for VBR traffic over UDP in 802.11g and GPRS networks. We focus on Inter-Packet Arrival Time (IPRT) and Inter-Packet Transmission Delay (IPTD) and observe that the later has a significant impact on the round trip delay. Numerical parameters for Weibull, Exponential and Normal distribution in order to represent such traffic are also presented.
Internet traffic on a network link can be modeled as a stochastic process. After detecting and quantifying the properties of this process, using statistical tools, a series of mathematical models is developed, culminating in one that is able to generate ``traffic'' that exhibits --as a key feature-- the same difference in behavior for different time scales, as observed in real traffic, and is moreover indistinguishable from real traffic by other statistical tests as well. Tools inspired from the models are then used to determine and calibrate the type of activity taking place in each of the ti…
Due to the COVID-19 pandemic, many governments imposed lock downs that forced hundreds of millions of citizens to stay at home. The implementation of confinement measures increased Internet traffic demands of residential users, in particular, for remote working, entertainment, commerce, and education, which, as a result, caused traffic shifts in the Internet core. In this paper, using data from a diverse set of vantage points (one ISP, three IXPs, and one metropolitan educational network), we examine the effect of these lockdowns on traffic shifts. We find that the traffic volume increased by …
The plethora of Internet of Things (IoT) devices leads to explosive network traffic. The network traffic classification (NTC) is an essential tool to explore behaviours of network flows, and NTC is required for Internet service providers (ISPs) to manage the performance of the IoT network. We propose a novel network data representation, treating the traffic data as a series of images. Thus, the network data is realized as a video stream to employ time-distributed (TD) feature learning. The intra-temporal information within the network statistical data is learned using convolutional neural netw…
University of Western Ontario（西安大略大学）；Juniper Networks（瞻博网络）
Real-world IP network traffic is susceptible to external and internal factors such as new internet service integration, traffic migration, internet application, etc. Due to these factors, the actual internet traffic is non-linear and challenging to analyze using a statistical model for future prediction. In this paper, we investigated and evaluated the performance of different statistical prediction models for real IP network traffic; and showed a significant improvement in prediction using the rolling prediction technique. Initially, a set of best hyper-parameters for the corresponding predic…
Mobile nodes are typically equipped with multiple radios and can connect to multiple radio access networks (e.g. WiFi, LTE and 5G). Consequently, it is important to design mechanisms that efficiently manage multiple network interfaces for aggregating the capacity, steering of traffic flows or switching flows among multiple interfaces. While such multi-access solutions have the potential to increase the overall traffic throughput and communication reliability, the variable latencies on different access links introduce packet delay variation which has negative effect on the application quality o…
Over the last years many technological advances were introduced in Internet television to meet user needs and expectations. However due to an overwhelming bandwidth requirements traditional IP-based television service based on simple client-server approach remains restricted to small group of clients. In such situation the use of the peer-to-peer overlay paradigm to deliver live television on the Internet is gaining increasing attention. Unfortunately the current Internet infrastructure provides only best effort services for this kind of applications and do not offer quality of service. This p…
Getting good statistical models of traffic on network links is a well-known, often-studied problem. A lot of attention has been given to correlation patterns and flow duration. The distribution of the amount of traffic per unit time is an equally important but less studied problem. We study a large number of traffic traces from many different networks including academic, commercial and residential networks using state-of-the-art statistical techniques. We show that traffic obeys the log-normal distribution which is a better fit than the Gaussian distribution commonly claimed in the literature.…
With the inevitable dominance of video traffic on the Internet, providing perceptually good video quality is becoming a challenging task. This is partly due to the bursty nature of video traffic, changing network conditions and limitations of network transport protocols. This growth of video traffic has made Quality of Experience (QoE) of the end user the focus of the research community. In contrast, Internet service providers are concerned about maximizing revenue by accepting as many sessions as possible, as long as customers remain satisfied. However, there is still no entirely satisfactory…
The DDoS attack is a serious threat to Internet of Things (IoT). As a new class of DDoS attack, Link-flooding attack (LFA) disrupts connectivity between legitimate IoT devices and target servers by flooding only a small number of links. In this paper, we propose an active LFA mitigation mechanism, called Linkbait, that is a proactive and preventive defense to throttle LFA for IoT. We propose a link obfuscation algorithm in Linkbait that selectively reroutes probing flows to hide target links from adversaries and mislead them to identify bait links as target links. To block attack traffic and f…
The Internet is transforming our society, necessitating a quantitative understanding of Internet traffic. Our team collects and curates the largest publicly available Internet traffic data containing 50 billion packets. Utilizing a novel hypersparse neural network analysis of "video" streams of this traffic using 10,000 processors in the MIT SuperCloud reveals a new phenomena: the importance of otherwise unseen leaf nodes and isolated links in Internet traffic. Our neural network approach further shows that a two-parameter modified Zipf-Mandelbrot distribution accurately describes a wide varie…
Network neutrality is related to the non-discriminatory treatment of packets on the Internet. Any deliberate discrimination of traffic of one application while favoring others violates the principle of neutrality. Many countries have enforced laws against such discrimination. To enforce such laws, one requires tools to detect any net neutrality violations. However, detecting such violations is challenging as it is hard to separate any degradation in quality due to natural network effects and selective degradation. Also, legitimate traffic management and deliberate discrimination methods can be…
Internet performance is tightly related to the properties of TCP and UDP protocols, jointly responsible for the delivery of the great majority of Internet traffic. It is well understood how these protocols behave under FIFO queuing and what the network congestion effects. However, no comprehensive analysis is available when flow-aware mechanisms such as per-flow scheduling and dropping policies are deployed. Previous simulation and experimental results leave a number of unanswered questions. In the paper, we tackle this issue by modeling via a set of fluid non-linear ODEs the instantaneous thr…
Volumetric attacks, which overwhelm the bandwidth of a destination, are among the most common DDoS attacks today. Despite considerable effort made by both research and industry, our recent interviews with over 100 potential DDoS victims in over 10 industry segments indicate that today's DDoS prevention is far from perfect. On one hand, few academical proposals have ever been deployed in the Internet; on the other hand, solutions offered by existing DDoS prevention vendors are not a silver bullet to defend against the entire attack spectrum. Guided by such large-scale study of today's DDoS defe…
Despite the prevalence of Internet of Things (IoT) devices, there is little information about the purpose and risks of the Internet traffic these devices generate, and consumers have limited options for controlling those risks. A key open question is whether one can mitigate these risks by automatically blocking some of the Internet connections from IoT devices, without rendering the devices inoperable. In this paper, we address this question by developing a rigorous methodology that relies on automated IoT-device experimentation to reveal which network connections (and the information they ex…
The literature survey typically predicated sharp growth for IP-based video traffic i.e., 30% or more annually. For the Internet TV in mobile networks, video traffic growth rate is expected to rise 80% or more. These high growth rates of video traffic will account for a large portion of the bandwidth. The performance of video-on-demand system during real-time data streaming greatly depends on the session oriented data-storage finding in the mass scale distributed storage architecture. At the storage end, data is broken up into manageable chunks of data packets, which could be smoothly, deliver …
Due to the rapid growth of data transmissions in internet of vehicles (IoV), finding schemes that can effectively alleviate access congestion has become an important issue. Recently, many traffic control schemes have been studied. Nevertheless, the dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies, which is significant for the random access resource allocation. In this paper, we consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it. Firstly, IoV devices are divided i…
Internet service discovery is an emerging topic to study the deployment of protocols. Towards this end, our community periodically scans the entire advertised IPv4 address space. In this paper, we question this principle. Being good Internet citizens means that we should limit scan traffic to what is necessary. We conducted a study of scan data, which shows that several prefixes do not accommodate any host of interest and the network topology is fairly stable. We argue that this allows us to collect representative data by scanning less. In our paper, we explore the idea to scan all prefixes on…
ISPs are increasingly selling "tiered" contracts, which offer Internet connectivity to wholesale customers in bundles, at rates based on the cost of the links that the traffic in the bundle is traversing. Although providers have already begun to implement and deploy tiered pricing contracts, little is known about how such pricing affects ISPs and their customers. While contracts that sell connectivity on finer granularities improve market efficiency, they are also more costly for ISPs to implement and more difficult for customers to understand. In this work we present two contributions: (1) we…
A COllective INtelligence (COIN) is a set of interacting reinforcement learning (RL) algorithms designed in an automated fashion so that their collective behavior optimizes a global utility function. We summarize the theory of COINs, then present experiments using that theory to design COINs to control internet traffic routing. These experiments indicate that COINs outperform all previously investigated RL-based, shortest path routing algorithms.
