Multipath routing is useful for networks to achieve load sharing among multiple routing paths. Multipath BGP (MBGP) is a technique to realize inter-domain multipath routing by enabling a BGP router to install multiple equally-good routes to a destination prefix. Most of previous works did not distinguish between intra-domain and inter-domain multipath routing. In this paper, we present a measurement study on the deployment of M-BGP in a large Internet service provider (ISP) network. Our method combines control-plane BGP measurements using Looking Glasses (LG), and data-plane traceroute measurements using RIPE Atlas. We focus on Hurricane Electric (AS6939) because it is a global ISP that connects with hundreds of major exchange points and exchanges IP traffic with thousands of different networks. And more importantly, we find that this ISP has by far the largest number of M-BGP deployments among autonomous systems with LG servers. Specifically, Hurricane Electric has deployed M-BGP with 512 of its peering ASes at 58 PoPs around the world, including many top ASes and content providers. We also observe that most of its M-BGP deployments involve IXP interconnections. Our work provides insights into the latest deployment of M-BGP in a major ISP network and it highlights the characteristics and effectiveness of M-BGP as a means to realize load sharing.
Multipath BGP (M-BGP) allows a BGP router to install multiple 'equally-good' paths, via parallel inter-domain border links, to a destination prefix. M-BGP differs from the multipath routing techniques in many ways, e.g. M-BGP is only implemented at border routers of Autonomous Systems (ASes); and while it shares traffic to different IP addresses in a destination prefix via different border links, any traffic to a given destination IP always follows the same border link. Recently we studied Looking Glass data and reported the wide deployment of M-BGP in the Internet; in particular, Hurricane El…
BGP-Multipath (BGP-M) is a multipath routing technique for load balancing. Distinct from other techniques deployed at a router inside an Autonomous System (AS), BGP-M is deployed at a border router that has installed multiple inter-domain border links to a neighbour AS. It uses the equal-cost multi-path (ECMP) function of a border router to share traffic to a destination prefix on different border links. Despite recent research interests in multipath routing, there is little study on BGP-M. Here we provide the first measurement and a comprehensive analysis of BGP-M routing in the Internet. We …
Many organizations have the mission of assessing the quality of broadband access services offered by Internet Service Providers (ISPs). They deploy network probes that periodically perform network measures towards selected Internet services. By analyzing the data collected by the probes it is often possible to gain a reasonable estimate of the bandwidth made available by the ISP. However, it is much more difficult to use such data to explain who is responsible of the fluctuations of other network qualities. This is especially true for latency, that is fundamental for several nowadays network s…
Internet is composed of numbers of independent autonomous systems. BGP is used to disseminate reachability information and establishing path between autonomous systems. Each autonomous system is allowed to select a single route to a destination and then export the selected route to its neighbors. The selection of single best route imposes restrictions on the use of alternative paths during interdomain link failure and thus, incurred packet loss. Packet loss still occurs even when multiple paths exist between source and destination but these paths have not been utilized. To minimize the packet …
As the needs of Internet users and applications significantly changed over the last decade, inter-domain routing became more important to fulfill these needs. The ways how data flows over the Internet are still completely in the hand of network operators, who optimize traffic according to their own, local view of the network. We observe two potential limitations from this: Optimizing according to the local view may a) result in unused capacities in the global network and b) not meet the actual needs of users and applications. To identify and overcome these limitations, we present our BitTorren…
BGP route leaks frequently precipitate serious disruptions to interdomain routing. These incidents have plagued the Internet for decades while deployment and usability issues cripple efforts to mitigate the problem. Peerlock, introduced in 2016, addresses route leaks with a new approach. Peerlock enables filtering agreements between transit providers to protect their own networks without the need for broad cooperation or a trust infrastructure. We outline the Peerlock system and one variant, Peerlock-lite, and conduct live Internet experiments to measure their deployment on the control plane. …
The treatment of Internet traffic is increasingly affected by national policies that require the ISPs in a country to adopt common protocols or practices. Examples include government enforced censorship, wiretapping, and protocol deployment mandates for IPv6 and DNSSEC. If an entire nation's worth of ISPs apply common policies to Internet traffic, the global implications could be significant. For instance, how many countries rely on China or Great Britain (known traffic censors) to transit their traffic? These kinds of questions are surprisingly difficult to answer, as they require combining i…
Expanding the reach of the Internet is a topic of widespread interest today. Google and Facebook, among others, have begun investing substantial research efforts toward expanding Internet access at the edge. Compared to data center networks, which are relatively over-engineered, last-mile networks are highly constrained and end up being ultimately responsible for the performance issues that impact the user experience. The most viable and cost-effective approach for providing last-mile connectivity has proved to be Wireless ISPs (WISPs), which rely on point-to-point wireless backhaul infrastruc…
We propose an all-optical networking solution for a wide area network (WAN) based on the notion of multipoint-to-multipoint lightpaths that, for short, we call "multipaths". A multipath concentrates the traffic of a group of source nodes on a wavelength channel using an adapted MAC protocol and multicasts this traffic to a group of destination nodes that extract their own data from the confluent stream. The proposed network can be built using existing components and appears less complex and more efficient in terms of energy consumption than alternatives like OPS and OBS. The paper presents the…
There are currently no requirements (technical or otherwise) that BGP paths must be contained within national boundaries. Indeed, some paths experience international detours, i.e., originate in one country, cross international boundaries and return to the same country. In most cases these are sensible traffic engineering or peering decisions at ISPs that serve multiple countries. In some cases such detours may be suspicious. Characterizing international detours is useful to a number of players: (a) network engineers trying to diagnose persistent problems, (b) policy makers aiming at adhering t…
The security of the Internet's routing infrastructure has underpinned much of the past two decades of distributed systems security research. However, the converse is increasingly true. Routing and path decisions are now important for the security properties of systems built on top of the Internet. In particular, BGP poisoning leverages the de facto routing protocol between Autonomous Systems (ASes) to maneuver the return paths of upstream networks onto previously unusable, new paths. These new paths can be used to avoid congestion, censors, geo-political boundaries, or any feature of the topol…
In this work, we develop a distributed source routing algorithm for topology discovery suitable for ISP transport networks, that is however inspired by opportunistic algorithms used in ad hoc wireless networks. We propose a plug-and-play control plane, able to find multiple paths toward the same destination, and introduce a novel algorithm, called adaptive probabilistic flooding, to achieve this goal. By keeping a small amount of state in routers taking part in the discovery process, our technique significantly limits the amount of control messages exchanged with flooding -- and, at the same t…
Inter-domain routing is a crucial part of the Internet designed for arbitrary policies, economical models, and topologies. This versatility translates into a substantially complex system that is hard to comprehend. Monitoring the inter-domain routing infrastructure is however essential for understanding the current state of the Internet and improving it. In this paper we design a methodology to answer two simple questions: Which are the common transit networks used to reach a certain AS? How much does this AS depends on these transit networks? To answer these questions we digest AS paths adver…
Internal routing inside an ISP network is the foundation for lots of services that generate revenue from the ISP's customers. A fine-grained control of paths taken by network traffic once it enters the ISP's network is therefore a crucial means to achieve a top-quality offer and, equally important, to enforce SLAs. Many widespread network technologies and approaches (most notably, MPLS) offer limited (e.g., with RSVP-TE), tricky (e.g., with OSPF metrics), or no control on internal routing paths. On the other hand, recent advances in the research community are a good starting point to address t…
Modern Internet applications, from HD video-conferencing to health monitoring and remote control of power-plants, pose stringent demands on network latency, bandwidth and availability. An approach to support such applications and provide inter-domain guarantees, enabling new avenues for innovation, is using centralized inter-domain routing brokers. These entities centralize routing control for mission-critical traffic across domains, working in parallel to BGP. In this work, we propose using IXPs as natural points for stitching inter-domain paths under the control of inter-domain routing broke…
There have been several approaches to provisioning traffic between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase network capacity, and enhance network security services. MATE (Multipath Adaptive Traffic Engineering) protocol has been proposed for multipath adaptive traffic engineering between an ingress node (source) and an egress node (destination). Its novel idea is to avoid network congestion and attacks that might exist in edge and node disjoint paths between two core network nodes. This paper builds an adaptive, rob…
Several approaches have been proposed to the problem of provisioning traffic engineering between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase capacity, and enhance security services between two core (relay) network nodes, an ingress node and an egress node. MATE (Multipath Adaptive Traffic Engineering) has been proposed for multipath adaptive traffic engineering between an ingress node (source) and an egress node (destination) to distribute the network flow among multiple disjoint paths. Its novel idea is to avoid netwo…
BGP is the de-facto Internet routing protocol for exchanging prefix reachability information between Autonomous Systems (AS). It is a dynamic, distributed, path-vector protocol that enables rich expressions of network policies (typically treated as secrets). In this regime, where complexity is interwoven with information hiding, answering questions such as "what is the expected catchment of the anycast sites of a content provider on the AS-level, if new sites are deployed?", or "how will load-balancing behave if an ISP changes its routing policy for a prefix?", is a hard challenge. In this wor…
