An Introduction to Overlay NetworksPlanetLab: A Virtual Overlay Network Testbed
Suhas [email protected]
Communication Networks II Spring 2005
Talk Outline
Introduction: The future internet & related challengesOverlay Networks Planet Lab
What is PlanetLab ? Infrastructure and working conceptsThe purpose of PlanetLabPlanetLab OS Architecture Goals
Concluding remarks
Innovator’s Dilemma
The Internet is an enormous success storyCommerciallyImpact on our daily livesGlobal reach
Success has an unexpected cost: ossificationDifficult to deploy disruptive technology* (e.g.. IP Multicast)
Correct vulnerabilitiesIntroduce new capabilities
* A technology that would require major changes in the current internet.
Present day applications that require "disruption“:1. Application that may require Customized Routing 2. Peer-to-peer file sharing3. Content Distribution
These evolving applications require major changes in the internet (e.g. more intelligent routers).This change is resisted for obvious reasons (financial, etc) !!
What is an Overlay Network ?A logical network on top of the physical network.
Purpose-built virtual networks that use the existing Internet for transmission
The Internet was once deployed as an overlay on top of the telephony network
Overlay Networks: P2P example
Single Point of failurePerformance Bottleneck
Centralized Directory P2P Overlay (Napster Model) Hierarchical P2P Overlay
(KaZaa Model)
Overlay Networks: CDNThe First and the Last Mile
Peering Points
CDN company installs hundreds of servers throughout the Internet, typically placing them in lower-tier ISPs and Web hosting centers
Customer data is replicated on CDN servers; when a customer data is updated, the CDN network refreshes the contents on its (cache) servers
A CDN server contains data from many different content providers
DNS/ HTTP redirection in order to guide browsers to the right server
Motivation for a virtual overlay testbedThe research community is teeming with innovative planetary-scale applications and servicesOverlays offer an attractive way to introduce disruptive technology into the Internet, but…
There is a high barrier-to-entryThe next Internet will be created as an overlay on the current oneThere is NO vehicle to try out the next great ideas in this area (Enter )
What is PlanetLab ?
A collection of machines distributed over the globePlanetLab currently consists of 565 machines, hosted by 266 sites, spanning over 25 countries.
An overlay network testbedResearchers can experiment with a variety of planetary-scale servicesserves as a meta testbed
A deployment platformProvides seamless migration of an application from early prototype to a popular service that continues to evolve
A microcosm of the next InternetSupports the introduction of disruptive technologies into the Internet through the use of overlay networks
A consortiumA collection academic, industrial, and government institutions
Unfortunately no one can be told what PlanetLab is.. You have to see it for yourself!
Nodes
565 machines, hosted by 266 sites, spanning over 25 countries
All of the machines are connected to the Internet. The goal is for PlanetLab to grow to 1,000 widely distributed nodes
The PlanetLab Architecture
A PlanetLab user sees a private set of machines (nodes) on which he can run his applications (A slice)
User 1 User 2
The PlanetLab ArchitectureUsers are granted slices – Sets of nodes on which users get a portion of the resources
Internet
PrincetonRutgers
University
Intel Labs
Wash U
The PlanetLab Architecture
So, a PlanetLab user is using a set of distributed Virtual Machines (VMs)
User 1 User 2
PlanetLab OS Node Architecture Goals
SliceDistributed set of Virtual Machines treated as a single entity, providing the resources required by a single PlanetLab experiment
Provide a virtual machine for each service running on a node (a part of the total service slice)
Isolate virtual machines (Node Virtualization)
Allow maximal control over virtual machines
Allocation of resources to the VMsNetwork, CPU, memory, disk
Fair AllocationGuaranteed Allocation
VMM
Nod
e M
anag
er
PrivilegedServices
Un-privilegedServices
PlanetLab NodeArchitecture
Slices (VMs)
Loca
lA
dmin
PlanetLab Network ActorsResource Broker (per service)
Obtain tickets from agents on behalf of service managers
Service Managers (per service)Obtain tickets from brokerRedeem tickets with node managers to acquire resourcesIf resources can be acquired, start service
PlanetLab Network ActorsAgents (centralized)
Track nodes’ free resourcesAdvertise resources to resource brokersIssue tickets to resource brokers
Tickets may be redeemed with node managers to obtain the resource
Node manager (one per node)Create slices for service managers
When service managers provide valid ticketsAllocate resources for vservers
Resource Monitor (one per node)Track node’s available resourcesTell agents about available resources
Obtaining a Slice
Agent
Service Manager
Brokerticket
Resource Monitor
Resource Monitor
ticket
ticket
Current projects runningMonitoring the Network's Health
Monitor the health of the Internet, including watching for anomalies (worms, DDoS) and diagnosing failures.
Routing Overlays: Find alternative routes through the Internet, for e.g., routes that better match the application's requirements, and routes that more quickly recover from failures.
Content Distribution NetworksThese applications distribute content (e.g., web pages, domain names) widely across the Internet, with the goal of improving system throughput under heavy load (e.g., flash crowds and DDoS attacks)
Internet InfrastructureProvide new services for the Internet at large. They may provide new and better replacements for existing Internet services, or they may integrate entirely new functionality into legacy services, or both.
Distributed Query ProcessingThese applications have grown out of the intersection of sensor nets and distributed databases. They allow users to submit queries about information reported by sensors distributed across Planetlab.
etc…
Overlay Testbeds“The X-Bone dynamically deploys and manages Internet overlays to reduce configuration effort and increase network component sharing. The X-Bone discovers, configures, and monitors network resources to create overlays over existing IP networks. “
“Develop and deploy advanced network applications and technologies, accelerating the creation of tomorrow’s Internet.”
Focus is on automated establishment and management of overlays
Allows multiple levels of virtualization but closely tied to current internet architecture
1. A large set of geographically distributed machines2. A realistic network substrate that experiences
congestion, failures, and diverse link behaviors3. Allows services and applications based on
architectures radically different from current arch. to be tested and evaluated
4. Serves as a deployment platform and a catalyst for bringing about necessary change to the present internet.
Live PlanetLab Node monitoring (Using Ganglia)*
Load, Memory, CPU usage and Network throughput for the last 1 Hour (29th APR 2005) at one of the 6 PlanetLab nodes currently at MIT:
* http://planetlab.millennium.berkeley.edu/ganglia_v2/
Questions ?
ReferencesOvercoming the internet impasse through virtualization: L. Peterson, S. Shenker, and J. Turner. Third Workshop on Hot Topics in Networking (HotNets-III) (November 2004).A Blueprint for Introducing Disruptive Technology into the Internet: Larry Peterson, Tom Anderson, David Culler and Timothy Roscoe, Proceedings of ACM HotNets-I Workshop, Princeton, New Jersey, USA, October 2002.Operating systems support for planetary-scale network services: Andy Bavier, Mic Bowman, Brent Chun, David Culler, Scott Karlin, Steve Muir, Larry Peterson, Timothy Roscoe, Tammo Spalink, and Mike Wawrzoniak: First Symposium on Networked Systems Design and Implementation (NSDI) (March 2004), 253-266
AcknowledgementsKishore Ramachandran, Grad Student, WINLAB, RutgersPravin Shankar, Grad Student, DISCO lab, CS Dept, Rutgers.