View
265
Download
2
Category
Preview:
Citation preview
Peer-to-peer Virtual Private Networks and Applications
Renato Jansen Figueiredo
Associate Professor
Cloud and Autonomic Computing Center/ACIS Lab
University of Florida
Visiting Researcher at VU
2
Backdrop
Virtual machines in cloud computing
On-demand, pay-per-use, user-configurable
Federated environments
End-to-end Internet connectivity hindered by address
space and presence of NATs, firewalls
Network virtualization – seamlessly connecting
virtual machines across multiple providers
3
Rationale
Virtualization techniques for decoupling,
isolation, multiplexing also apply to networking
E.g. VLANs, VPNs
However, there are challenges in configuration,
deployment, and management
Peer-to-peer techniques provide a basis for
scalable routing, and self-management
Software routers, integration at network end-points
enables deployment over existing infrastructure
Architecture, design needs to account for
connectivity constraints, and support TCP/IP
efficiently; optimize for common cases
Application Examples
Cloud-bursting
Run additional worker VMs on a cloud provider
Extending enterprise LAN to cloud VMs – seamless
scheduling, data transfers
Federated “Inter-cloud” environments
Multiple private clouds across various institutions
Virtual machines can be deployed on different sites
and form a distributed virtual private cluster
Connecting devices of social network peers
Media streaming, file sharing, gaming, …
4
5
Talk - Outlook
Background
Architecting self-organizing virtual networks
Topology, routing, tunneling, addressing, NAT
traversal, performance
Uses in Grid/cloud and end-user environments
Virtual Private Clusters
Social VPNs
Applications
FutureGrid – high-throughput computing virtual
appliances
ConPaaS
6
Resource Virtualization
Virtual machines (Xen, VMware, KVM) paved
the way to Infrastructure-as-a-Service (IaaS)
Computing environment decoupled from physical
infrastructure
Pay-as-you-go for computing cycles
Virtual networks complement virtual machines
for increased flexibility and isolation in IaaS
VMs must communicate seamlessly – regardless of
where they are provisioned
Traffic isolation; security, resource control
7
VMM +
VN
Virtual Machines and Networks
WAN
Domain A Domain C
Domain B
V1
V2
V3
Physical
Infrastructure
Virtual
Infrastructure
Virtual Networks
Single infrastructure, many virtual networks
E.g. one per user, application, project, social
network…
Each isolated and independently configured
Addressing, protocols; authentication, encryption
Multiplexing physical network resources
Network interfaces, links, switches, routers
8
Network Virtualization – Where?
9
Network Device
Network Device
Network Fabric
(Virtual) machine
(Virtual) machine
Software Software Virtualized endpoints
Virtualized Fabric (e.g VLAN, OpenSwitch)
10
Landscape
Peer-wise Internet connectivity constrained
IPv4 address space limitations; NATs, firewalls
Challenges - shared environment
Lack of control of networking resources
Cannot program routers, switches
Public networks – privacy is important
Often, lack privileged access to underlying resources
May be “root” within a VM, but lacking hypervisor privileges
Dynamic creation, configuration and tear-down
Complexity of management
Technologies and Techniques
Amazon VPC:
Virtual private network extending from enterprise to resources at
a major IaaS commercial cloud
OpenFlow:
Open switching specification allowing programmable network
devices through a forwarding instruction set
OpenStack Quantum:
Virtual private networking within a private cloud offered by a
major open-source IaaS stack
ViNe:
Inter-cloud, high-performance user-level managed virtual
network
IP-over-P2P (IPOP)
Peer-to-peer, inter-cloud, self-organizing virtual network
11
Example: OpenFlow
Towards an open platform foundation
supporting “Software-Defined Networks” (SDN)
Interface standardized by Open Networking
Foundation (ONF)
Board members (as of 6/12): Google, Microsoft,
Facebook, Yahoo!, Deutsche Telekom, NTT, Verizon
Dozens of members: Citrix, Huawei, Orange, IBM,
Dell, HP, Oracle, Goldman-Sachs, …
Current (as of 6/12): OpenFlow Switch 1.3.0
12
OpenFlow Switch and Controller
13
Controller
Secure Channel
Group Table
Flow Table
Flow Table Pipeline
OpenFlow Protocol
Add, update, delete
Match flow table entry
Table miss
OpenFlow ingress port
OpenFlow output port
14
Peer-to-Peer Virtual Networks
Overview
User-level IP overlays deployable on Internet end
resources (software routers, virtual NICs)
Why virtual?
Hide complexities associated with NAT traversal,
IPv4 address space constraints from applications
Support unmodified applications
Why peer-to-peer?
Self-organizing - reduce management complexity
and cost
Decentralized architecture for scalability and
robustness
15
The IP-over-P2P (IPOP) Approach
Isolation Virtual address space decoupled from Internet
Packets picked, encapsulated, tunneled and delivered within the scope of virtual network
Self-organization Overlay topology, routing tables
Autonomously deals with joins, leaves, failures
Decentralized P2P messaging architecture No global state, no central point of failure
Tunnels (UDP, TCP, …), routing
Decentralized NAT traversal No need for STUN server infrastructure
[IPDPS 2006, Ganguly et al]
Application
VNIC
Virtual Router
Virtual Router
VNIC
Application Wide-area Overlay network
Isolated, private virtual address space
10.10.1.2
10.10.1.1
Unmodified applications Connect(10.10.1.2,80)
Capture/tunnel, scalable, resilient, self-configuring Routing; object store
IPOP: Architecture Overview
n1 n7
Bi-directional ring ordered by 160-bit IPOPid’s
Structured connections:
• “Near”: with neighbors
• “Far”: across the ring
Near
Far
Multi-hop path
between n1 and n7
n5
n6 n2
n3 n4
n8
n9
n10
n11
n12
n1 < n2 < n3 <……. < n13 < n14
P2P Overlay (Brunet)
IPOPid
18
Overlay: Edges and Routing
Overlay edges
Multiple transports: UDP, TCP, TLS
NAT traversal (UDP hole-punching)
Greedy routing
Deliver to peer closest to destination IPOPid
Constant # of edges per node (average k)
O((1/k)log2(n)) overlay hops
On-demand edges
Created/trimmed down based on IP communication
19
Creating Overlay Edges
• A sends a Connect-to-me (CTM) request to B’s IPOPid
• Contains all its URIs (UDP/TCP IP:port endpoints)
• Routed over P2P overlay to B
A B
CTM request
Overlay path
CTM reply
• B sends CTM reply with its URIs – overlay routed
Link request
• B initiates “linking” with A
• Attempts linking with parallel requests to A’s URIs
A’s endpoint URIs: tcp://10.5.1.1:3000 (local) udp://128.227.56.9:4433 (NAT – learned)
B’s endpoint URIs: tcp://172.16.2.1:5000 udp://129.15.56.9:6000
20
NAT Traversal
A B
Direct edge between A
and B
Technique for “cone” UDP NATs: A’s link request message to B creates ephemeral state in A’s NAT allowing messages from B to pass through NAT (and vice-versa) Overlay: manage keep-alives so NAT mapping “holes” stay open; re-link if NAT mappings expire
21
Naming and Multiplexing
One P2P overlay can multiplex multiple VNs E.g. multiple virtual clusters from different projects
IP routing within the scope of a namespace
User-provided string identifies IPOP namespace Each IPOP node is configured with a namespace
IP-to-P2P address resolution: DHT-Get(namespace:IP) -> IPOPid
22
Managing Virtual IP Addresses
Address assignment: static, or dynamic Supports DHCP
Store configuration (including base address, mask) on DHT entry bound to namespace
DHCP proxy runs on each IPOP node Pick DHCP request
Lookup DHCP configuration for namespace
Guess an IP address at random within range
Attempt to store in DHT; wait for majority to acknowledge; retry upon failure
23
At each node:
Count IP-over-P2P packets to other nodes
When number of packets within an interval
exceeds threshold:
Initiate connection setup; create edge
Trimming on-demand edges no longer in use
Overhead involved in connection maintenance
Optimization: On-demand edges
24
Optimization: Tunnel Edges
Peers X, Y may not be able to communicate
directly if they are behind symmetric NATs
X, Y exchange list of neighbor URIs
Each attempts to create edge to common
intermediary Z to serve as proxy
Routing – abstracted as regular overlay edge
X-Y connected by virtual edge
Useful to maintain ring topology in the face of failures
(routing outages, symmetric NATs)
25
Implementation
IPOP open-source system
C# user-level router
Tap virtual network device
Performance
1GbE physical LAN
Latency (ms) Bwidth (Mb/s) Mem (KB)
Host 0.27 941 n/a
IPOP 0.52 284 38312
IPOP+sec 0.75 55 50976
Performance (WAN)
EC2/UF EC2GoGrid UF/GoGrid
Netperf stream
native (Mbps)
89.2 35.9 30.2
Netperf stream
IPOP (Mbps)
75.3 19.2 25.7
Netperf RR
trans/s native
13.4 11.1 10.0
Netperf RR
trans/s IPOP
13.3 10.7 9.8
27
IPOP provides core primitives for packet
capture/injection and overlay routing
How to control which nodes connect to a
particular IP namespace?
Focus on two approaches:
Each peer decides the peers they connect with –
SocialVPN
Peers join groups and agree on a trusted third party
as mediator – GroupVPN
Access Control
28
Users now commonly manage relationships to
social peers through Online Social Networks
Facebook, Google+
Communication hindered by OSN provider
APIs, privacy concerns
A generic IP network can enable existing and
new social network applications
But users don’t have public IPs, don’t want to
necessarily open NATs/firewalls to all users
Users don’t want to configure and discover network
services manually
SocialVPN
Bob Alice
Alice's Compute Node Alice's Friend's Compute Node
Carl
Bob's Compute
Node on EC2
Social VPNs
OSN
XMPP
IP-over-P2P Tunnel
30
Social VPNs
From a user’s perspective: it’s simple
My computer gets a virtual network card
It connects me directly to my social peers
All IP packets: authenticated, encrypted, end-to-end
Leverage well-known PKI techniques
No configuration besides establishing social links
All I need to do to is log in to a web based social network
Applications, middleware work as if the
computers were on the same local-area network
Including multicast-based resource discovery –
UPnP, mDNS
31
Applications
Social VPN is not the application
It is not tied to an application either
It enables applications that are of interest for collaboration
Security – needed beyond network layer
Authenticated end-to-end private IP tunnels provide a
foundation
Traditional applications
Media streaming, desktop sharing, file sharing, cycle
sharing
Platform for decentralized social network
applications
Fault-tolerant micro-blogging, private file sharing, ..
32
Social VPN Internals
IPOP
NAT traversal and routing core
Private end-to-end tunnels
Peer discovery and certificate exchange
XMPP – Jabber, Google
Facebook APIs (was in first prototype; no longer in the code)
Dynamic IP address assignment
Facebook: more users than IPv4 24-bit private space
Also must avoid conflicts with local private networks, and
support mobility
Addressing and Mapping
160-bit P2P IDs used for overlay routing
Each node generates random P2P ID
Node issues a self-signed public key certificate with
its P2P identifier; publishes through OSN APIs
Certificates of friends’ nodes are discovered,
retrieved, revoked through OSN APIs
IPv4 addresses seen by applications
Dynamically-generated non-conflicting private subnet
Local node and friends’ nodes are mapped
dynamically to addresses within range
Naming possible through SocialDNS
IP src/dest addresses translated (ports are not)
[COPS 2008]
Alice
Alice's Compute Node Alice's Friend's Compute Node Bob's Compute
Node on EC2
Address Translation
Send-to BobP2P
SVPN: 192.168.0.0/16 Alice: 192.168.0.1 Bob: 192.168.2.2 -> BobP2P
SVPN: 172.16.0.0/16 Bob: 172.16.0.1 Alice: 172.16.3.4 -> AliceP2P
0.1 2.2 3.4 0.1 Recv-from AliceP2P
35
Group-oriented VPNs
Well-suited for collaborative environments for
cluster computing
Nodes who join a group have peer-wise
connectivity to all other nodes
Based on public key cryptography
Owner of a group is a certificate authority signing
GroupVPN certificates
Centralized Web-based interface hides low-
level management from users
Users create groups, determine who can join group
Certificate signing automated; group membership
Certificate revocation lists disseminated via P2P
36
Grid appliance clusters
Virtual appliances
Encapsulate software environment in image
Virtual disk file(s) and virtual hardware configuration
The Grid appliance
Encapsulates cluster software environments
Current examples: Condor, MPI, Hadoop
Homogeneous images at each node
IPOP/GroupVPN connecting nodes forms a cluster
Deploy within or across domains
37
Grid appliance - virtual clusters
Same image, per-group VPNs
copy
instantiate
Condor
+
Virtual
Network A Condor worker Another Condor worker
Repeat…
Virtual
machine
Group
VPN
GroupVPN
Credentials
(from
Web site) Virtual IP - DHCP
10.10.1.1 Virtual IP - DHCP
10.10.1.2
38
Grid appliance configuration
At the end of GroupVPN initialization: Each node of a private virtual cluster gets a DHCP
address on virtual tap interface
A barebones cluster
Additional configuration required depending on middleware Which node is the Condor negotiator? Hadoop front-end?
Which nodes are in the MPI ring?
Leverage P2P/IPOP primitives: Distributed hash table
Advertise (put namespace,managerIP); discover (get namespace)
IP multicast discovery over GroupVPN
39
Applications in FutureGrid
FutureGrid testbed
DAS-like system distributed across US institutions
Research, education, development, testing of Grid
and cloud computing middleware, applications
IaaS partitions: Nimbus, OpenStack, Eucalyptus
Virtual networks: ViNe and IPOP
Virtual appliances
Lower barrier to entry – pre-configured environments
40
IPOP + ConPaaS at VU
ConPaaS – framework/runtime to manage platform-as-
a-service environments
Examples: Web service, task farming service
Build upon IaaS primitives to create VMs
Integration with IPOP
Allow deployments to span across multiple providers
(federation; bursting; fault-tolerance)
Within VMs - no changes to IaaS stack
Isolate “data plane” communications from public Internet
Thilo Kielmann, Guillaume Pierre, Contrail/ConPaaS teams
41
IPOP + ConPaaS
WAN Private
Cloud
EC2
Zone
DAS
site
N1
N2
N3
IaaS
Providers
ConPaaS applications,
IPOP namespaces
172.16.10.10 172.16.10.20
172.16.10.20
PlanetLab bootstrap overlays
Grid appliance deployments:
Archer - ~700-CPU cluster
SocialVPN deployments:
Thousands of downloads, hundreds of
deployed nodes
Deployed Systems
43
Integration of IPOP with IPsec for dynamically-
provisioned cloud virtual networks
Contrail, ConPaaS
Overlay by-pass, integration with OpenFlow
software-defined networks
IPv6/IPv4 overlays, virtual clusters for high-
throughput computing, education
Archer (computer architecture)
FutureGrid (virtual appliances for education)
PRAGMA (Pacific Rim Grid)
Unstructured P2P SocialVPN
Discover, bootstrap, route through friends
On-going Work
44
Acknowledgments
ACIS P2P group (IPOP) Over the years: P. O. Boykin, Heungsik Eom, Arijit Ganguly,
Pierre St. Juste, Kyungyong Lee, Yonggang Liu, Girish
Venkatasubramanian, David Wolinsky, Jiangyan Xu
Vrije Universiteit,
ConPaaS team
FutureGrid, National Science Foundation
Awards 0751112, 0758596, 0910812
45
Thank you
For more information and downloads:
http://ipop-project.org
http://socialvpn.org
http://futuregrid.org
http://grid-appliance.org
46
Related Work
There exist several VPN technologies:
Enterprise VPNs (e.g. Cisco); Open-source (e.g
OpenVPN); Consumer/gaming/SMB (e.g. Hamachi)
Not easily applicable to federating cloud resources
Proprietary code; difficulty in configuration/management
Research work in the context of Grid/cloud
computing
VNET (Northwestern University), VIOLIN (Purdue
University), Private Virtual Cluster (INRIA), ViNe
(Tsugawa, Fortes @ UF)
Smartsockets @ VU
47
48
Bootstrapping a New Node
MyIPOPid
Forms a “leaf” connection with a public node - forwarder
Selected at random from list of “bootstrap” nodes
Sends CTM request addressed to its own IPOPid
Received by nearest neighbors
Creates structured connections; trims leaf connection
Forwarder
CTM (MyIPOPid)
Received by left and
right neighbors
(c) Renato Figueiredo 49
B2
IPOP Namespaces
N1
Namespace N1: 10.128.0.0/255.192.0.0
A1: 10.129.6.83
C1
B1: 10.129.6.71
D1
N1:10.129.6.71→
IPOPid x1
x1 x2
x4
x3
x5 x6 x7
x8
N2
A2
C2 D2
DHTLookup(N1:B1)
x1 IPOPid
“ARP cache”
IPOP packet DHTCreate(N2, 10.128.0.0/255.192.0.0)
N2:10.129.6.71→
IPOPid x2
DHTCreate(N2:A2,x2)
50
Social DNS
Motivation: • Users cannot define domain names used to access
their services in VPN settings
• Dynamic private networks; difficult to keep track of services by IP addresses
Objective: • A decentralized, naming service that gives
individuals the ability to select and share the domain names for their resources with SocialVPN peers
Approach: • Short names within social context
• Decentralized architecture where each node runs a local DNS server and communicates via SocialVPN
• Rank-based name conflict resolution
51
Security
End-to-end authentication and encryption IPsec tunneling over IPOP
Reuse existing software stack
End-to-end security implemented in IPOP RSA priv/pub keys
X.509 certificates
Point-to-point authentication and encryption TLS edges have been implemented
Difficulty to deal with NAT traversal
Point-to-point security in IPOP: ongoing work Reuses framework and code base from end-to-end
Avoid double-traversal of security stack for performance (e.g. shortcut connections based on IP traffic inspection)
52
Security
Appliance firewall Block outgoing packets to physical net
Except DHCP, DNS, IPOP’s UDP port
Confine traffic to within WOW and host-only
IPsec or IPOP security With IPsec, kernel/user tools reused – unmodified
Network routing is P2P, however: Trust can be managed by central CA
All intra-WOW communication authenticated and end-to-end encrypted using X.509-based PKI
Private net/netmask 10 lines of IPsec configuration for entire WOW
53
Linking and NAT traversal
Src = R:A
Dst = N:Y
Src = N:Y
Dst = M:X
R:A M:X N:Y S:B
Outgoing packet to M:X
(hole punched)
Src = S:B
Dst = M:X
Src = M:X
Dst = N:Y
Outgoing packet to N:Y
(hole punched)
Src = M:X
Dst = S:B
Exchange each other’s NAT assigned IP:port
M:X N:Y
NAT N 128.139.156.90
NAT M 128.227.56.83
S:B R:A Allow
Dropped
VNIC
Virtual Router
Virtual Router
VNIC
Application
Wide-area Overlay network
Local Interface
LAN Router
NIC
Application
NIC
Application
Avoiding LAN overheads
Supporting IPOP Routers
● Single IPOP router for a
(V)LAN
● Avoid need for IPOP
software stack on end
host
● Avoid IPOP overhead on
LAN communication
IP=10.1.1.2
Eth=A:B:C:D:E:0
IP=10.1.1.3
Eth=A:B:C:D:E:1
IP=10.1.1.4
Eth=A:B:C:D:E:2
TAP
Device
VPN
SoftwareNIC0
NIC1
Virtual Router
Internet
DHCP
● Provides address allocation and DNS settings
● IPOP router keeps a history of allocations and
ignores packets destined for them sent within the
(V)LAN
IP=10.1.1.2
Eth=A:B:C:D:E:0
IP=10.1.1.3
Eth=A:B:C:D:E:1
IP=10.1.1.4
Eth=A:B:C:D:E:2
TAP
Device
VPN
SoftwareNIC0
NIC1
Virtual Router
Internet
DHT
DHCP request
ARP
● Lookup Ethernet address from IP address
● IPOP router ignores ARP if IP in (V)LAN
● If destination is not on the LAN, check if such a
peer exists in the overlay
● Reply IPOP router addr.
IP=10.1.1.2
Eth=A:B:C:D:E:0
IP=10.1.1.3
Eth=A:B:C:D:E:1
IP=10.1.1.4
Eth=A:B:C:D:E:2
TAP
Device
VPN
SoftwareNIC0
NIC1
Virtual Router
Internet
ARP
not in router table? DHT
Local reply
Connectivity
Each node is given an IP
address and domain name
Access Control
The user locally decides to
allow or block another user
Trust
Use current social networking
systems (XMPP) to bootstrap
secure connections with
friends
Social VPN Prototype
Bob Alice
Alice's Compute Node Alice's Friend's Compute Node
Better QoS by leveraging
proximity or social trust
Flexibility in selecting
from a collection of
trusted compute nodes Alice can leverage
Her own resources
to add more
computational
power to her device
and save energy
Carl
Bob's Compute
Node on EC2
Cloud provider is now just
one more compute node
Computation offloading
Example: Resource discovery
Service discovery time
100 UPnP servers over WAN
U. Chicago, UC San Diego, and U. Texas
UPnP client located at U. Florida
Servers connected to PlanetLab SocialVPN overlay
Resource Discovery
Service discovery time
U. Texas U. Chicago UC San Diego
Service discovery time
(ms)
min max min max min max
27.0 29.4 45.4 47.6 54.6 57.0
• SocialVPN supports
unmodified UPnP applications
with service discovery time
commensurable to WAN
latency
• Wi-Fi setup has longer service
discovery time than wired LAN
(figure)
Offloading
Offloading to PC and EC2
Energy consumption
• The benefits are compelling at large image sizes
• Higher power consumption of offloading to Amazon EC2 than
offloading to local workstation due to network latency
Grid Appliance / Archer
2. Boot appliance:
automatically
joins Archer pool
Free pre-packaged Archer
Virtual appliances - run
on free VMMs (VMware,
VirtualBox, KVM)
Portal and Wiki:
Community-contributed
content: applications,
datasets, tutorials
Archer seed resources
300+ cores – Fall 2008
Archer Global
Virtual Network
System software:
Condor scheduler
NFS file systems
Simulators:
Simics, SESC,
Simplescalar
1: Download
appliance
3. Run architecture
simulation jobs on the
Archer pool through Condor
2. Boot appliance:
automatically
joins Archer pool
Free pre-packaged Archer
Virtual appliances - run
on free VMMs (VMware,
VirtualBox, KVM)
Portal and Wiki:
Community-contributed
content: applications,
datasets, tutorials
Archer seed resources
300+ cores – Fall 2008
Archer Global
Virtual Network
System software:
Condor scheduler
NFS file systems
Simulators:
Simics, SESC,
Simplescalar
1: Download
appliance
3. Run architecture
simulation jobs on the
Archer pool through Condor
Recommended