Upload
on-lab
View
5.504
Download
0
Tags:
Embed Size (px)
Citation preview
ONOS:Open Network Operating System
Pankaj Berde
Open Networking Laboratory
ONOS: Distributed Network OS for Large Scale Networks
With Scale-out, High Availability and High Performance
Why? How? Results?
2
Example Large Networks: Service Provider Networks WAN core backbone
Multiprotocol Label Switching (MPLS) with Traffic Engineering (TE)
Metro networks Metro cores for access networks
Cellular access network LTE for a metro area
Wired access/aggregation Access network for homes DSL/Cable
Core
Cellular Metro
Wired Access
200-500 routers5K- 10K ports
20K-100K devices100K-10M+ ports
10K-50K routers2M-3M+ ports
10K-50K devices 100K- 1M+ ports
3
Key Performance Requirements for Core Backbone Network
Network
ONOS
Application ApplicationHigh Throughput ~500K - 1M path
setups/sec
Low Latency~10 - 100ms
4
~200GB - 1TBAnd will grow…
We decided to build a distributed NOSto meet these performance requirements
4
Global Network View/State ~3-6M Network State op/sec
Takeaways
Large networks require distributed network OS with scale-out, high performance and high availability.
Here distributed does not mean geo-distributed, but a single cluster of 5-25 co-located physical servers
5
Can One Build Distributed Network OS Stacking Open-Source Blocks?
Yes, one can. We did it. BUT …
6
Distributed Network OS withSimple Scale-Out
Controller Instance 2 Instance 3
Instance 1
Data plane
Network GraphGlobal network view
Control Application Control Application
Interconnection (e.g. 10G Ethernet)
Interconnection (e.g. 10G Ethernet)
7
Cassandra Distributed Key-Value Store
Titan Graph DB
Network Graph (Eventually consistent)
ONOS Architecture April 2013
Instance 1
OpenFlow Manager+
Instance 2
OpenFlow Manager+
Instance 3
OpenFlow Manager+
+ Floodlight Drivers
Control Application Control Application
Scale-out
Dis
trib
uted
Re
gist
ry(S
tron
gly
Cons
iste
nt)
Zook
eepe
r
Coor
dina
tion
Distributed Network
Graph/State
Applications Blueprints API
8
Host
Host
Host
Can One Build Distributed Network OS Stacking Open-Source Blocks?
Good for rapid prototypingBUT …
Lacks performance and performance visibility 9
Why Is It Hard To Get Performance Using Off-The-Shelf Software?
Bloated off-the-shelf open source components
Difficult to get visibility under the hood
Difficult to tweak for custom optimizations
10
Standard Ideas for Scalability and their Challenges
Partition data & processing so each instance can run independently
How do you partition optimally and specifically for NOS?
Replicate data to eliminate remote memory access
How do you keep replicas consistent as network state changes?
Make processing stateless to make failure recovery easy
How do you make state durable and where do you persist it?
11
Takeaways
Large networks require distributed network OS
Distributed network OS that can scale and have high performance with fault-tolerance poses many hard problems
12
ONOS: A Key SDN Building Block
13
Robust, High Performance Foundation
Cassandra Distributed Key-Value Store
Titan Graph DB
Network Graph (Eventually consistent)In-memory Network Graph (eventually consistent)
Host
Host
Host
Instance 1 Instance 2 Instance 3
OpenFlow Manager+
OpenFlow Manager+
OpenFlow Manager+
+ Floodlight Drivers
Control Application
Dis
trib
uted
Re
gist
ry(S
tron
gly
Cons
iste
nt)
Zook
eepe
r Control Application
Scale-out
Coor
dina
tion
Distributed Network
Graph/State
Applications ONOS Graph API
IndexingONOS Graph Abstraction
RAMCloudUltra-low latency distributed data store in DRAM
Even
t Noti
ficati
ons
Haz
elca
st
ONOS Today: Performance Focused
14
ONOS-Specific Data Model2013 Today: ONOS Optimized
ONOS Network Graph
Generic Graph
Generic Data Model ONOS-Specific Data Model
Generic Graph
Generic Data Model
ONOS Network Graph
ONOS Network Graph
• Data model optimized for ONOS• Direct access to data via known keys• Reduced read/write operations
• Network graph built on top of generic graph model
• Extra read/write operations due to indirection
15
How Did We Solve Hard Performance Challenges
Used lightweight open-source components which we can tweak like RAMCloud
Partitioning based on switches to exploit locality of data
Some state cached and replicated with state machine replication for sequential consistency
Used kernel-bypass for low-latency remote communication
16
Performance MetricsBasic network state update latency
Switch, port, and link discovery
Network event processing Rerouting of paths in response to link failures
Application path setup throughput
Important note:
These are relevant network OS performance metrics compared to what we measure using micro-benchmarks like cbench.
17
Performance Goals
Large Network
ONOS
Application Application
High Throughput: 500K – 1M setups/sec
Low Latency:10 – 100ms
Global Network View/State (200GB – 1TB+)
0.01
0.1
1
10
100
1000
Kernel Bypassw/ Infiniband
Latency [ms] (log scale)
(*)
(*)
Add Switch (w/ 4 ports)
Add Link
Network State Updates
0.0750.0990.150.2440.722
22.205
18
Network Event Processing
~40ms - ONOS latency~60ms – w/ FlowMods
~10k path reroutes/sec
Latency to reroute 1000 x 6-hop paths:
Path rerouting throughput:
19
Application Path Setup
~25ms – ONOS latency~60ms – w/ FlowMods
~20k path setups/sec
Latency to setup 1000 x 6-hop paths:
Path setup throughput:
20
ONOS Journey (so far…)
Oct, 2012 Apr, 2013 Dec, 2013 Today
Open-source building blocksHorizontally scalable
Highly availablePartition tolerant
Network graph abstraction
Proactive flow setupSDN-IP peering as an app
Demo app for service provider
New data modelIn-memory cache
Low-latency store (RAMCloud)Northbound API
Northbound events
10-100x Performance
NotificationsReactive Forwarding
Proxy ARP
CI and QA processImproved performance
Development VMIndustry standard tools/process
Proof of Concept Limited Release Performance
21
Start
Takeaways
Large networks require distributed network OS.
Distributed network OS poses many hard problems.
ONOS is on track to deliver a distributed OS with scale-out, high
availability and high performance.
22
Build your SDN using ONOS
ON.Lab will do an open source release with several use cases - Q4 2014
23
Team (We are hiring )
24
ONOS Demos at ONS 2014
ONOS on Internet2’s SDN network (Showcase)Key features on a real wide-area network
ONOS performance demo at ON.Lab boothLatest performance results w/ experts to answer questions!
25
Backup