Upload
laurence-chambers
View
213
Download
0
Embed Size (px)
Citation preview
Copyright 2004 Lucent Technologies Inc. All rights reserved.
MPLS Network Tuning:How to Squeeze Most From Your
Network
Swarup Acharya ([email protected])
Technical Manager, Multiservice Networking Research Group
Optical Networking Division, Bell Laboratories
2Copyright 2004 Lucent Technologies Inc. All rights reserved.
Multi-Protocol Label Switching
MPLS has emerged as the foundation of next-generation data networks
– Provides the underpinnings of the converged network vision
– “Connection-oriented” veneer on an inherent connectionless IP network
IP/MPLS market continues to grow
– Year-over-year traffic growth: 119% (2002), 118% (2003), 84% (2004) [Infonetics Research]
Drivers for data network convergence
– CapEx Savings, OpEx savings, Competition, Convergence..
This talk will focus on Network Management challenges in delivering the lower CapEx, OpEx promise of MPLS
3Copyright 2004 Lucent Technologies Inc. All rights reserved.
MPLS Traffic Engineering
The cornerstone of MPLS is Traffic Engineering (TE)
– Given a new demand, how best to route it in the network?
– No longer limited by IGP (OSPF, IS-IS) restrictions of:
• Destination-based forwarding
• Simple “additive” min-cost routing, ignorant of bandwidth
MPLS TE enables:
– Constrained-based routing
• Bandwidth, delay etc..
– Explicit routing (aka Source routing)
– Label Switched Paths (LSP)
• With appropriate resources reservations via RSVP
4Copyright 2004 Lucent Technologies Inc. All rights reserved.
TE Benefits
C
D
B
E
A L3
L2
L1
Path from L1-L3Path from L2-L3
IGPShortest PathRouting
“Longer” paths under-utilized,“Shorter” paths bottlenecked!
C
D
B
E
A L3
L2
L1
TrafficEngineeredPaths
TE enables a more load-balanced network..
5Copyright 2004 Lucent Technologies Inc. All rights reserved.
Will TE alone provide all the efficiency you can get?
MPLS + TE moves the network from packet-switched to a “virtual” circuit-switched IP network..
What about bandwidth inefficiencies inherent in circuit-switched environments such as SONET,SDH, ATM…?
Will MPLS suffer the same fate?
TE Necessary, But is it Sufficient?
General belief that Traffic Engineering enables efficient MPLS networks
– “Traffic engineering reduces the overall cost of operations by more efficient use of bandwidth resources… .. [Cisco documentation]
6Copyright 2004 Lucent Technologies Inc. All rights reserved.
SONET/SDH Bandwidth Fragmentation
Primary cause of poor SONET/SDH efficiency
Circuit churn leaves behind “stranded” capacity
New requests often denied even if sufficient capacity exists
– Often at only ~30-40% network utilization
SONET NMSs have had “constrained-based” routing for a while now..
– Bandwidth accounted in routing
– Arcane SONET/SDH constraints
Increasingly, defragmentation tools used to “recover” capacity
– Often, in conjunction with newer Optical Control Planes
7Copyright 2004 Lucent Technologies Inc. All rights reserved.
SONET/SDH Link Fragmentation
2.5 Gbps Link on a Fiber
622 Mbps 622 Mbps 622 Mbps
b c d fa e
622 Mbps
c d
622 Mbps 622 Mbps 622 Mbps
b fa e
622 Mbps
New request for a622 Mbps demand rejected!
Request accommodated!
• By rearranging traffic carried on different time slots, capacity on the link can be freed and reused
Symbol Bandwidth SDH SONET
Fiber/link 2.5 Gbps STM-16 OC-48
622 Mbps STM-4 OC-12
155 Mbps STM-1 OC-3
Legend/Glossary
e
a
8Copyright 2004 Lucent Technologies Inc. All rights reserved.
SONET/SDH Ring Fragmentation
New STS-3 demand fromNode2-4 rejected!
Accepted here
7-Node STS-48 BLSR RingAlternate Routing of
Same Demands on Ring
Nodes
Tim
e S
lots
Is there an MPLS analogue?
9Copyright 2004 Lucent Technologies Inc. All rights reserved.
MPLS Network Example
: Bandwidth of all Links
L1, L2, L3: LSPs of /2 b/w
New Service Request:
Setup LSP L4 between A and C, Bandwidth
C
DBF
E A
L2 L1
L3
Router A rejects request:
No available route meeting b/w requirements
No available bandwidth? Or, Is the bandwidth fragmented?
10Copyright 2004 Lucent Technologies Inc. All rights reserved.
Avoiding Bandwidth Fragmentation
TE alone does not guarantee high efficiency
– L1-L3 were “optimally” routed in both cases, yet fragmentation occurs
Key: Can the demands be routed without adding new hardware?
Lower Fragmentation Higher Utilization Lower CapEx
C
DBF
E A
L1
L2
L3
L4Alternate routing for L1-L3,
enables L4 to be met
11Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Tuning
Fragmentation is a problem for MPLS networks too
In general, Network Management systems need to provide Network Engineering tools to address fragmentation
– “Traffic Engineering puts traffic where the bandwidth is, Network Engineering creates bandwidth where the traffic will be..”
– Relatively little focus on engineering tools
– Network engineering requires “global” knowledge, TE is a per-LSP optimization
However, network engineering operation cannot be service disruptive
Network Tuning: Hitless, Disruption-free Network Engineering
– Network tuning is NOT network planning
– For live, operational networks, not greenfield designs
12Copyright 2004 Lucent Technologies Inc. All rights reserved.
Re-cap
Did the Net-Heads check with the Bell-Heads as to what NM quagmire they were getting into?
MPLS Traffic Engineering:
• Helps avoid congestion prevalent in native IP networks
• Limited ability to mitigate circuit-switched inefficiencies
• On its own, cannot extract the most juice from the network
Critical need for Network Tuning tools
No reason why MPLS cannot become equally inefficient down the road..
Key Tradeoff:
Grow infrastructure to meet traffic demand [CapEx Hit], OR,
Tune network for improved efficiency [OpEx Hit]?
13Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Tuning Scenarios
The router rejected a new LSP setup request due to insufficient bandwidth. Can I engineer the current LSP routes to “free” the necessary bandwidth for the new one?
I need to bring down a router for an OS upgrade. Can I:
a) Re-route the LSPs on the router to avoid bringing them down?
b) Upgrade the router OS and then revert them back to their original routes?
The traffic on a node/set of links had exceed the recommended load threshold. Can I move traffic from the “hot zone” to minimize damage in case of failure?
Can I have automated, scalable Network Tuning tools?
14Copyright 2004 Lucent Technologies Inc. All rights reserved.
Bell Labs Möbius Tool:MPLS Provisioning, Tuning System
Link color indicates load (Red: high, Green: acceptable load)
Support For:- Cisco
72*/75*/120*
- JNPR M*, T* - ERX
16Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Tuning: Fail-Setup Optimization
17Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Tuning: Fail-Setup Optimization
18Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Tuning: Fail-Setup Optimization
19Copyright 2004 Lucent Technologies Inc. All rights reserved.
Optimization Done!
Network Tuning: Fail-Setup Optimization
21Copyright 2004 Lucent Technologies Inc. All rights reserved.
Impacted Circuits (Old + New Routes)
1: Re-route
2: Re-route 3: Provision
22Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network View (Before)
Network View (After)
Clear Traffic in this“Hot Zone” below
specific threshold..
Network Tuning: Load Balance (“Hot Zone” Clearing)
23Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Engineering Requirements
Step-by-step Migration Sequence– Operating on live traffic -- providing a design for an “optimized” layout
does not help
How do I get from current LSP layout to the new layout?
– Original LSP QoS constraints have to be maintained on new route
C
DBF
E A
L2L1
L3
C
DBF
E A
L2
L1
L3
C
DBF
E A
L2
L1
L3L4
Re-route L2 Re-route L3 Provision L4
24Copyright 2004 Lucent Technologies Inc. All rights reserved.
Algorithmic Challenge
Migration a very challenging theoretical problem (“NP-hard”)
– Problem of scale -- exponential search space
Requires innovative algorithms for large networks
How to scale to a network with 10s of routers, 100s of links and 1000s of LSPs?
In Bell Labs, we have patent-pending algorithms to provide migration sequence in “real-time”
– Milliseconds to seconds for reasonable sized networks
0
10
20
30
40
50
60
70
80
90
100
30 50 70 80 90
Network Load (%)
% H
ot
Zo
ne
Cle
ared
HZ LSP Rerouted Non HZ LSP Rerouted
Hot Zone Load Balancing
• Hot Zone size = 10% Network• Goal: Clear all LSPs in Hot Zone• Chart shows contribution of LSPs outside the hot zone in meeting the goal as network loads increase
25Copyright 2004 Lucent Technologies Inc. All rights reserved.
Requirements II: Hitless, Disruption-free Engineering
“Hitless” is not zero packet loss
• In reality, everything is only near-hitless
Requirement: It should be perceived as hitless from the application’s perspective
• SONET/SDH has a 50ms grace during protection switching
• Even with a 100 ms hit, can do >500 re-routes before a 4-9s reliability SLA is broken.
MPLS provides infrastructure for hitless re-routing
26Copyright 2004 Lucent Technologies Inc. All rights reserved.
MPLS make-before-break
Mechanism to achieve hitless LSP re-routing – Signal new route, switch traffic and delete old route
Signaling protocols use intelligence when reserving bandwidth on new route
– E.g., Shared Explicit (SE) style flag in RSVP to avoid double bandwidth reservation on common links
Possibility of packets going out-of-order – If new route is significantly shorter than original route
– Even if it occurs, very short duration and bounded
27Copyright 2004 Lucent Technologies Inc. All rights reserved.
Inducing make-before-break
Typically, make-before-break is an internal function
– Used by router to re-route LSPs (e.g, if ‘re-optimize’ flag on)
For network engineering, make-before-break needs to be triggered from the outside. Also:
– New path is given (as opposed to router calculating it)
– If the new path is bad, traffic should not switch and bring the LSP down!
Routers need to provide mechanism to trigger make-before-break
– Backdoors available - varies by vendor OS
– Insert the new path with a higher priority (lower path option) and force a re-optimization
– Replace the Explicit Route Object (ERO) with a new route
28Copyright 2004 Lucent Technologies Inc. All rights reserved.
Requirements III: Preserve Network Stability
Should not bring down customer traffic
• Key: Traffic should switch only if new path is up!
LSP re-routes does not change the IP topology
• Only the path is changed, not the connectivity
Will cause OSPF updates
• Bandwidth on links will change
Should be attempted during “lean” traffic periods
29Copyright 2004 Lucent Technologies Inc. All rights reserved.
CapEx Savings from Tuning
Simulation Model
40 Node, 100 Link network (to start)
LSP Traffic randomly generated and routed on shortest available path
On failure to setup LSP due to lack of bandwidth:
Case I: No Engineering
A new link added between source and destination
Case II: With Engineering
Attempt to re-route LSPs to create “free” space for new one
On failure, new link added between source and destination
150
200
250
300
350
400
450
100000 300000 500000 700000 900000 1100000
Traffic Volume (MB)
#OC
192
Por
ts
Norm al Operation Mobius--- No Engineering --- With Engineering
More traffic for same infrastructure
Network Growth with Increased Traffic
30Copyright 2004 Lucent Technologies Inc. All rights reserved.
CapEx Savings - II
20
40
60
80
100
120
140
100000 200000 300000 400000 500000
Traffic Volume (Mb)
# O
C-1
92 P
orts
No Engineering With Engineering
Alternative Simulation Model
10 Node, 15 Link network (to start)
LSP Traffic randomly generated and routed on shortest available path
On failure to setup LSP due to lack of bandwidth:
Case I: No Engineering
A new link added on the hop that is out of capacity
Case II: With Engineering
Attempt to re-route LSPs to create “free” space for new one
On failure, new link added on the hop that is out of capacity
31Copyright 2004 Lucent Technologies Inc. All rights reserved.
Network Management Systems
Traditional IP networking view is that router has all the smarts
Engineering “intelligence” requires network-wide view
– Has to reside in a single “entity”
– Too complicated to co-ordinate engineering operations across different routers in a distributed fashion
Good choice: MPLS Network Management System (NMS)
NMS needs to provide support for:
– Algorithmic and graphical tools for what-if scenarios
– Seamless, point-n-click support to execute optimizations
– Support for proactive engineering
• No longer limited to “reactive” operational mindset
Requisite NMS tools can lower operations overhead from hours/days to minutes!
32Copyright 2004 Lucent Technologies Inc. All rights reserved.
Fast Network Adaptors Creation Large Multi-vendor Testing Labs
Committed Corporate Partnership with Hardware Vendors
Component-Based, Multi-Vendor Activation
Rich set of Services over L2 & L3
Software Development Kits for Network Element Support
NavisNavis Provisioning ManagerProvisioning Manager
WorkManager
WorkManager
L2&3
Order GatewayOrder Gateway
Inventory Gateway
Inventory Gateway
Network AdaptorsNetwork Adaptors
Service Modules Service Modules
Routing & MPLS TERouting & MPLS TEL2&3
Lucent’s Navis Provisioning Manager
Component-based, Multi-vendor Layer 2/3 NMS
ActivationFlow
ConfigurationFlow
Core IP/MPLS EMS / NECore IP/MPLS EMS / NE FR/ATM EMS / NEFR/ATM EMS / NE Access EMS / NEAccess EMS / NE
Service Modules
VPN
ATM Frame Relay xDSL
MPLS
ATMoMPLS Ethernet IPSEC
Möbius
33Copyright 2004 Lucent Technologies Inc. All rights reserved.
Conclusions
MPLS gaining momentum in service provider networks
Industry focus on Traffic Engineering
– Does not suffice to get the most from the network
– Need to also consider network engineering and hitless tuning
MPLS provides the necessary infrastructure for network tuning
– Necessary requirement to avoid inefficiencies in circuit-switched environments
Effective network tuning improves network utilization, lowers CapEx!