Upload
marta-pacyga
View
366
Download
0
Embed Size (px)
Citation preview
You can also use images
that are high tech and
interesting.
To change
background image;
Go to View tab, slide master
PLNOG 2015
SDN and NFV
Krakow, 28th & 29th September 2015
You can also use images
that are high tech and
interesting.
To change
background image;
Go to View tab, slide master
Assuring Performance, Scalability and
Reliability in NFV Deployments
Ronald Mai
Senior Systems Engineer - EMEA
‹#› PROPRIETARY AND CONFIDENTIAL
Agenda
Testing NFV
• Benefits of NFV and the Testing Implications
• Challenges for NFV and the Testing Requirements
Test Tools Old & New
• What Existing Tools Do
• Hardware Tester Architecture – benefits/challenges
• Virtual Machine Tester Architecture – benefits/challenge
• What can we learn and what do we lose?
New NFV Test Methodologies
‹#› PROPRIETARY AND CONFIDENTIAL
TESTING NFV PLNOG 2015
‹#› PROPRIETARY AND CONFIDENTIAL
PASS Methodology
Performance
• Data-plane throughput,
latency, latency variation etc.
Availability
• Control-plane convergence
and failover mechanisms
• Data-plane reliability under
load
Security
• VLAN/VPN leakage
• Firewall Performance
Scale
• Control-plane peer scale
• Routing table scale
• Session quantity and
establishment rate
• Flow scale
Conformance
Interoperability
‹#› PROPRIETARY AND CONFIDENTIAL
Benefits of NFV and the Testing Implications
Benefit Impact
Reduced Equipment Cost and
Reduced Power Consumption Equivalent Testing Costs must fall
Reduced Time-to-Market for
Innovative New Services
Test systems must integrate with
new lab platforms and be capable of
automation
Possibility of Running Production,
Test and Reference Facilities on the
same infrastructure
As above. Integrating Test System
with Orchestration is key
Virtual Test Ports, Standard APIs and Orchestration
Integration are key
‹#› PROPRIETARY AND CONFIDENTIAL
Benefits of NFV and the Testing Implications (cont.)
Benefit Impact
Optimizing network configuration/
topology in near real-time based on
traffic and service demand
Test it!
What effect does this have on QoE for
service user.
Temporarily repair failures by
automated re-configuration and moving
network workloads onto spare capacity
Test it!
Do the failover mechanisms work?
What is the service impact during re-
configuration
Rapid Scaling of Services to meet real-
time demand. Scaling-up and scaling-
out of capacity under orchestration
control
Test it!
New methodologies required.
Does the orchestration mechanism
respond correctly to demand. How is
existing traffic affected when it does?
‹#› PROPRIETARY AND CONFIDENTIAL
Challenges for NFV and the Testing Requirements
Challenge Requirement
Portability/
Interoperability
Test the functionality and performance in all
data centre environments that will be
encountered in service
Performance Trade-Offs
when using industry
standard hardware
Benchmark existing services (e.g. latency,
delay variation, power consumption for
different service levels). Determine the
resources required to continue meeting SLAs
Migration and Co-
existence/ Compatibility
with legacy platforms
Test services using a mixture of virtual and
physical network appliances
‹#› PROPRIETARY AND CONFIDENTIAL
Challenges for NFV and the Testing Requirements (Cont.)
Challenge Requirement
Network Stability
Determine stability of data and control-planes
when large numbers of VMs are being created
or re-located
Integration
Test service chains as well as individual VNFs.
Requires complex protocol support from test
ports
Security and Resilience
Induce failure and test service downtime
(while network function is re-created).
Test servers, hypervisors, virtual appliances
and orchestration mechanisms against security
attacks.
‹#› PROPRIETARY AND CONFIDENTIAL
Testing Within the NFV Infrastructure
Test Path Possibilities
vSwitch performance, availability and scalability
VNF performance, availability and scalability
Server performance, availability and scalability
‹#› PROPRIETARY AND CONFIDENTIAL
PASS Methodologies for NFV
Performance
• Data-plane throughput, latency,
latency variation etc.
• VNF vs Dedicated hardware
• Effect of real-time optimization
on QoE
• Performance per environment
• Service Chain performance
• Power Consumption
Availability
• Control-plane convergence
• Data-plane reliability under load
• Migration and Auto-scaling
(SLA Maintenance)
Security
• VLAN/VPN leakage
• Firewall Performance
• Security of virtual
infrastructure
Scale
• Control-plane peer scale
• Routing table scale
• Session quantity and
establishment rate
• Capacity of NFVI
‹#› PROPRIETARY AND CONFIDENTIAL
TEST TOOLS OLD AND NEW
PLNOG 2015
‹#› PROPRIETARY AND CONFIDENTIAL
Test Ports Emulate Complex Environments
1G or 10G
Ethernet
V4 & V6
Addresses
RIP, BGP, IS-
IS or OSPF
10G, 40G or 100G
Ethernet
MPLS Label Stack
IS-IS or OSPF
Multi-Protocol iBGP
LDP
BFD
VRFs
Firewall Functions
Border Relay
‹#› PROPRIETARY AND CONFIDENTIAL
Hardware-based Tester Model
Advantages
Repeatable results
Line rate traffic
High-scale control-plane
Accurate (to ~5nS) across
millions of streams
Single management interface
Easily automated
Cost effective
• Emulate realistic environment
• Power, real-estate
Hardware-based
Test Device
Data-plane traffic
Control-plane peering,
updates etc.
‹#› PROPRIETARY AND CONFIDENTIAL
Module Module
Module Module
Architecture of a Hardware Test Device
Controller
Module
CPU /
MEM
CPU /
MEM
CPU /
MEM
CPU /
MEM
GPS
PTP Compute
Resource
CPU
Core
CPU
Core
CPU
Core
PHY
PHY
PHY
FPGA
FPGA
FPGA
16 PROPRIETARY AND CONFIDENTIAL
Constant Bitrate
Traffic (CBR)
Variable Bitrate
Traffic (VBR)
Continuous Burst
Microburst
Realizing Test Functionality in VM Equivalents
CPU
Stateful control-plane protocols
Emulated and Simulated devices (L2-7)
• 1000s of peers per port
• Millions of routes per port
Test Configuration and control
Results processing and database
FPGA
Line rate performance
Traffic Generation and Analysis
• Sophisticated scheduling
• 1 Million flows per port
individually measureable in real-
time
Accurate & Stable time stamping
High-resolution sampling
Line Rate Capture buffers
Work is in progress to enhance soft FPGA performance
In cases where replicating hardware performance is not possible new methodologies are being developed
17
SAMPLE NFV TEST
METHODOLOGIES
PLNOG 2015
18
Target D/SUTs
• vBNG (PPPoE/DHCP)
• vCPE [vFW, vLB, vRouter] (IGMP, DHCP, OSPF/BGP, Stateful traffic)
• vPE (BGP, MPLS VPN, VPLS)
• System Infrastructure performance – Hypervisor, OS, vSwitch, vNIC
Measure
• Forwarding throughput (RFC 2544)
• Latency/Jitter – TWAMP Latency
• Orchestration with VM/VNF/(V)TA auto-scaling
Forwarding Performance Benchmarking of a VNF
Test Topology
(Virtual) Test Appliance (virtual) Test Appliance VNF under test
(V)TA (V)TA
Traffic
(V)TA (V)TA
19
Objective - Test the fail-over convergence time when one of the VNFs fails and back up path has been configured for the test topology
Convergence Configurations
• ECMP Load sharing over Active/Active Paths
• Active/Standby Paths – Failover to Standby Path
Measure:
• Convergence Time
• Impact on convergence time of route/VRF table size
Fail-over Convergence Measurement
(Virtual) Test
Appliance
Test Appliance
VTA/
TA
Simulated
Endpoints
DUT Virtual Routers
(VNFs)
Emulated
Router
VTA/
TA
VTA/
TA
Traffic
.
.
.
.
Simulated
Endpoints
VTA/
TA
.
.
.
.
20
Objective - Determine the performance of a distributed VNF during and after the migration of one or more constituent VMs
• Migration of a constituent VM from one physical server to another
• Migration of a VNF in service chain from one physical server to another
• Migration of VM or VNF across data centres
Measure (during a scheduled VNF VM Migration)
• Throughput and Latency before and after migration
• Service disruption time
Performance Impact of VM Migration
Test Appliance Test Appliance
.
.
.
.
VTA/
TA
VTA/
TA
.
.
.
.
Simulated
Server Cloud
Simulated
Workload Clients
Server 1
Server 2
Traffic
21
Objective - Test the auto-scaling functionality of the VNF.
• Auto scaling triggered by mechanisms such as an embedded monitoring function/
threshold crossing detection & event notification
• Example – an increase in the number of PPPoE or DHCP incoming session requests
(beyond the scale supported by one VM)
Measure
• Disable the auto-scaling feature on DUT in order to base line the performance
• Re-Enable auto-scaling and gradually increase load
• Record the transactions/sec, average, min and max response time
• Record the total number of VMs instantiated by the VNF
• Record the NVFI resources used by the VNF (processor, memory, storage)
Auto Scaling of VMs in a VNF
Virtual Test Appliance Virtual Test Appliance VNF Under Test
.
.
.
.
VTA VTA
.
.
.
.
Simulated
Server Cloud
Simulated
Workload Clients
22
NFV brings new benefits and challenges that require new testing techniques
Existing testing technology has been virtualized thus building on many years of experience
Virtualized test environments are challenging. The test community is:
• Addressing the challenges where this is technically feasible
• Creating new methodologies where it is not
Conclusions
PROPRIETARY AND CONFIDENTIAL
Back-up Slides
25 PROPRIETARY AND CONFIDENTIAL
Control-plane Functionality of a Hardware Tester
Application Layer Protocols and Triple Play
HTTP, FTP, SIP, Video, DPG, XMPPvJ CIFS, Storage IO
IPTV & Video Quality Analysis
Switching
OpenFlow, TRILL, FC, FCoE, LACP, LLDP/DCBX, SPB, STP,
VEPA, VIC
Carrier Ethernet
EOAM, IEEE 1588v2 Link-OAM, SyncE, TWAMP
MPLS & MPLS-TP
6PE/6VPE, LDP, BGP VPLS, LDP VPLS, GMPLS, RSVP-TE,
Multicast VPN, LSP-Ping, MPLS-TP Y,1731OAM
Routing
BGP, OSPFv2 & v3, ISIS RIP(NG) BFD, PIM, LISP
Access
ANCP, DHCP. DHCPv6/PD L2TP, PPPoX, IGMP/MLD 802.1X,
IPv6 Autoconfiguration
Confo
rmance |
| F
unctio
nal |
| P
erfo
rmance
Next generation platform from Spirent
26
PPPoE connections
MPLS tunnels
BFD for fault detection
Physical test devices emulate DSLAMS and 1000s of PPPoE client on one side and Edge and core routers on other side
In above example, the VNFs under test are virtualized BNG/PE running on standard server
• PPPoE and MPLS connections formed between test device and
VNF under test
VNF Functional & Performance Testing 1. Using physical test devices to validate performance of virtual BNG
Test system
emulates
DSLAMs & PPPoE
clients
Test system
emulates Edge
and core routers
27
Virtual test appliances emulate realistic video and web clients and servers generating stateful L4-7 traffic
In above example, the VNFs under test are virtualized Firewall, Load Balancer and CE router running as a service chain inside a standard server
VNF Functional & Performance Testing 2. Using virtual test devices to validate performance Service Chain
vLoad
Balancer Emulated
Video/Web Clients
Emulated
Video/Web Server
Service chain
vFirewall vCE
28
The following metrics are measured/verified by the test appliances for service chains that include virtual appliances such as Firewalls, IDS/IPS, DPI, Load Balancers, Traffic Classifiers, WAN Accelerators and CE devices
• Sustained packet forwarding rate
• Connection establishment rate & transactions per second
• Total number of connections
• Round trip time and goodput
• Denial of service handling & packet loss
• Service chain scale (and interference)
• Packet leakage across service chains
• Time between VM instantiation and first available packet
Service chain validation (e.g. vFirewall, vLB & vCE)
29
The following methodology is used to ensure portability of VNFs and stability of NFV environment
• Virtual Test Appliance is connected to Service Chain as
• “x” service chains are created. The test appliances ensure that adding the
“x + 1”th service chain does not degrade the performance of the first “x”
service chains more than expected levels
• Tests are repeated for a number of different hypervisors and vSwitches and
the test appliances verify that the VNF performance is consistent across
different hypervisors
Service Chain Stability, Portability and
Scalability
30
Testing reliability and availability of VNFs
LAG 1
Server A Server B
BFD
BFD
Virtual test devices form connections with primary and backup VNFs over a LAG
High frequency BFD running between the test devices and VNFs constantly monitor the connection liveliness
LAG 2
BFD
31
The following methodology is used to ensure availability of VNFs
• Initially Port 1 is active on Both LAGs 1 and 2
• High frequency BFD monitors connection liveness
• VM Migration is initiated from Server A to Server B
• Port 2 becomes active and port 1 becomes standby on both LAGs
• Number of packets lost in forward direction is TX packets on Stream ID 1 on
LAG 1 minus RX packets on Stream ID 1 on LAG 2
• Number of packets lost in reverse direction is TX packets on Stream ID 2 on
LAG 2 minus RX packets on Stream ID 2 on LAG 1
• VM migration time is the greater of [Time of arrival of first packet on Port 2
of LAG 2 – Time of arrival of last packet on Port 1 of LAG 2] and [Time of
arrival of first packet on Port 2 of LAG 1 – Time of arrival of last packet on
Port 1 of LAG 1]
Reliability & availability of VMs (VM Migration)
32
Monitoring
probe
Monitoring
probe
On on-demand basis, insert virtual monitoring probes
in the service chain, for active or passive monitoring
Performance monitoring
interface to OSS/BSS
NFV Service Assurance
vLoad
Balancer vFirewall vCE
vMonitoring
Probe
vMonitoring
Probe
33
The following methodology is used to perform active and passive monitoring of NFV environments
• A combination of virtual and physical monitoring probes are used
• Probes provide information to OSS/BSS systems
• Virtual monitoring probes are inserted on an on-demand basis at various
points in the service chain to test a subset of or all of the functions of a
service chain
• For active monitoring, the virtual probes originate and terminate packets;
for passive monitoring, the virtual probes just tap in to the service chain
Active and passive monitoring of NFV
environments
34
Traditional Router Architecture
Router
Router
Packet Forwarding Hardware
Network OS
App App App
Packet Forwarding Hardware
Network OS
App App App
Custom Designed Specialized Hardware
Based on ASIC, FPGA or Network
Processors
Proprietary Network Operating System
e.g. Cisco IOS or JUNOS
Embedded Software - Routing Protocols,
Routing Data Bases, SPF Algorithms,
Firewall Functionality etc.
35
Conventional Routing The Control and Data Plane
Router
Router
Packet Forwarding
Hardware
Network OS
App App App
Router
Router
Packet Forwarding
Hardware
Network OS
App App App
Router
Router
Packet Forwarding
Hardware
Network OS
App App App
Router
Packet Forwarding
Hardware
Network OS
App App App
Router
Packet Forwarding
Hardware
Network OS
App App App
Router
Router
Packet Forwarding
Hardware
Network OS
App App App
Routers ‘talk’ to one another
via routing protocols to
discover neighbours and
topology
Each Router builds a database of
the network topology which it
uses to determine how to switch
data packets
36
For
• Established - Tried and Tested
• Bomb Proof!
Against
• Inflexible – Changes require weeks to implement
• Expensive - Every node requires compute resources
• Proprietary - Every vendor implements routing algorithms in their own way
• Hard to Maintain - Every node must be visited for software maintenance
• Vulnerable to control-plane attack
Conventional Routing Pros and Cons
37
Controller
SDN – What Changes?
Firewall
Network OS
Router
Network OS
Load Balancer
Network OS
Router
Network OS
Router
Network OS
Router
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Network OS
38
Controller
SDN and OpenFlow
Switches built from cheap merchant (off-the-shelf) silicon
OpenFlow is a component of SDN
Applications perform path calculations (like SPF today)
Much greater flexibility to add new functionality (e.g. SJ-BPF)
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
Packet Forwarding
Hardware
SDN Controller
(South-bound interface)
40
Simplified Provisioning of Complex Topology SDN will enable dynamic
provisioning across network layers
Data Centre A
Data Centre B App App App
SDN Controller