Performance Testing of Roaming, VoWLAN & Wifi … 2006...Acme corporation is holding a training seminar for their 60+ employees. As the trainees enter the conference they are asked

The Leader in Wireless LAN Testing

Performance Testing of Roaming, VoWLAN & Wifi-Mesh on Highly Scalable WLAN Systems

Veriwave Confidential and Proprietary information2

VeriWave Mission Statement

Enable the creation of high performance WLAN systems for mission critical enterprise applications by• Providing WLAN equipment manufacturers with the tools necessary

to accurately analyze their products thus improving performance,interoperability, and profitability

• Supplying service providers and enterprise users with the tools necessary to make the right choice when selecting WLAN equipment for deployment in their networks


Agenda

WLAN Market & Trends

Challenges in WLAN deployments

Metrics & Measurements

How to test and measure using Veriwave products

Importance of Performance Testing

Benefits of Veriwave Solutions

Veriwave Roadmap


Market, Big and Expanding

802.11 is the fastest growing communications protocol ever deployed

Enterprise WLAN to more than double by 2009, Up from $1.1B in 2005 to $3.5B in 2009 (Dell’Oro Group)

Service Provider Access Devices to grow 5x, from 13 million units in 2005 to 67 million units in 2009 (Dell ‘Oro Group)

The market for wireless test equipment will be larger than wired Ethernet by 2010 due to a broader applications space and more classes of products, (i.e. Wifi Mesh, mobile terminals, industrial & retail systems)

Consumer electronics and Voice-over-IP telephones will be two of the biggest application opportunities for WLAN (or Wi-Fi) IC sales, accounting for around half of an estimated $2.8 billion total market size in 2008

Nearly $400 million will be spent on WLAN performance test equipment in 2006, over $700 million in 2009

Organizations of all sizes across the world will increasingly turn to the deployment of wireless LANs over the next two years. Growing availability of wireless VoIP handsets and voice-enabling wireless infrastructure are the prime drivers behind the expected increase from 10% of use to 31% by 2007


Challenges in WLAN deployments

Biggest challenges in WLAN systems today are in the following areas• Support sufficient bandwidth performance in APs and WLAN controllers to

accommodate ever expanding capacity requirements• Reduce the Roaming delay contributed by the WLAN infrastructure to the

overall roam time of mobile clients• Provide for proper QoS performance in order to support a good end-user

experience for delay-sensitive applications• Enable support for bigger and larger WLAN deployments that can provide

nominal performance• Improve Service Level Agreements for broadband services enabled by

WiFi Mesh networks


Bandwidth Scenario

Acme corporation is holding a training seminar for their 60+ employees. As the trainees enter the conference they are asked to turn on their laptops and begin downloading the necessary material. Consequently the material that is only supposed to take a few minutes to download using the high speed WiFi internet equipped in the classroom is taking much longer. The seminar is now running late.

What is the problem?


Bandwidth Problem

Several of the Acme trainees are logging onto the WLAN network through near by access points, all within a few seconds of one another which is slowing down the connection time

In addition multiple users are trying to access fairly large training material possibly affecting the download speed for everyone


Factors Affecting Bandwidth

The slow connectivity and download speed at the ACME training seminar may be due to• A lack of network’s capability to handle advanced

authentication schemes efficiently• Lower bandwidth support on the APs• Throughput performance degraded by data encryption

schemes and high number of users

In order for 60+ users to be able to connect simultaneously and download large files the WLAN should be capable of connecting all users to the network in a few seconds and provide a nominal bandwidth (1 to 2Mbps) per user


Metrics and Measurements of Bandwidth

Throughput, packet loss, and latency are the key metrics that embody the performance of WLAN networks

We should test throughput, packet loss, and latency of the wireless network so that there is no problem of bandwidth and we make sure that every client’s performance need is met


Benchmarking – IEEE 802.11.2 Standard

Is there a Benchmarking standard to test ?

Yes, there is FINALLY a BENCHMARKING standard. - The IEEE 802.11.2 STANDARD for BENCHMARK

TESTING WIRELESS SWITCHES, WIRELESS ACCESS POINTS, WIRELESS LAN CONTROLLERS

VeriWave has a LEADING position on the IEEE 802.11.2 standard


IEEE – 802.11.2 Benchmarking Standard

IEEE 802.11T, the Test task group, has incorporated throughput, latency, packet loss, maximum client capacity and maximum forwarding rate and related measurement methodologies in the IEEE 802.11.2 specification

VeriWave’s IEEE 802.11.2 Standard WLAN Benchmarking Test offers the complete set of Layer 2 and Layer 3 performance metrics (throughput, maximum forwarding rate, and packet latency) defined by the IEEE 802.11.2 WLAN benchmarking methodology for testing Wireless Access Points (APs), WLAN controllers and WLAN switches


VeriWave - Relationship To IEEE Standard

Veriwave’s Chief Technology Officer (CTO) Dr. Tom Alexander is on the committee of this standard

All the Benchmarking Tests from VeriWave follow theIEEE 802.11.2 standard

WaveApps, the Test Application from VeriWave has all the Benchmarking tests included in it to test as per the IEEE 802.11.2 Standard


What products should test by IEEE 802.11.2 standard ?

Wireless Products made by all equipment manufacturers should test as per the IEEE 802.11.2standard

• Wireless Access Points (APs)

• Wireless Switches

• Wireless Controllers


Who tests currently using this standard?

Major 802.11 wireless equipment manufacturersin North America (U.S.A and Canada) are testing their wireless products such as Wireless Access Points, Wireless LAN Switches, Wireless Controllers using IEEE 802.11.2 Standard using the VeriWave WaveApps Application for example :

CiscoArubaTrapezeLVL 7Foundry NetworksTranzeoTropos


Industry First 802.11 WLAN Performance Benchmark

Test scale• 1 – 100’s of APs• Multiple WLAN controllers or

switches or Wireless Access Points

Determine key metrics of your system:• Throughput• Maximum Client Capacity• Maximum Forwarding Rate• Packet Loss• Latency


Industry First 802.11 WLAN Performance Benchmark

Fully automated

Granular control of key parameters: number of emulated clients, security type, frame type, etc.

Test Results• Automatically generated test reports to convey

results to management or to your customers• Test results automatically compared versus

theoretical maximum achievable metrics• Real-time status of client connections• Real-time graphs of metrics being measured


WaveApps – IEEE 802.11.2 Benchmarks

Max Client Capacity• Determines the maximum no. of client connections that the

DUT can support at user specified intended load and packet loss % for different frame sizes and security types using goal seeking algorithm

Throughput• Determines the maximum forwarding rate of the DUT at user

specified packet loss levels for different frame sizes and security types using goal seeking algorithm

Maximum Forwarding Rate • Determines the maximum forwarding rate of the DUT for

different frame sizes and security types using goal seeking algorithm

Veriwave® Confidential and Proprietary information


WaveApps – IEEE 802.11.2 Benchmarks

Packet Loss• Determines the rate of loss of packets forwarded by the

DUT for different traffic load levels, frame sizes and security types

Latency• Determines the minimum, maximum and average

latencies experienced by packets forwarded by the DUT for different traffic load levels, frame sizes and security types



Mobility Scenario

LA Memorial Hospital, has made a strategic decision to equip the entire hospital with wireless infrastructure in order to significantly improve communication between medical teams who are unable to use cell phones inside the hospital, but require a quick response time.

Dr. Joe has just received a call on his WiFi phone that one of his patients on the other side of the hospital has begun to have severe seizures. While walking down the Hospital corridor Joe continues to drop calls, making it difficult to find out what medication was given to his patient that may have caused these seizures. Dr. Joe continues calling his nurse but is having no luck in reaching her due to all of his dropped calls. He is now forced to leave his patient suffering while he makes his way to the other end of the hospital.



Mobility Problem

As Dr. Joe walks through the hospital corridors he moves from the coverage area of one access point to another. Dr. Joe’s continuously dropped calls can be attributed to his WiFi phone being unable to roam seamlessly across access points


Factors Affecting Mobility

The dropped calls from Dr Joe’s WiFi Phone may be attributed to• The inability of the WiFi phone to quickly decide to roam as it is moving out

of the coverage area of the current AP into the coverage area of the new AP

• The WLAN network (APs + WLAN switch) not being able to rapidly authenticate and initiate incoming connections

In order for a VoIP call to be mobile and avoid being dropped, the roam delay must be under 100 ms


Metrics and Measurements of Mobility

Roaming delay is the amount of elapsed time between the last data packet received on the previous AP to the first data packet received on the new AP

Roaming delay along with min/max/avg packet latency and packet loss properties of the WLAN network equipment reflect on its roaming performance, or in other words the WLANs ability to provide uninterrupted end-to-end service to its mobile clients

VeriWave’s WLAN Roaming Test offers a complete set of automated tests to analyze the ability of infrastructure-class WLAN networks and network components to handle hundreds of mobile clients roaming between Access Points

The test provides precise insight into roaming delay, call quality during roaming, and network's overall capacity to handle mobile clients


The ‘walkabout’ method of roaming test

Is this repeatable, scalable and efficient?

The Traditional Approach to test mobility

APAP

AP

Cart full of laptops, moved from AP to AP

$%@*#!!Roam

Between APs


The ‘attenuator’ method of roaming test

Sounds simple – twiddle the attenuators and watch ‘em roamBut there are some problems …

A Somewhat Better Way


What’s Wrong With The Attenuator Approach?

Doesn’t scale• Real clients (laptops, PCs, NICs) consume lots of space, power, cost• Test becomes impractical beyond about 10-20 clients

Can’t separate client problems from AP/switch issues• In many cases the client contributes most of the roaming delay• If a connection is dropped, was it the AP’s fault, or the client’s?

Hard to repeat and control• Real clients are difficult to set up, coordinate, keep alive• Clients are not designed to serve as test equipment

Notoriously variable in performance


Scalable Roaming Testing – The VeriWave Way

VeriWave system emulates client mobility• Up to 500 mobile clients, all roaming

independently and concurrently• Controller roaming load can be varied from

1 roam/hour to 15 roams/second

VeriWave system generates data• Each client continuously sources and

sinks traffic while roaming• Controller data load can be varied from

kilobits/second to gigabits/second

VeriWave system makes all roaming measurements• End-to-end measurements on actual traffic• Min/max/average delay, packet loss, etc• Measure over millions of roams

Daisychainedchassis


A Look Under The Hood

What happens when a simulated client roams?• Each WT-90 port supports a separate instance of each client• Roaming essentially deactivates one instance and activates another

Realistic but repeatable roaming process• Controlled power & BER ramp up/down, probes before re-association, etc


Correlating to End-User Experience

Current methods just measure re-association time• Merely the time for clients to reconnect to APs at Layer 2

End-users actually care about connection re-establishment time• This is when data can flow end-to-end and in both directions

Laptop screens update & VoIP audio resumes at this point

Need to measure end-to-end service restoration• This can take 1000’s of msec longer than re-association

Depends on controller loading, traffic level, etc.• Requires end-to-end tracking and end-to-end timestamps

This is what VeriWave equipment provides


Controllability And Repeatability

Precise control of every roam by every client• User sets desired roaming profile; VeriWave equipment manages

roaming events and data traffic to exactly fit the profile• Roam events occur in the same way at the same time, every time

Accurate timestamps and end-to-end measurements• Unique signature with nanosecond timestamps in every data packet• VeriWave equipment tracks packets end-to-end for every roam

event, and reports what happened

Test results show high absolute repeatability• Not average …• Not probabilistic …• Just plain repeatable, test run after test run

That’s what getting the real clients out of the picture does for you


Roaming Delay Test

Determines the minimum, maximum and average roaming delays, min/max/avg packet latency and packet loss experienced by actively transmitting and receiving clients when roaming from one AP to another over multiple roam sessions

Reports per client• Frames transmitted and received• Min/Max/Avg Roaming delay• Average Packet Loss per roam

Reports per BSS• Min/Max/Avg Roaming delay• Average Packet loss per roam• Client arrival rate• No. of failed roams• No. of disconnections

Controllable parameters• Intended load (in packets per sec)• SSID• Number of ports (APs)• Automatic settings

Number of clients per port (AP)Client roam interval

• Roam sequence and Dwell time• Variation of client Tx power level• Security type used by clients

Fast Roaming (Pre-authentication, Key caching)

• MAC and IP addresses


Mobility Test Report


Companies using WaveApps to test Roaming

All major US equipment manufacturers using the Roaming Application in WaveApps to test the roaming on their Wireless Access Points. Wireless Switches and Wireless Controller Cards

CiscoArubaTrapezeLVL7TroposFoundry NetworksTranzeo


VoWLAN Issues

CAN YOU HEAR ME NOW?

John is the top sales rep, for afortune 500 company, and is just about to close his biggest deal yet, however in order to do so he must first speak to his sales manager.

Using his WiFi phone John calls his sales manager but is havingtrouble hearing him due to therepeated break-up of the conversation. Is the poor call quality going to cost John this deal, or even possibly his job?



Factors Affecting VoWLAN

The low quality of John’s VoIP (VoWLAN) conversation may be attributed to• VoIP call experiencing high delays• Bad QoS (Quality of Service) performance of the WLAN network

VoIP is a delay sensitive traffic which requires a low latency network and therefore should receive the highest priority on the network.

Typically, voice calls with greater than a R-value of 80 or MOS score of 4 will have reasonable call quality


Metrics and Measurements of VoWLAN

R-value and MOS score measurements are industry standard to quantify the call quality and the VoIP performance of the system under test

VeriWave VoIP QoS application offers an automated approach to determine a network’s VoIP call• Service level capacity (for a given Service Level Agreement)• Service level assurance (in conjunction with background traffic)• Call quality metrics (R Value, MOS scores) per call • Service level differentiation (in the presence of multiple VoIP calls, video

and data traffic)


VoIP QoS Service Assurance Test Suite -Answers to real-world questions

Is your infrastructure equipment capable of

correctly categorizing and differentiating traffic as per

802.11e / WMM?

Is your WLAN infrastructure equipment

properly designed to deliver the promised SLA?

What is the capacity of the WLAN to support voice calls when employing 802.11e / WMM QoS?

Get a definitive measure of quality of voice traffic carried

across your WLAN



VeriWave VoIP / QoS Tests - Overview

Each VeriWave blade supports up to 500 independent, statefull, VoIP clients (handsets)VeriWave statefull clients support• RTP traffic• 802.11e Access Category (AC) properties and User Priorities• G.711, G.729, and G.723 codec types • Vocera badges



VoIP QoS Service Assurance Test Suite

Service Level Capacity Test

Determine the capacity of the WLAN infrastructure to support specified high priority traffic (VoIP) under a given Service Level

Features• Measure the maximum number of voice calls supported for a given SLA

(voice call R-value / MOS score) in the presence of a given low priority background and best-effort traffic load

• Number of high priority traffic calls (voice) or streams (video) is increased until SLA is not met

• See the effect of varying SLAs on maximum number of voice calls

What is the capacity of the WLAN to support voice calls when employing 802.11e / WMM QoS?




Service Level Assurance Test

Determine the Service Level Agreement (SLA) break-point of WLAN infrastructure equipment handling a constant VoIP call load in the presence of changing Best effort traffic load

Features• Measure achieved Service Level of high priority (voice and video) traffic as

background and best-effort (low priority) traffic is ramped up• Observe change in real-time voice quality (R-Value / MOS) as background

traffic load is increased to the point of system over-subscription• Validate complex usage scenarios through changing traffic patterns

Is your WLAN infrastructure equipment

properly designed to deliver the promised SLA?




Determine the performance, capacity and call quality metrics offered by infrastructure WLAN equipment in the presence of multiple VoIP calls and data traffic

Features• Measure overall, real-time, and over-time R-

Value (MOS score) for individual VoIP calls• Create complex scaling scenarios with multiple

QoS priority types and analyze impact on the WLAN AP/Switch performance

Number of sustained and dropped callsAggregate VoIP goodput and “R-Value” for all VoIP calls

• VoIP clients utilize RTP traffic & QoS power-save protocol

• Support for one to hundreds of independent statefull VeriWave VoIP clients

Service Differentiation Test Is your infrastructure equipment capable of

correctly categorizing and differentiating traffic as per

802.11e / WMM?



VoIP QoS Service Assurance Test Suite – Usage Example

Determine the maximum number of Vocera badge calls that a wireless LAN infrastructure can support without any calls being dropped.

Features• Precisely quantify the number of Vocera calls placed, the number of successful calls,

the number of failed calls and the min/max/avg latency and jitter of each call.• VeriWave’s statefull Vocera clients emulate all aspects of the Vocera call, including

badge to badge calls and badge to call server connectivity.• Find the maximum number of Vocera badges the wireless infrastructure can support,

WITHOUT THE COMPLEXITY AND COST of setting up actual Vocera badges and call servers

Vocera® Call Capacity TestGet the definitive answer -

how many Vocera badges can my WLAN support and at

what quality?

Vocera® is a registered trademark of Vocera Communications, Inc.



R-Value vs. Number of Calls


Forwarding Rate vs. Number of Calls


Packet Loss vs. Number of Calls per AP


Average Latency vs. Number of Calls per AP


Scalability Scenario

This University has over 75,000 students 1,000 professors, and 15,000 employees. Large Universities such as Arizona are moving more and more towards online education tools. This requires the Universities WLAN network to support the vast number of students, faculty, and employees on and around the campus as well as deliver a certain level of guaranteed performance.

With finals just around the corner, many students at the University have been logging onto the campus network to check online notes, chat with other students, view past lectures online through streaming video, and making VoIP calls to other students on campus, however most of these students have been experiencing issues with the above activities. Along with the students, the many faculty members and employees who also use the University network are having the same problems while researching online, conducting online classes, completing accounting activities, and/or other administrative work.



Scalability Problem

This University’s large number of users have overloaded the WLAN consisting of hundreds of APs and several WLAN controllers

The result is a congested network which is causing the throughput of data, video, and voice applications to degrade


Factors Affecting Scalability

The problems experienced by the University community on their WLAN may be due to• High volume of users• Sudden increase in traffic load• Multiple traffic types in play (data, voice, video)• Size of the WLAN network (hundreds of APs and several WLAN controllers)

The campus WLAN network must be designed to provide capacity forthousands of users running multiple applications, and to support a nominal level of performance for the data, voice, and video applications used by the University community


Metrics and Measurements of Scalability

Scalability related metrics such as throughput, packet loss, latency, quality of service, and client capacity properties of the WLAN network equipment characterize it’s ability to scale performance for a large campus network

Veriwave’s WaveTest™, a Wireless LAN and Ethernet performance test platform, is capable of stress testing a complete WLAN network, including Access Points (APs) and WLAN Switches/ Controllers

WaveTest™ provides simultaneous generation of traffic from thousands of 802.11 and Ethernet clients, with line speed analysis of the behavior of the system being tested

Unprecedented network scale can be achieved by daisy-chaining up to six WaveTest™ WT90 chassis for a total of 27,000 emulated clients


WiFi Mesh Scenario

Portal AP Mesh

AP

Wired LAN

Wireless

Backhaul

Gateway

Wireless Uplinks

Barstow, offers wireless broadband to one-third of its homes and businesses over its Wi-Fi Mesh Network. The city of Barstow charges residents $15.99 a month with download speeds of between 750Kbps and 1.2Mbps.

About 75 percent of Barstow residents received good quality internet connection, while for the other 25 percent of the residents internet connection quality was poor, and struggled to deliver the SLA.



WiFi Mesh Problem

The 25% of Barstow’s wireless broadband subscribers who experienced poor internet quality either lived on the outer perimeter of the WiFi Mesh coverage area, or encountered interference issues


Factors Affecting WiFi Mesh

The problems with poor internet connection quality may be due to• Lack of adequate signal strength reaching the outer perimeter of residents• Performance degradation (reduced throughput, and higher packet loss)

due to increased number of hops between the gateway and the outer mesh node

• High packet rate caused by RF interference from non-subscribers with home/personal WLAN or other wireless systems

WiFi mesh network, must be designed to deliver high performance (nominal throughput, preserve QoS) in the presence of increased number of hops, and must be robust to accommodate impairments caused by interfering devices


Metrics and Measurements of WiFi Mesh

Throughput per hop, latency per hop, aggregate throughput, aggregate latency, client capacity, and failover recovery (roaming) are properties that characterize the performance of a WiFi mesh network

VeriWave’s WiFi Mesh test suite is a comprehensive offering that covers the measurements such as

Throughput between the Ethernet backhaul and each hopLatency between the Ethernet backhaul and each hop Client capacity for a given Service Level Agreement Recovery time for a WiFi mesh link failure Roaming capacity of the WiFi mesh in the presence of mobile clients


Mesh Performance Benchmarks

Throughput per Hop• Basic traffic capacity metric – measures the throughput between the Ethernet backhaul and

each hopLatency per Hop

• Quantifies ability to support delay-sensitive applications like VoIP – measures the latency between the Ethernet backhaul and each hop

• Conducted at low intended loads as well as at the maximum throughputAggregate Throughput and Aggregate Latency of all Hops

• Does the sum of the parts equal the whole? How well does the network handle traffic on multiple ports simultaneously?

Client Capacity• Measure how many users can use the mesh network for a given offered load and an given

Service Level AgreementFailover recovery

• Determines ability to self-heal and recover from outages – measures recovery time when a hop is taken off-line

• Recovery Time = Packet Loss / n [n=offered load in packets/sec or bits/sec]

Roaming• Measures ability to support a large number of mobile clients• Roaming delay and failed roams, measured as a function of number of clients and traffic load


VeriWave Mesh Tests - Overview

Simple setup

Scalable

Repeatable

Flexible

Enables full range of tests

Extensive automation support


Capabilities

Scalable test setups for truly large-scale mesh performance testing• Test from 3 to 48 mesh APs in a single setup

Pre-verify an entire mesh deployment• Test with 1 to 24,000 clients

Ensure that client capacity targets are met

Wide range of performance tests in a single test setup• Multi-hop traffic capacity measurements

Throughput and forwarding rate; load mesh to full bandwidth• Multi-hop VoIP capacity & QoS

Latency and packet loss; verify that QoS targets for VoWLAN are met• Roaming measurements: roaming delay, failed roams, packet loss

Roam up to 500 clients between any combination of mesh APs• Availability & uptime

Verify self-healing time, overall uptime

Repeatable, automatable, easy to use• GUI, script or fully automated operational modes• Simple RF setup avoids complicated plumbing• Eliminates client control and RF variability issues


Fixed attenuators isolate mesh BSSs60 dB attenuation between clients in different BSSs

Mesh Test Setup - uplink and backhaul on same band

Attenuators for BSS isolation

Different WaveTest Clients Experience

Different Numbers of Hops


2:1 RF power dividers (splitters) to segregate trafficBoth radios on DUT operate simultaneously

WaveTest feeds client uplink radioIntra-mesh traffic flows separately over backhaul radio

Mesh Test Setup – uplink and backhaul on different bands

Splitters Isolate Mesh BSSs


Rate vs. Path Loss – predicting Rate vs. Range

Methodology:• Traffic is generated by Client(s) on WaveBlade #2• Effects of range emulated by Client on WaveBlade #1

Increasing Frame Error RatioReducing transmit power

• Forwarding rate measured for Client on WaveBlade #1Advantages• WaveTest offers precise control over Frame Error Rate (FER) and transmit power, allowing for

controlled step down of forwarding rate by AP under test• Results are calibrated to actual test environment including cable losses and other fixed path loss

WLAN Switch

Measure:Forwarding Rate

DUT

WaveBlade #2

WaveBlade #1

Frame Error Ratio

Control

Transmit power

Ethernet

RF


Rate vs. Path Loss – example results

Forwardingrate

Path loss [dB]


Summary

WLAN networks require scalable testing

WaveTest offers a powerful solution

WLAN testing can be greatly simplified and scaled

Automation and executive reports speed up test process


Importance of Performance Testing

Loading and scalability of infrastructure devices

Technology and application convergences

Determines robustness of the design and implementation

Determines the efficiency of the solution and consequently the quality the end-user can expect

Determines the systems ability to support mission critical applications and its ability to always stay on


Test as per IEEE 802.11.2 Standard

All wireless equipment manufacturers should test as per IEEE 802.11.2 Standard

VeriWave has a leading position on the IEEE 802.11.2 Standard

VeriWave’s Chief Technology Officer (CTO) Dr. Tom Alexander is on the committee of this standard

All major U.S companies like Cisco, Trapeze, Aruba, LVL 7, Tranzeo, Tropos already test as per the IEEE 802.11.2 Standard


Test Using WaveApps – Benchmarking Application

All equipment manufacturers of Wireless Access Points, Wireless Switches, Wireless Controllers should test using WaveApps

WaveApps has all Benchmarking tests as per the IEEE 802.11.2 standard

WaveApps also helps you also test Roaming, VoIP, QoS and Mesh


VeriWave’s Technological Focus

Statefull behavior – real 802.11 clients

Loading and scalability of infrastructure devices

Mobility - large scale and repeatable roaming test

Technology & apps convergence• Wireless and wired • Voice• QoS – prioritization, admission control, bandwidth utilization• Mesh networks• Security


Benefits – efficiency gains & cost reductions

Increase test coverage

Identify bugs early in development & QA cycle

Automate – create a standardized regression test environment

Optimize the efficiency of test engineers

competitiveadvantage

customersatisfaction

shorter timeto market


Timeline

WaveTest 90 Traffic Generator / Analyzer• The industry’s first scalable WLAN test solution - designed to test

enterprise class WLAN systems

WaveApps - 802.11 WLAN Performance Benchmark Test• The first automated, repeatable test solution for measuring key

performance metrics of large-scale WLAN systems to 802.11 standards

WaveApps - WLAN Roaming Test • The first test solution to measure mobility and roaming metrics of a

WLAN at an enterprise scale

WaveApps – VoIP / QoS Service Assurance Test Suite• Measure and verify quality and service levels of voice and video in a

triple-play WLAN

12/2005 Introduced to beta customers

2/2006 General availability

4/2006 Available

5/2006 Available

7/2006 Available


Roadmap Q3-Q4/2006

Rate vs Range Test Now

VoIP over WLAN and QoS testing• VoIP traffic generation – RTP traffic Now• VoIP QoS (R-Value, MOS) Service Assurance Test Suite

Service Differentiation Test NowService Level Assurance Test NowService Level Capacity Test Now

Enhanced functionality of VeriWave’s stateful clients• Power save and QoS Power Save (uAPSD) Aug 06• Client physical contention (collision) emulation Now


Roadmap Q3-Q4/2006

Mesh Tests Now• Per hop throughput and latency• Aggregate throughput and latency• Client capacity• Failover recovery

VoIP QoS Service Assurance Test Suite• Roaming VoIP clients Sep 06• 802.11e Admission Control Sep 06


Stateful clients – Layer 2 through 7Available Q4 2006

WaveTest generates stateful, scalable, mobile WLAN clients•Up to 500 per port, >50,000 per setup

•Independent MAC, L3 and 802.1X protocol state machines per client

•Independent test traffic flows per client

•Full mobility (roaming) support

Clients & traffic flows precisely controlled for repeatable tests•Air-rate / wire-rate transmit & receive

•Powerful traffic scheduler for realistic contention and QoS scenarios

L2- L3

L4-L7• Stateful TCP/IP

• Traffic flows- ICMP, Trace route, HTTP, TLS, RTSP, P2P, Real Media

• WiMix – Real-world traffic scenarios

Client Experts – L2 through L7


Roadmap Q1/2007

802.11n• WaveBlade TGA supporting 500 clients and Air-rate traffic• Supports different configurations 2x2, 2x3, 4x4 etc• IEEE TGn channel models• Bi-directional• Same form factor as current Wifi WaveBlade• Backwards compatible


802.11n

The scope of TGn's objective is to define modifications to the Physical Layer and Medium Access Control layer (PHY/MAC) that deliver a minimum of 100 Mbps throughput at the MAC SAP.Increasing the physical transfer rate of wireless systems by using multiple antenna systems for both the transmitter and the receiver. This technology is referred to as multiple-input multiple-output (MIMO), or smart antenna systems.MIMO technology offers the ability to coherently resolve information from multiple signal paths using spatially separated receive antennas. Possible use of wider (40MHz) channels to achieve higher data rates.

Use more complex modulation and coding techniques to improve spectral efficiency and hence increase the data rates.MAC layer improvements such as aggregating multiple MAC Protocol Data Units (MPDUs) into single PHY Protocol Data Units (PPDUs). Acknowledging multiple MPDUs with a single block acknowledgement (Block ACK) in response to a block acknowledgement request (BAR)


Leadership – customers, partners and labs


Backup


VeriWave WT90

WT90 / WT20• Scalable traffic generator/analyzer

WaveTest generates stateful, scalable, mobile WLAN clients• Up to 500 per port, >50,000 per setup• Independent MAC, L3 and 802.1X protocol

state machines per client• Independent test traffic flows per client• Full mobility (roaming) support

Clients & traffic flows precisely controlled for repeatable tests• Air-rate / wire-rate transmit & receive• Powerful traffic scheduler for realistic

contention and QoS scenarios

RF effects are controlled• Isolate the DUT from interference• Emulate effects of distance, contention


WaveTest – Chassis

WaveTest 9019” rack mountNine WaveBlades per chassisCombine wireless (802.11) & wireline(802.3) WaveBlades

WaveTest 20PortableTwo WaveBlades per chassisCombine wireless (802.11) & wireline(802.3) WaveBlades



WaveTest 90/20 – a comprehensive test solution

RF isolation enclosure• Supplied by VeriWave, or• Use off-the-shelf from other vendors

RF isolation enclosures from VeriWave:• Fit up to 2 per 19” unit (height = 11”)• 80dB isolation between enclosures• Each enclosure supports:

RF SMA connector10/100 RJ-45 connector (PoE)DC power posts



Security

WEP-40 and WEP-104TKIP (WPA encryption method)CCMP (WPA2 encryption method)

Encryption

PEAP / MSCHAPv2 TLS v1.0 with per-client certificate processingLEAP / EAP-FASTTTLS / GTC

EAP

802.1X EAPOLPre-Shared Key (PSK)Open-system

Authentication

802.1x authentication and 802.11i encryption support for mobile client emulationPer client control of both authentication and encryptionUp to 4 Group Temporal Keys (GTKs) per BSSID



Traffic Analysis

Per port (channel) counters• Complete set of layer 2 counters

Examples: Number of MAC data frames, management frames, retry counts, probe requests, authentication requests, etc.

• Complete set of layer 3 and 4 countersExamples: Number of IP, UDP, ICMP, TCP, ARP, DHCP frames

• Complete set of layer 2, 3, and 4 rate counters• Number of associated clients, number of authenticated clients, number of

active flows

Per client, per flow, or per flow-group counters• Frames sent/received, bytes sent/received, Sequencing (out-of-order),

payload integrity, etc.• Latency histograms

Capture and log• Review log of all captured traffic - up to 256MB of data• PCAP export for post-test analysis (Ethereal, Sniffer, AeroPeek, etc.)



A Comprehensive Test Automation Environment

Create a complete regression test setup to run hundreds of test cases, unattended and with minimal effort

Features• Configures and controls VeriWave’s WT90/WT20• Configures System Under Test (SUT) - VeriWave tailors code to control

SUT• Schedules and launches tests• Creates and saves test results and reports

• Support for entire set of VeriWave automated tests (WaveApps and scripts)• OS’s supported

Linux (2.6 kernel)Windows XP


Client (STA) Collision Emulation

Client Collisions result in:Corrupted FCS packets with a valid header and a valid payloadMalformed packetsUser controlled parameter to set collision probability

STA 1

TrafficScheduler MAC & PHYSTA 2

STA 3

Observed on “the air”

Collision frames

Regular traffic



Traffic Scheduling

Fully interleaved traffic flows• Emulates real mobile clients contending for the shared medium• scheduler dynamically adjusts offered load to approximate intended loadLoad resolution 0.5%

1 2 1413 11 132 1

1 sec

1 4

1 sec

no ACK received for this

data frame;re-transmission

is required

no ACK received for this

data frame;re-transmission

is required

re-transmission of data frame

3 1

re-transmission of data frame

1 2 12413 1124 132 1241 4

1 sec

client 2 startsre-authentication

client 2de-authenticated

by DUT

client 2Completes

re-authentication

1 sec

client 1 - 40%client 2 - 30%client 2 control trafficclient 3 - 10%client 4 - 20%

12

34

client 1 - 40%client 2 - 10%client 3 - 10%client 4 - 10%

1234



Ease of Use (cont.)

Create your own automation scripts

Start with a canned application and modify script

Create starting configuration in WaveManager; save configuration as script and customize via Tcl, Perl, or Python

All use-models utilize same library functions for easeof portability between use-modes

TCL API Python API Perl APIWaveManager

Roaming Benchmarking VoIP

Custom applications

Network

Custom Traffic Generation / Analysis Hardware

802.11 Radio / 802.3 Phy

Configuration Upload/Download Statistics Processor

VeriWave Command Library (VCL)

ARP request/response, DHCP

discover

Security Settings

ports, clients, flows, etc.802.11 & 802.1x

Authentication and Encryption Statistics Collection

Configuration/Control Processor Authentication/Statistics Processor

WaveBlade

usergenerated

scripts

ARP, DHCP, 802.11 state

machine


Documents

Performance Testing of Roaming, VoWLAN & Wifi … 2006...Acme corporation is holding a training seminar for their 60+ employees. As the trainees enter the conference they are asked