cFrame framework slides

cFrame Presentation

Introduction to the Distributed Mobile Network Performance Testing Automation Tool

1

2

Who We Are

3

What is cFrame?

“cFrame provides an open automated platform for mobile network performance testing in both real and RF channel-simulated environments”

4

Current RF Testing LandscapeDescribe current state of RF Optimization and Testing: how are things being done now? Basically you are going to set the context of current methodology and later show how cFRAME dramatically improves on this.

State the basics but remember you want to define this on your own terms so that later when you present your solution it really shines and stands out.

The goal with the above is for Rubedo to control the conversation.

5

Key RF Performance Indicators

Throughput (main KPI) Latency (delay in network response) Attach/Detach success rate (make/hang up call) Sensitivity (related to energy efficiency) Anything else worth mentioning???

6

Real vs. Simulated Environment Real environment

Unpredictable Time consuming Requires more planning (weather, resources)

Simulated environment Stay in the lab (preferred by engineers) Don’t need over-the-air RF transmit license Don’t mess up with live existing networks Can simulate ideal conditions (or any other) Expensive channel emulators (Azimuth, Anritsu, etc.)

7

Key Features Real-time RF channel and network bandwidth monitoring Point-to-point as well as point-to-multipoint comm. link testing Communication with external SW/HW over Telnet and SSH Managing test SW on access point, core network server, end user PC User-definable application execution scenarios on different sub-networks Ability to trigger one scenario from another scenario Flexible application logging to single/multiple files/windows Automatic log persistence to local/remote storage over FTP IPv6 PTP (Peer-to-Peer) protocol used for internal communication Integrates with industry-standard as well as home-made tools Supports Win-XP and Win-7 platforms Light-weight and resource not hungry

8

What makes cFrame different? Consolidated test bed automation model

Requires high power dedicated servers and software ~$10K for server + ~$5K for network HW per test bed Limited user modification (closed system)

Distributed test bed automation model Promotes reuse of existing test bed HW/SW Corresponding savings per test bed Virtually unlimited user modification (open system)

9

Industries and Applications Mobile network operators (carriers)

Network installation Network upgrades Network monitoring

Vendors of network infrastructure and services New product development Product design verification Product analysis (competition, performance) New application verification

Other Industries Defense & public security (analyze TETRA, APCO networks, etc.) Wireless internet providers (improve network quality, etc.) Broadcasting (terrestrial TV and radio service quality) Regulation institutions (analyze interference of all operators, etc.)

10

Mobile networktest bed configuration withControlling & Monitoring Devices

WireLess / RF channelEthernet / LANEthernet / WAN

Mobile DeviceAccess PointCore Network CNServer/Services

Pic 2. End-To-End WireLess Network Control/Monitor structure

RF Channel Simulator

UE Control / Monitor Win PC

RF Control/Monitor Win PC

AP Control/Monitor Linux/Win PC

CN Control/Monitor Linux/Win PC

Service Control /Monitor Linux/Win PC

Ethernet / LAN

Local ConectionPhysilcal / Logical

11

Controlling / Monitoring ToolsC:\>iperf -c27.135.2.98 -p9004 -u -b25M -i1 -fm -l1350------------------------------------------------------------Client connecting to 27.135.2.98, UDP port 9004Sending 1350 byte datagramsUDP buffer size: 0.01 MByte (default)------------------------------------------------------------[1912] local 27.148.0.43 port 1610 connected with 27.135.2.98 port 9004[ ID] Interval Transfer Bandwidth[1912] 0.0- 1.0 sec 0.26 MBytes 2.20 Mbits/sec[1912] 1.0- 2.0 sec 0.08 MBytes 0.69 Mbits/sec[1912] 2.0- 3.0 sec 0.01 MBytes 0.10 Mbits/sec[1912] 3.0- 4.0 sec 0.02 MBytes 0.19 Mbits/sec[1912] 4.0- 5.0 sec 0.11 MBytes 0.93 Mbits/sec[1912] 5.0- 6.0 sec 0.05 MBytes 0.38 Mbits/sec[1912] 6.0- 7.0 sec 0.16 MBytes 1.38 Mbits/sec[1912] 7.0- 8.0 sec 0.00 MBytes 0.01 Mbits/sec[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec[1912] Server Report:[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec 57.882 ms 0/ 546 (0%)[1912] Sent 546 datagrams

C:\>iperf -c27.135.2.98 -p9004 -u -b25M -i1 -fm -l1350------------------------------------------------------------Client connecting to 27.135.2.98, UDP port 9004Sending 1350 byte datagramsUDP buffer size: 0.01 MByte (default)------------------------------------------------------------[1912] local 27.148.0.43 port 1610 connected with 27.135.2.98 port 9004[ ID] Interval Transfer Bandwidth[1912] 0.0- 1.0 sec 0.26 MBytes 2.20 Mbits/sec[1912] 1.0- 2.0 sec 0.08 MBytes 0.69 Mbits/sec[1912] 2.0- 3.0 sec 0.01 MBytes 0.10 Mbits/sec[1912] 3.0- 4.0 sec 0.02 MBytes 0.19 Mbits/sec[1912] 4.0- 5.0 sec 0.11 MBytes 0.93 Mbits/sec[1912] 5.0- 6.0 sec 0.05 MBytes 0.38 Mbits/sec[1912] 6.0- 7.0 sec 0.16 MBytes 1.38 Mbits/sec[1912] 7.0- 8.0 sec 0.00 MBytes 0.01 Mbits/sec[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec[1912] Server Report:[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec 57.882 ms 0/ 546 (0%)[1912] Sent 546 datagrams

WireLess / RF channel

Ethernet / LANEthernet / WAN


Pic 4. End-To-End WireLess Network Control/Monitor structure with APPs

UE Control / Monitor Win PC



CN Control/Monitor Linux/Win PC

Service Control /Monitor Linux/Win PC

Ethernet / LAN

Local ConectionPhysilcal / Logical

IPERF TCP / UDP traffic genAPP

IPERF TCP / UDP traffic genAPP


NPS> /on a1

Plugs to be turned on

Plug LOCAL A1: Local_InfeedA_Outlet1

Are you sure? (Y/N): yProcessing - please wait or <CR> to continue...

Network Power Switch Site ID: (undefined)

PLUG | NAME | STATUS | DELAY | DEF | PRI |--------+--------------------------+--------+-------+-----+-----+ A1 | Local_InfeedA_Outlet1 | ON | 0.5 S | ON | 1 | A2 | Local_InfeedA_Outlet2 | OFF | 0.5 S | ON | 2 | A3 | Local_InfeedA_Outlet3 | OFF | 0.5 S | ON | 3 | A4 | Local_InfeedA_Outlet4 | OFF | 0.5 S | ON | 4 | B1 | Local_InfeedB_Outlet1 | OFF | 0.5 S | ON | 5 | B2 | Local_InfeedB_Outlet2 | OFF | 0.5 S | ON | 6 | B3 | Local_InfeedB_Outlet3 | OFF | 0.5 S | ON | 7 | B4 | Local_InfeedB_Outlet4 | OFF | 0.5 S | ON | 8 |

WireSharkL3/L2 logs

APP

RF Channel Model control

APP

12

Mobile networktest bed configuration(multi user/channel)

Ethernet / LAN


Access Point’s

Core Network CN

Server/Services

Pic 3. Multi End-To-End WireLess Network Control/Monitor structure

RF Channel Simulator UE Control / Monitor Win PC



CN Control/Monitor Linux/Win PCService Control /

Monitor Linux/Win PC

LAN / WAN



Mobile Device’s

. . .

. . .. . .

13

cFrame Deployment

CMAP

WireLess / RF channelEthernet / LANEthernet / WAN


CoMa Master/Slave structure


PC-5PC-4PC-3PC-1 PC-2UE Control / Monitor

Win PCRF Control/Monitor

Win PCAP Control/Monitor

Linux/Win PCCN Control/Monitor

Linux/Win PCService Control /

Monitor Linux/Win PC

Ethernet / LAN

COMA Slave’s

COMA Master

14

cFrame Integration<agent> <terminals> <terminal> <name>Server1</name> <ip>192.168.1.3</ip> <system>Linux</system> <user></user> <password></password> <enabled /> </terminal> <terminal> <name>Server2</name> <ip>192.168.1.2</ip> <system>Windows</system> <user></user> <password></password> <enabled /> </terminal> </terminals>…

…<tools> <tool> <name>LocalIperf</name> <description>Bandwidth measuring software.</description> <run>shell</run> <enabled /> </tool> <tool> <name>LinuxIperf</name> <description>Bandwidth measuring software.</description> <run>telnet</run> <terminal>Server1</terminal> <enabled /> </tool> <tool> <name>WindowsIperf</name> <description>Bandwidth measuring software.</description> <run>telnet</run> <terminal>Server2</terminal> <enabled /> </tool> </tools></agent>

The easy aspects of integration should be listed in 3 bullet points on this slide.

15

Test Automation Script sample:DL-UDP TPUT with 2xUE’sConsole.Write("T15 Script Start");Toolset.Set("TOOL_SET1");Scenario.Run("SET_JFW_0.mss", 0);Scenario.Run("T1_PS_test.mss");string UE1_addr = Node.GetSubscriberIP("UE1", "27/8");string UE2_addr = Node.GetSubscriberIP("UE2", "27/8");Task iperfSrv = Task.Define("IPERF-UE", "iperf -s -u -i1 -fm -l1300");Task iperfClnt = Task.Define("IPERF-LIN","iperf -c " + UE1_addr + " -i1 -fm -u -l1300 -b30M -t60" );Task iperfSrv2 = Task.Define("IPERF-UE2", "iperf -s -u -i1 -fm -l1300");Task iperfClnt2 = Task.Define("IPERF-LIN","iperf -c " + UE2_addr + " -i1 -fm -u -l1300 -b30M -t60" );iperfSrv.Init(); iperfClnt.Init();iperfSrv2.Init(); iperfClnt2.Init();for (int i = 0; i< 60; i+=3){

Console.Write("Step:" + i);Scenario.Run("SET_JFW_0.mss", i);iperfSrv.Start();iperfSrv2.Start();iperfClnt.Start();iperfClnt2.Start();Scenario.Wait(15000);Console.Write("IPERF STOP");iperfClnt.Stop();iperfClnt2.Stop();iperfSrv.Stop();iperfSrv2.Stop();

}Scenario.Run("SET_JFW_0.mss", 0);Console.Write("T15 Script Ended");

16

Sample Test Plan Layout (NSN)RPT Test Cases Analysis

RPT Test Area # of TestCases

# of AutomatedCases % Automated

RPT - Multi UE - Bi_Directional 95 95 100%RPT - Single UE - Bi_Directional 96 96 100%RPT - Multi UE - Single Direction 157 157 100%RPT - Single UE - Single Direction 156 156 100%RPT - Attach 12 0 0%RPT – HARQ (Hybrid Automated Repeat Req.) 15 0 0%RPT - Latency 12 0 0%RPT - Sensitivity 34 0 0%

RPT - UL Power 24 0 0%

Total 601 504 83%

17

Sample Test Plan Layout (cont.)

TPUTAttachHARQLatencySensitivityUL Power

18

cFrame-collected Test Results

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 790

10

20

30

40

50

60

70 Throughput (Mbits/s) vs. Path Loss (dB)

DL-UDP-5.0DL-UDP-5.0-8h

19

cFrame collected Test Results

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 790

10

20

30

40

50

60

70Throughput (Mbit/s) vs. Path Loss (dB)

DL_UDP_TPUTDL_TCP_TPUTDL-UDP-5.0DL-TCP-5.0UL-UDP-5.0UL-TCP-5.0

20

Workflow/Process Benefits For engineers

No more terminal windows overhead No more manual input to remote user applications No more manual scheduling of user applications No server administration to join new nodes to the map Any node can become master (conductor) node Easy learning curve (C# scripting language)

For business Higher ROI for expensive HW/SW (e.g. Azimuth) Faster product development (shorter testing cycles) Fewer skilled personnel required to control the test

21

ROI and Payback Period ROI can be a value band where you have high, medium

and low ROI results as long as each scenario results in cFRAME results that are higher than traditional RF testing/optimization.

If applicable state a Payback period; meaning how quickly after a clients initial investment

State how quickly a client can implement cFrame over other solutions (lead time).

22

Contact

<Sales contact and stuff>

Software

cFrame framework slides