cFrame Presentation
Introduction to the Distributed Mobile Network Performance Testing Automation Tool
1
2
Who We Are
3
What is cFrame?
“cFrame provides an open automated platform for mobile network performance testing in both real and RF channel-simulated environments”
4
Current RF Testing LandscapeDescribe current state of RF Optimization and Testing: how are things being done now? Basically you are going to set the context of current methodology and later show how cFRAME dramatically improves on this.
State the basics but remember you want to define this on your own terms so that later when you present your solution it really shines and stands out.
The goal with the above is for Rubedo to control the conversation.
5
Key RF Performance Indicators
Throughput (main KPI) Latency (delay in network response) Attach/Detach success rate (make/hang up call) Sensitivity (related to energy efficiency) Anything else worth mentioning???
6
Real vs. Simulated Environment Real environment
Unpredictable Time consuming Requires more planning (weather, resources)
Simulated environment Stay in the lab (preferred by engineers) Don’t need over-the-air RF transmit license Don’t mess up with live existing networks Can simulate ideal conditions (or any other) Expensive channel emulators (Azimuth, Anritsu, etc.)
7
Key Features Real-time RF channel and network bandwidth monitoring Point-to-point as well as point-to-multipoint comm. link testing Communication with external SW/HW over Telnet and SSH Managing test SW on access point, core network server, end user PC User-definable application execution scenarios on different sub-networks Ability to trigger one scenario from another scenario Flexible application logging to single/multiple files/windows Automatic log persistence to local/remote storage over FTP IPv6 PTP (Peer-to-Peer) protocol used for internal communication Integrates with industry-standard as well as home-made tools Supports Win-XP and Win-7 platforms Light-weight and resource not hungry
8
What makes cFrame different? Consolidated test bed automation model
Requires high power dedicated servers and software ~$10K for server + ~$5K for network HW per test bed Limited user modification (closed system)
Distributed test bed automation model Promotes reuse of existing test bed HW/SW Corresponding savings per test bed Virtually unlimited user modification (open system)
9
Industries and Applications Mobile network operators (carriers)
Network installation Network upgrades Network monitoring
Vendors of network infrastructure and services New product development Product design verification Product analysis (competition, performance) New application verification
Other Industries Defense & public security (analyze TETRA, APCO networks, etc.) Wireless internet providers (improve network quality, etc.) Broadcasting (terrestrial TV and radio service quality) Regulation institutions (analyze interference of all operators, etc.)
10
Mobile networktest bed configuration withControlling & Monitoring Devices
WireLess / RF channelEthernet / LANEthernet / WAN
Mobile DeviceAccess PointCore Network CNServer/Services
Pic 2. End-To-End WireLess Network Control/Monitor structure
RF Channel Simulator
UE Control / Monitor Win PC
RF Control/Monitor Win PC
AP Control/Monitor Linux/Win PC
CN Control/Monitor Linux/Win PC
Service Control /Monitor Linux/Win PC
Ethernet / LAN
Local ConectionPhysilcal / Logical
11
Controlling / Monitoring ToolsC:\>iperf -c27.135.2.98 -p9004 -u -b25M -i1 -fm -l1350------------------------------------------------------------Client connecting to 27.135.2.98, UDP port 9004Sending 1350 byte datagramsUDP buffer size: 0.01 MByte (default)------------------------------------------------------------[1912] local 27.148.0.43 port 1610 connected with 27.135.2.98 port 9004[ ID] Interval Transfer Bandwidth[1912] 0.0- 1.0 sec 0.26 MBytes 2.20 Mbits/sec[1912] 1.0- 2.0 sec 0.08 MBytes 0.69 Mbits/sec[1912] 2.0- 3.0 sec 0.01 MBytes 0.10 Mbits/sec[1912] 3.0- 4.0 sec 0.02 MBytes 0.19 Mbits/sec[1912] 4.0- 5.0 sec 0.11 MBytes 0.93 Mbits/sec[1912] 5.0- 6.0 sec 0.05 MBytes 0.38 Mbits/sec[1912] 6.0- 7.0 sec 0.16 MBytes 1.38 Mbits/sec[1912] 7.0- 8.0 sec 0.00 MBytes 0.01 Mbits/sec[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec[1912] Server Report:[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec 57.882 ms 0/ 546 (0%)[1912] Sent 546 datagrams
C:\>iperf -c27.135.2.98 -p9004 -u -b25M -i1 -fm -l1350------------------------------------------------------------Client connecting to 27.135.2.98, UDP port 9004Sending 1350 byte datagramsUDP buffer size: 0.01 MByte (default)------------------------------------------------------------[1912] local 27.148.0.43 port 1610 connected with 27.135.2.98 port 9004[ ID] Interval Transfer Bandwidth[1912] 0.0- 1.0 sec 0.26 MBytes 2.20 Mbits/sec[1912] 1.0- 2.0 sec 0.08 MBytes 0.69 Mbits/sec[1912] 2.0- 3.0 sec 0.01 MBytes 0.10 Mbits/sec[1912] 3.0- 4.0 sec 0.02 MBytes 0.19 Mbits/sec[1912] 4.0- 5.0 sec 0.11 MBytes 0.93 Mbits/sec[1912] 5.0- 6.0 sec 0.05 MBytes 0.38 Mbits/sec[1912] 6.0- 7.0 sec 0.16 MBytes 1.38 Mbits/sec[1912] 7.0- 8.0 sec 0.00 MBytes 0.01 Mbits/sec[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec[1912] Server Report:[1912] 0.0- 8.5 sec 0.70 MBytes 0.70 Mbits/sec 57.882 ms 0/ 546 (0%)[1912] Sent 546 datagrams
WireLess / RF channel
Ethernet / LANEthernet / WAN
Mobile DeviceAccess PointCore Network CNServer/Services
Pic 4. End-To-End WireLess Network Control/Monitor structure with APPs
UE Control / Monitor Win PC
RF Control/Monitor Win PC
AP Control/Monitor Linux/Win PC
CN Control/Monitor Linux/Win PC
Service Control /Monitor Linux/Win PC
Ethernet / LAN
Local ConectionPhysilcal / Logical
IPERF TCP / UDP traffic genAPP
IPERF TCP / UDP traffic genAPP
RF Channel Simulator
NPS> /on a1
Plugs to be turned on
Plug LOCAL A1: Local_InfeedA_Outlet1
Are you sure? (Y/N): yProcessing - please wait or <CR> to continue...
Network Power Switch Site ID: (undefined)
PLUG | NAME | STATUS | DELAY | DEF | PRI |--------+--------------------------+--------+-------+-----+-----+ A1 | Local_InfeedA_Outlet1 | ON | 0.5 S | ON | 1 | A2 | Local_InfeedA_Outlet2 | OFF | 0.5 S | ON | 2 | A3 | Local_InfeedA_Outlet3 | OFF | 0.5 S | ON | 3 | A4 | Local_InfeedA_Outlet4 | OFF | 0.5 S | ON | 4 | B1 | Local_InfeedB_Outlet1 | OFF | 0.5 S | ON | 5 | B2 | Local_InfeedB_Outlet2 | OFF | 0.5 S | ON | 6 | B3 | Local_InfeedB_Outlet3 | OFF | 0.5 S | ON | 7 | B4 | Local_InfeedB_Outlet4 | OFF | 0.5 S | ON | 8 |
WireSharkL3/L2 logs
APP
RF Channel Model control
APP
12
Mobile networktest bed configuration(multi user/channel)
Ethernet / LAN
RF Channel Simulator
Access Point’s
Core Network CN
Server/Services
Pic 3. Multi End-To-End WireLess Network Control/Monitor structure
RF Channel Simulator UE Control / Monitor Win PC
RF Control/Monitor Win PC
AP Control/Monitor Linux/Win PC
CN Control/Monitor Linux/Win PCService Control /
Monitor Linux/Win PC
LAN / WAN
RF Channel Simulator
RF Channel Simulator
Mobile Device’s
. . .
. . .. . .
13
cFrame Deployment
CMAP
WireLess / RF channelEthernet / LANEthernet / WAN
Mobile DeviceAccess PointCore Network CNServer/Services
CoMa Master/Slave structure
RF Channel Simulator
PC-5PC-4PC-3PC-1 PC-2UE Control / Monitor
Win PCRF Control/Monitor
Win PCAP Control/Monitor
Linux/Win PCCN Control/Monitor
Linux/Win PCService Control /
Monitor Linux/Win PC
Ethernet / LAN
COMA Slave’s
COMA Master
14
cFrame Integration<agent> <terminals> <terminal> <name>Server1</name> <ip>192.168.1.3</ip> <system>Linux</system> <user></user> <password></password> <enabled /> </terminal> <terminal> <name>Server2</name> <ip>192.168.1.2</ip> <system>Windows</system> <user></user> <password></password> <enabled /> </terminal> </terminals>…
…<tools> <tool> <name>LocalIperf</name> <description>Bandwidth measuring software.</description> <run>shell</run> <enabled /> </tool> <tool> <name>LinuxIperf</name> <description>Bandwidth measuring software.</description> <run>telnet</run> <terminal>Server1</terminal> <enabled /> </tool> <tool> <name>WindowsIperf</name> <description>Bandwidth measuring software.</description> <run>telnet</run> <terminal>Server2</terminal> <enabled /> </tool> </tools></agent>
The easy aspects of integration should be listed in 3 bullet points on this slide.
15
Test Automation Script sample:DL-UDP TPUT with 2xUE’sConsole.Write("T15 Script Start");Toolset.Set("TOOL_SET1");Scenario.Run("SET_JFW_0.mss", 0);Scenario.Run("T1_PS_test.mss");string UE1_addr = Node.GetSubscriberIP("UE1", "27/8");string UE2_addr = Node.GetSubscriberIP("UE2", "27/8");Task iperfSrv = Task.Define("IPERF-UE", "iperf -s -u -i1 -fm -l1300");Task iperfClnt = Task.Define("IPERF-LIN","iperf -c " + UE1_addr + " -i1 -fm -u -l1300 -b30M -t60" );Task iperfSrv2 = Task.Define("IPERF-UE2", "iperf -s -u -i1 -fm -l1300");Task iperfClnt2 = Task.Define("IPERF-LIN","iperf -c " + UE2_addr + " -i1 -fm -u -l1300 -b30M -t60" );iperfSrv.Init(); iperfClnt.Init();iperfSrv2.Init(); iperfClnt2.Init();for (int i = 0; i< 60; i+=3){
Console.Write("Step:" + i);Scenario.Run("SET_JFW_0.mss", i);iperfSrv.Start();iperfSrv2.Start();iperfClnt.Start();iperfClnt2.Start();Scenario.Wait(15000);Console.Write("IPERF STOP");iperfClnt.Stop();iperfClnt2.Stop();iperfSrv.Stop();iperfSrv2.Stop();
}Scenario.Run("SET_JFW_0.mss", 0);Console.Write("T15 Script Ended");
16
Sample Test Plan Layout (NSN)RPT Test Cases Analysis
RPT Test Area # of TestCases
# of AutomatedCases % Automated
RPT - Multi UE - Bi_Directional 95 95 100%RPT - Single UE - Bi_Directional 96 96 100%RPT - Multi UE - Single Direction 157 157 100%RPT - Single UE - Single Direction 156 156 100%RPT - Attach 12 0 0%RPT – HARQ (Hybrid Automated Repeat Req.) 15 0 0%RPT - Latency 12 0 0%RPT - Sensitivity 34 0 0%
RPT - UL Power 24 0 0%
Total 601 504 83%
17
Sample Test Plan Layout (cont.)
TPUTAttachHARQLatencySensitivityUL Power
18
cFrame-collected Test Results
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 790
10
20
30
40
50
60
70 Throughput (Mbits/s) vs. Path Loss (dB)
DL-UDP-5.0DL-UDP-5.0-8h
19
cFrame collected Test Results
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 790
10
20
30
40
50
60
70Throughput (Mbit/s) vs. Path Loss (dB)
DL_UDP_TPUTDL_TCP_TPUTDL-UDP-5.0DL-TCP-5.0UL-UDP-5.0UL-TCP-5.0
20
Workflow/Process Benefits For engineers
No more terminal windows overhead No more manual input to remote user applications No more manual scheduling of user applications No server administration to join new nodes to the map Any node can become master (conductor) node Easy learning curve (C# scripting language)
For business Higher ROI for expensive HW/SW (e.g. Azimuth) Faster product development (shorter testing cycles) Fewer skilled personnel required to control the test
21
ROI and Payback Period ROI can be a value band where you have high, medium
and low ROI results as long as each scenario results in cFRAME results that are higher than traditional RF testing/optimization.
If applicable state a Payback period; meaning how quickly after a clients initial investment
State how quickly a client can implement cFrame over other solutions (lead time).
22
Contact
<Sales contact and stuff>