UPTEC F14022

Examensarbete 30 hpJuni 2014

Detecting changes in UERC switches

A sequence analysis of UERC switches in

a mobile network

Jacob HellmanLars-Gunnar Olofsson

Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student

Abstract

Detecting changes in UERC switches

Jacob Hellman, Lars-Gunnar Olofsson

This thesis investigates the possibility to analyse a mobile network with sequences of UERC switches specific to each user equipment. An UERC is essentially a channel that carries information and a user equipment connects to different UERCs depending on whether they want to talk and/or send data with different qualities. As a major player in the mobile technology industry, Ericsson strives to optimise the use of the UERCs and are looking for an automated way to detect changes. The first task was to identify and retrieve the required events from the network log files in order to create the UERC sequences. As a way to test the thesis assumption, and give a proof of concept, two different data sets were analysed were changes had been made to the network settings that should have affected the UERC sequences. With the use of n-grams, Markov chains and Bayesian Estimation testing, the changes could be identified and the thesis assumption could be confirmed - UERC sequences provides a possible way of analysing a mobile network.

Sponsor: Ericsson ABISSN: 1401-5757, UPTEC F14022Examinator: Tomas NybergÄmnesgranskare: Bengt CarlssonHandledare: Jan Frank Rune Pettersson

Popularvetenskaplig sammanfattningMobiltelefonindustrin har under de senaste decennierna utvecklats fran enkla telefonermed begransad funktionalitet till hogteknologiska apparater med tal, data och videooverforingar. Denna extrema utveckling har stallt hoga krav pa de aktorer som distribu-erar det mobila natverket till slutanvandarna. For att kunna havda sig pa marknadenoch forbli konkurrenskraftig kravs saledes att det mobila natet standigt utvecklas for attmota anvandarnas krav. Utvecklingen gors normalt i labbmiljo for att testa eventuellafordelar samt nackdelar innan en eventuell liveuppdatering. Aven de mest sofistikeradelabbmetoderna kan fa svart att detektera allt, vilket kan leda till att nya uppdatering-ar som tas i bruk kan paverka anvandarens upplevelse pa ett negativt sett. Eftersomanvandarens upplevelse alltid star i fokus har detta varit valdigt kritiskt och nagot somindustrin lagt stora resurser pa genom aren. Denna uppsats har i samarbete med Erics-son AB undersokt mojligheten att automatisera denna process for att snabbare detekteraeventuelle forandringar i natverket mellan nya uppdateringar.

For att detektera dessa forandringar anvandes data fran ett storre sportevenemang. An-ledningen till detta var framst till den hoga koncentrationen av anvandare pa en litenarea, samt att de flesta tenderar att stanna hela evenemangstiden vilket resulterar i lang-re och intressantare sekvenser. De matematiska algoritmerna som anvandes kunde pa ettfortjanstfullt satt detektera de forandringar som efterfragats och kan ligga som grund foren framtida utveckling av ett automatiserat verktyg.

PrefaceThis master thesis was written by Jacob Hellman and Lars-Gunnar Olofsson, EngineeringPhysic student’s at Uppsala University. The thesis was performed at Kista, Stockholmin collaboration with Ericsson AB.

Both students have participated throughout the whole project and have a good under-standing of all parts, Jacob had the major responsibility for the ASN.1 parser, construct-ing the sequences and the methods in discrete-time. Lars-Gunnar’s main responisbilitywas the methods in continuous-time; the BEST analysis as well as constructing thecontinuous-time Markov process.

Contents

1 Introduction 11.1 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 32.1 WCDMA network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 RRC and RAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1.2 User Equipment RAB Combination . . . . . . . . . . . . . . . . . 5

2.2 General Performance Event Handling . . . . . . . . . . . . . . . . . . . . 52.2.1 Storing and retrieving GPEH logs . . . . . . . . . . . . . . . . . . 6

2.3 User Equipment Real-Time Trace . . . . . . . . . . . . . . . . . . . . . . 7

3 Theory 83.1 Null Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2 UERC Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.3 Normalized Sequences . . . . . . . . . . . . . . . . . . . . . . . . 93.2.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 Sequence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.4 n-gram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.5 Discrete-time Markov Process . . . . . . . . . . . . . . . . . . . . . . . . 113.5.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.5.2 Transition Matrix and Absorbing States . . . . . . . . . . . . . . 123.5.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.6 Non-parametric Chi-Square . . . . . . . . . . . . . . . . . . . . . . . . . 133.6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.6.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.7 Continous-time Markov Process . . . . . . . . . . . . . . . . . . . . . . . 143.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.7.2 The Q-matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.7.3 Exponential time and Kolmogorov equations . . . . . . . . . . . . 153.7.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.8 Bayesian estimation testing . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.8.2 Variable distributions . . . . . . . . . . . . . . . . . . . . . . . . . 173.8.3 Mean and standard deviation . . . . . . . . . . . . . . . . . . . . 183.8.4 Effect size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.8.5 Region of Practical Equivalence . . . . . . . . . . . . . . . . . . . 213.8.6 Selecting sample size . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Work Flow, Technology and Data 244.1 Work Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.1.1 Identify events and create call sequences . . . . . . . . . . . . . . 244.1.2 Pre-processing of the data . . . . . . . . . . . . . . . . . . . . . . 254.1.3 Discrete-time analysis . . . . . . . . . . . . . . . . . . . . . . . . 254.1.4 Continuous-time analysis . . . . . . . . . . . . . . . . . . . . . . . 25

4.2 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.2.1 Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.2.2 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.2.3 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 GPEH Test Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5 Results 295.1 User Equipment Real-Time Trace . . . . . . . . . . . . . . . . . . . . . . 295.2 Detecting Events and Sequences . . . . . . . . . . . . . . . . . . . . . . . 29

5.2.1 ASN.1 Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.3 n-grams and Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.3.1 Unigrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325.3.2 Bigrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.3.3 Trigrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.4 BEST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375.4.1 Define range of ROPE effect size . . . . . . . . . . . . . . . . . . 385.4.2 UERC 4 - (FACH) and 21 - (URA) . . . . . . . . . . . . . . . . . 385.4.3 UERC 25 - (EUL/HS) . . . . . . . . . . . . . . . . . . . . . . . . 38

5.5 Continous time Markov process . . . . . . . . . . . . . . . . . . . . . . . 39

6 Discussion 426.1 ASN.1 parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426.2 Sequence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6.2.1 UERC Switching Patterns . . . . . . . . . . . . . . . . . . . . . . 436.2.2 UERC Holding Times . . . . . . . . . . . . . . . . . . . . . . . . 43

7 Conclusion 457.1 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.2 Sequence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

List of Abbreviations

Abbreviation DescriptionCN Core Network

CS Circuit Switched

CTMP Continuous-time Markov Process

DTMP Discrete-time Markov Process

GPEH General Performance Event Handling

OSS Operational Support System

PS Packet Switched

RAB Radio Access Bearer

RBS Radio Base Station

RNC Radio Network Controller

RRC Radio Resource Control

UE User Equipment

UERC User Equipment RAB Combination

UERTT User Equipment Real-Time Trace

UMTS Universal Mobile Telecommunications System

WCDMA Wideband Code Division Multiple Access

Chapter 1

Introduction

Wideband Code Division Multiple Access (WCDMA) is a technology within the standard-ized 3G mobile communication system and although it is over a decade ago since the firstversion was released it is the most widely spread mobile network system in the world withover 1 billion subscribers. Ericsson supports more than 240 different customers aroundthe world with WCDMA technology and about 45 percent of all smartphone traffic goesthrough Ericsson networks (Ericsson 2014).

With this many users it is essential that the network is always operational and deliversa stable user experience. One of Ericsson’s tools for monitoring and trouble shootingthe network are the log files called General Performance Event Handling (GPEH). Theycontain information about the activities specific to every user equipment (UE) in thenetwork. Every network is administrated by a Radio Network Controller (RNC) whichcan handle up to 100,000 users at the same time. The log files quickly become very largewhich makes a fast manual analysis impractical.

User Equipment Real-Time Trace (UERTT) is another system recently deployed to enablereal-time monitoring for a faster analysis and trouble shooting of a WCDMA network.By changing the output format to protocol buffers, Ericsson managed to shrink the datasize which then made it possible for a real-time trace. The UERTT system is currentlyused for monitoring UEs experiencing problems in the network.

The WCDMA network is continuously evolving with upgraded software and tweakedsettings. It is very important to monitor the network after any changes have been madeto it in order to find abnormal behaviour. One way of doing that is by examining theGPEH or UERTT log files. Although built on different technologies, the data handledhave similar structure (key-value pairs) and represents the same events which means theycan be examined by the same analysis.

There are several ways of trouble shooting a network and finding ’abnormal behaviour’quickly becomes a philosophical question rather than a scientific one. This project istherefore limited to analysing sequences of switches between different Radio Access Bear-ers (RABs). A RAB is essentially a channel that handles data or voice with differentquality. If a UE requires both it can have two RABs simultaneously, a so called multi-RAB. Every RAB and multi-RAB has a Unique id number which is conveniently referred

1

to as the User Equipment RAB Combination (UERC).

To simplify it, imagine four different UERCs; 0 (idle), 1 (voice) , 2 (data) and 3 (voice+data).A UE will both start and stop in idle mode and in between it would jump amongst theother UERCs. Both the switches between and the time spent in each UERC will belogged. The time spent in a UERC is referred to as its holding time. A UERC sequencecould look like:

0 → 2 → 3 → 2 → 1 → 2 → 0.

where the UE started and stopped in idle and used both data and voice in between. Thissequence only used five UERCs but in reality there are in fact over 70 different. However,in most cases, they are not accessible all at once.

1.1 Problem descriptionGPEH and UERTT logs can be used to track the traffic of a specific UE and createsequence of UERC switches. This would open up for a new type of analysis based on thesequences with unexplored potential.

There are also issues regarding latency involved with the current analysis methods and itis desired to automate as much as possible by utilizing data mining and machine learningmethods. UERTT could, theoretically, provide a real-time analysis and find abnormalbehaviour much quicker than the GPEH logs could ever offer.

Given UERC sequences the model should be able to detect abnormal behaviour given ac-cess to sequences with ’normal’ behaviour. The question is, given two data sets, are thereany statistical differences between the data sets, i.e. can we reject the null hypothesis.

1.2 GoalIn order to improve the trouble shooting in the WCDMA network by analysing sequencesof UERCs there are several questions that needs to be answered. Which events arerelevant to UERC switches? Ca UERTT offer a quicker way of trouble shooting? Whatca UERC sequences tell us?

Besides answering these questions, this project aims to develop and implement methodsfor sequence analysis that should detect changes in UERC switches given two differentdata sets where the conditions have been changed. The analysis should detect the fol-lowing changes:

• a UERC have either been removed or added

• Switching behaviour have changed, e.g. switching between two UERCs have beenadded or removed

• The holding time in a UERC have changed

To reach these goals, methods in both discrete and continuous time need to be imple-mented. The discrete-time methods should detect changes in the switching pattern andthe continuous time methods should work with the holding times and detect changesthere.

2

Chapter 2

Background

The UERC sequnces are based on events in Ericsson’s WCDMA network so a basicunderstanding of how the network works and what the different events really mean hasbeen essential for the success of this master thesis. The way to construct sequences wasnot known from the beginning and it proved to be as much a philosophical questions asa scientific one.

The background section will go into how the WCDMA network works, its components,and other basic facts about the network. It will take a deeper look into the RRC and RABwhich play a major part in event handling, and the background of GPEH and UERTTwill also be explored.

2.1 WCDMA networkThe WCDMA Radio Access Network (RAN) provides the connection between the CoreNetwork (CN) and the UE. The network is comprised of RNCs and Radio Base Stations(RBSs). They communicate in different interfaces (see Figure 2.1) where transport ofsignaling and user data is performed.

• The UE includes all equipment types used by subscribers. They are divided intotwo types; the mobile equipment (ME) and the identity module (USIM). The MEpart is used for radio communication and the USIM holds the subscribers identity,performs authentication and stores information from the UE. (Eri 2002)

• The RBS is the component that serves one ore more cells with radio coverage. Itis also responsible for radio transmission and reception to the UE. (Eri 2002)

• The RNC controls the use and integrity of the system. It manages the RABs(between the UE and the CN) for data transport and mobility.(Eri 2002)

• The CN is responsible for switching and routing calls and data connections toexternal networks. It contains the physical entities that provide support for thenetwork features and telecommunication services.

• When traffic gets more sophisticated with both packet-oriented Internet traffic (3Gand 4G) and voice communication there is a need to track network components,usage and traffic patterns, billing and reporting. The Operating Support System

3

Figure 2.1: WCDMA network architecture.

(OSS) monitors, controls, analyse and manage the telephone and computer networkregarding performance.

The connection and maintenance of a call is handled by the RRC and RAB.

2.1.1 RRC and RABTwo of the most important concepts to understand when working with wireless networks,such as WCDMA, is the function of the RRC and RAB. Together they play a major partin the establishment and maintenance of a call, whether it is voice (CS) or data (PS).

The RRC is responsible for the signaling (or control) and the RAB for the data (orinformation) of a call. It is the job of the RRC to send requests and receipts and theRAB to carry the information across. Essentially, a RAB is channel that connects twopoints. There are a number of different RAB configuration (channels) which are useddepending on the requested service (voice, data or both) and quality. A UE can havemultiple RABs, one for voice and one for data (Pedrini 2013).

Figure 2.2 illustrates the use of the RRC and RAB in a mobile network. All informationsent within the network is carried by the RAB. The RRC controls the connection betweenthe UE and the RNC. Figure 2.2 also illustrates how a call is being set up by the RRC.

First, the UE sends the ”RRC Connection Request” event to tell the RNC it wants toestablish a connection. The RNC responds with a ”RRC Connection Setup” containing

4

Figure 2.2: Illustration of the communication channels between UE, RBS, RNC and CN.

setup information. When the setup has completed successfully a ”RRC Connection SetupComplete” event is transmitted from the UE and now the UE has gone from idle to aRAB called SRB. Shortly after that a ”Radio Bearer Setup” is sent with informationabout the available RABs to the UE and when it has connected to the requested RAB(s)it will send a ”Radio Bearer Setup Complete”.

2.1.2 User Equipment RAB CombinationThe UERC is a code formalism that tells which RAB(s) a UE is currently connectedto. Since a UE can have several RABs at the same time it helps to have a system thatcan identify all RABs as well as their combinations. This way, we reduce the number ofconnections to just one UERC instead of (possibly) two RABs.

Table 7.1 in Appendix A contains the full list of available RAB combinations, their IDand a short description.

2.2 General Performance Event HandlingThe communication in the WCDMA network can be monitored by examining the eventlogs created by the RNC and OSS. This binary data is called General Performance EventHandling (GPEH) and for this project they are the main source of information. These logscontain events regarding setting up a connection between the UE and RAN, measurementreports as well as changing RAB, and a lot more. This report will focus on events neededto trace which UERCs that have been used during a session.

In order to analyse the GPEH data it needs to be decoded from its binary format. Forthis purpose a Perl parser will be used that decodes the data into a ASN.1 1 formattedtextfile. Listing 2.1 shows a sample output from one of those textfiles showing the formatfor an event called ’internal-ue-move’.

1Abstract Syntax Notation number One (ASN.1) defines a formalism for representing, encoding anddecoding data regardless of language implementation and application. It is a standard of rules andstructures commonly used for describing telecommunication protocols (Union 2014).

5

Listing 2.1: GPEH event ’internal-ue-move’ in ASN.1 format.

[2013−12−30 2 1 . 2 9 . 5 9 : 7 8 9 ] RncTopUeC produced t h i s event event id = (417)BUS XXX i n t e r n a l−ue−move PDU ENCODED, ueRef = 2490 − − −value EVENT : :={scannerId ’00000000 00000000 00000100 ’B,hour 21 ,minute 29 ,second 59 ,m i l l i s e c 789 ,events i n t e r n a l−ue−move :{

ue−context v a l i d : 2490 ,rnc−module−id v a l i d : 12 ,c−id−1 i n v a l i d : 65535 ,rnc−id−1 i n v a l i d : 4095 ,c−id−2 i n v a l i d : 65535 ,rnc−id−2 i n v a l i d : 4095 ,c−id−3 i n v a l i d : 65535 ,rnc−id−3 i n v a l i d : 4095 ,c−id−4 i n v a l i d : 65535 ,rnc−id−4 i n v a l i d : 4095 ,source−connect ion−p r o p e r t i e s : 540 ,source−c−id−1−secondary−serv−hsdsch−c e l l i n v a l i d : 65535 ,source−connect ion−prope r t i e s−ext : 0 ,

}}

All events contains information regarding when it happened, in which rnc-module it wasrecorded and to which ue-context it is in regard to. Different events contain differentinformation, e.g. a ’internal-channel-switching’ event would also contain informationregarding source and target UERC.

A complication with the GPEH data is that most events do not contain the unique idfor the UE (IMSI 2), which can make it hard to know for which UE the specific eventbelongs to. The ue-context is only unique as long as the connection to the specific UE ismaintained. When the connection is lost another UE will be assigned the same ue-contextand it can be hard to distinguish when this has happened.

2.2.1 Storing and retrieving GPEH logsThe GPEH logs are stored in the OSS every fifteenth minute. They often occupy severalgigabytes of space and contains millions of events. Every RNC is divided into smallermodules and every one of these modules creates its own GPEH log file.

Getting GPEH log files can be done either by using one of Ericsson’s test sites to set upa network and record data from there. Another way, the way we did, is to contact one ofEricsson’s customers and ask for ’real’ data from one of their networks.

2International Mobile Subscriber Identity is a unique identification number that belongs to the SIM-card.

6

2.3 User Equipment Real-Time TraceThe UERTT is a relative new feature in the WCDMA network that enables an instanta-neous analysis of the network. The real-time trace is built on the same events that GPEHis comprised of but instead of storing the data locally the OSS streams the data directlythrough an outgoing port. This is possible with the use of Google Protocol Buffers (GPB)that uses significantly less space than GPEH data.

GPB is a flexible and efficient way of serializing structured data. It is similar to XMLbut smaller and faster. GPB is language and platform neutral with a primary target oncommunication protocols, data storage and more (Google 2012).

7

Chapter 3

Theory

The analysis part is built on the null hypothesis, which is an essential part of statisticalinference, and whether it is retained or rejected determines if the two data sets haveany statistical difference. The theory section also includes thesis specific topics, such asUERC sequences and normalized sequences, to more general statistics and data miningmethods.

3.1 Null HypothesisIn statistics a null hypothesis is an assumption about the difference or similarity betweentwo data sets. This assumption may or may not be true where the null hypothesis wouldbe a formal procedure to accept or reject the hypotheses. If the data set is not consistentwith the statistical hypothesis it will be rejected. There are two possible outcomes andsome researches argue that you either accept or reject the null hypothesis, while othersargue that you reject or you fail to reject the null hypothesis. The major issue with thedistinction is that a failure to reject the null hypothesis implies that the data sets do nothave sufficiently persuasive data to reject the null hypothesis.(Trek 2014)

There is two different practises to accept or reject the null hypothesis, with reference toa P-value or region of acceptance. The P-value rejects the null hypothesis if the value isless than the significance level (normally described as α). The value of α is set in relationto the confidence, a lower value would equal higher confidence. E.g., a α value of 0.05would imply a confidence level of 95 %. Region of acceptance is a range of values and ifstatistics falls within these limits the null hypothesis is not rejected. On the other handif statistics falls outside the region the null hypothesis is rejected. In a case like this thehypothesis has been rejected at α level of significance. (Trek 2014)

3.2 UERC SequencesThe UERC sequences introduce a new way of analysing a mobile network and sincethere has been no known previous work in the area it is important that the concept iswell-defined.

8

3.2.1 IntroductionWhether an UE is using voice or data in the mobile network it will always have to beconnected to at least one RAB throughout the whole session. As mentioned in section2.1.1, there are different RABs for voice (CS) and data (PS), but to make it easier theterm UERC is introduced to include the combined RABs and assign them an uniqueUERC id. An UERC sequence is basically a linked list of integers that describes in whichorder the UE has switched between the different UERC ids.

3.2.2 DefinitionAn UERC sequence is defined as a continuous sequences of UERC ids associated to thesame UE and session. Each sequence must start with either UERC id 1 (SRB) or id 21(URA) and end in 0 (idle) or 21. An example of sequences could be

(1) 1 4 21 4 25 21 4 25 0

(2) 1 2 123 9 2 123 25 4 21 4 25 4 21 4 25 4 21 4 0

but normally they would be much longer since the UE prefers staying in URA insteadof going to idle. Sequence (1) switches between five different UERCs while sequence (2)switches between eight different. The sequences are also accompanied by a log to keeptrack of the time when the UE entered and left the UERC state. The time spent in anUERC is referred to as the holding time.

3.2.3 Normalized SequencesA normalized sequence is not a scientific established term but rather something specificto this thesis. The UERC sequences generated from the data will contain some UERCsthat only occurs a very limited amount of times and not enough to be analysed and drawnconclusions from regarding switching patterns and holding times. This raised the need fora normalization of the sequences that disregards the rare UERCs and only considers thefrequent ones. The normalization is based on how frequent a pattern is in all sequences.

3.2.4 LimitationsIn this thesis only sequences that start and ends in the specified RNC will be considered,e.g. UEs coming from a soft or hard handovers from other RNCs will not be included inthe analysis.

3.3 Sequence AnalysisAnalysing and comparing UERC sequences towards either rejecting or accepting the nullhypothesis requires several different methods. A mindmap over the different methodsused in the sequence analysis and there key components can be seen in Figure 3.1. Thereare two groups of methods; the ones operating in discrete-time and the ones operating incontinuous time. Each group have two methods with n-gram and discrete-time MarkovProcess (DTMP) belonging to discrete-time and Bayesian Estimation Supersedes theT-test (BEST) and continuous-time Markov Process (CTMP) belonging to continuoustime.

9

SequenceAnalysis

BEST

ROPE

EffectSize

CTMP

Q-matrix

DTMP

Chi-square

P-matrix

n-gram

Discrete Time Continuous Time

Figure 3.1: Theory mindmap.

For the discrete-time methods the goal is to find the pattern of switches while the contin-uous time methods will focus on the holding times in each UERC. Some of the methodswill also collaborate and results can be shared among them. The DTMPs P-matrix is forexampled constructed by the results from the n-gram of order 2 and the P-matrix will inturn be used to built the Q-matrix in the CTMP.

3.4 n-gramA first approach of finding information from a sequence is to look at the frequency of theitems and in which order they usually appear. For this purpose the n-gram method isparticularly useful. Christopher D. Manning and Hinrich Schutze (Manning & Schutze1999) provided a good introduction to this topic that created the foundation for thissection.

3.4.1 IntroductionA n-gram is a sub-sequence of n continuous items with a corresponding number for howmany times that sub-sequence occurs in a given sequence. For example, how many timesthe 2-gram ”the apple” appears in a book. It is a common method in language processingbut can be applied to any types of sequences, such as UERC sequences.

10

3.4.2 DefinitionA n-gram is calculated by estimating the probability function

P = (wn|w1, . . . , wn−1) (3.4.1)

which basically says that the history (previous items) is used as a classification to predictthe next item. The estimation follows the Markov assumption, that the next item is onlyaffected by the (n − 1) previous items. In other words, an n-gram is equivalent to anMarkov Process of order (n− 1). Table 3.1 illustrates the most common n-grams, what

Table 3.1: Attributes for n-grams for n=1,2,3.

1-gram sequence 2-gram sequence 3-gram sequenceReferred to as: Unigram Bigram Trigram

Order of DTMP: 0 1 2

they are referred to as and their corresponding order of DTMP. The reason why higherorder are not so common is because they quickly grow in size. If a sequence consists ofN items, the parameter space is equal to (Nn −N) for a n-gram of order n (Manning &Schutze 1999).

3.5 Discrete-time Markov ProcessThis section is introduced to show how a DTMP of order 1 works and its applications.A discrete Markov chain is a mathematical system that experience transitions betweendifferent states in a state space. It is an memory less process where the next step dependsonly on the current state and not the sequence of events that it passed (only holds fororder 1). Kai Lai Chung (Chung 2000), Jacques Janssen and Raimondo Manca (Janssen& Manca 2006) provided a good introduction to this topic that created the foundationfor this section.

3.5.1 DefinitionIntroduce a system S with m possible states with space states I = [1, 2, 3, ....,m]. Thesystem S acts randomly in the discrete-time T = [t0, t1, t2, t3, ...., tm]. Define the state Xtn

of system S at time tn ∈ T and assume that in small interval of time there is no changes.The random sequence Xtn , tn ∈ T is a Markov process when for all xt0 , xt1 , xt2 , ...., xtn ∈ S

P [Xtn = xtn|Xt0 = xt0 , ..., Xtn−1 = xtn−1 ] = P [Xtn = xtn|Xtn−1 = xtn−1 ]. (3.5.1)

The random sequence Xtn , tn ∈ T is homogeneous if it jumps between states independentof time, i.e.,

P [Xtn = xtn|Xtn−1 = xtn−1 ] = [Xt1 = xt1|Xt0 = xt0 ]. (3.5.2)

If (3.5.2) not holds, the random sequence is non-homogeneous.

11

3.5.2 Transition Matrix and Absorbing StatesThe migration probability

P [Xtn = j|Xtn−1 = i] = ρij, ∀i, j ∈ I, tn ∈ T. (3.5.3)

Define vector p = (p1, p2, ..., pm) as the initial condition probabilities, with pi = P [X0 =i], i ∈ I. The following conditions should be satisfied by the probabilities.

{pi ≥ 0, ∀i ∈ I,∑

i∈I pi = 1, ∀i ∈ I(3.5.4)

A homogeneous DTMP is based on the couple (p, P ). If Xo = i, then the probabilitythat S starts from state i is equal to 1, then vector p will be pjδij,∀j ∈ I, where

δij =

{1, if i = j

0, Otherwise(3.5.5)

A state i ∈ I is absorbing if

• pij(n) = 0, ∀j 6= i, n ∈ T , and

• pii(n) = 1, ∀n ∈ T .

The system can not leave when it is in an absorbing state. And when a DTMP has atleast one absorbing state it is called absorbent which means that it can migrate from anynon-absorbing state to an absorbing. With at least one absorbing state the process cangive us the following information.

• The discrete-time it takes before it finds its way to an absorbing state.

• limt→∞ pij(t), ∀i, j ∈ I

3.5.3 ExampleSuppose a system S with four states where two of them are absorbing (see Figure 3.2).Assume that S is homogeneous in time with the following probability matrix.

123 4

P12(t)

P14(t)

P21(t)

P23(t)P33(t) P44(t)

Figure 3.2: A Markov chain with absorbing states.

P =

1− 4a a 0 3aa 1− 3a 2a 00 0 1 00 0 0 1

, 0 < a <1

4(3.5.6)

12

The migration matrix between non-absorbing states is given by A from the P matrix inequation 3.5.6

A =

(1− 4a aa 1− 3a

)(3.5.7)

3.6 Non-parametric Chi-SquareThe problem of comparing two transition matrices from different Markov processes canbe seen as a comparison of two proportions. In this case the proportions are each elementin the transition matrices, which describes the proportion of transitions from one stateto another given the total amount of switches from that specific state. One way of doingthis is by using the non-parametric chi-square test.

3.6.1 Introduction

The chi-square (χ2) test is used to compare two proportions by calculating frequencies.The non-parametric version of it does not assume any underlying distribution in contrastto the parametric one that assumes a binomial distribution. The non-parametric chi-square test either rejects or accepts the null hypothesis.

3.6.2 DefinitionFor two proportions, the chi-square test estimates the pooled proportion

P =n1 + n2

N1 +N2

(3.6.1)

where n1 and n2 are number of times this particular event happened and N1 and N2 aretotal number of events. The expected outcome ne is then calculated as

ne1 = N1Pne2 = N2P.

(3.6.2)

and two matrices constructed

O = [n1 N1 − n1 n2 N2 − n2]E = [ne1 N1 − ne1 ne2 N2 − ne2]

(3.6.3)

where O are the observed values and E are the expected ones. The chi-square value isthen calculated as

χ2 =4∑i=1

(O[i]− E[i])2

E[i](3.6.4)

and it is a measurement of how big the difference is between the two proportions. Theobtained χ2 is compared to a chi-square distribution table in order to get the p-valuewhich is used to test the null hypothesis. (McClean 2000)

13

3.7 Continous-time Markov ProcessThis section is introduced to show how a CTMP works and its applications. A continuousMarkov chain is a mathematical system that takes values in from a finite set of data. Itanalyses the time spent in each state before moving on to the next one and can only handlenon-negative real values. Same as the discrete-time process of order 1 it only dependson the current state and not historical behavior. Kai Lai Chung (Chung 2000), JacquesJanssen and Raimondo Manca (Janssen & Manca 2006) provided a good introduction tothis topic that created the foundation for this section.

3.7.1 IntroductionA continuous-time Markov process is a random walk (Xt : t ≥ 0) with m possible states,m ∈ N , and where I defines the sets of possible states with I = [1, 2, . . . ,m]. Supposethat the process starts in state space i1 ∈ I and stays there for a random time T1 > 0and jumps to a new state i2 6= i1, i2 ∈ I, where it stays for a random time T2 > 0 beforeit goes to yet another state i3 ∈ I and so on. The process has the following property:for a system S it has the waiting times t1 < t2 < ... < tk, with its associated statesX(t1), X(t2), ..., X(tk),

P [X(tn) = i|X(t1), X(t2), ...X(tn−1)] = P [X(tn) = i|X(tn−1)], i ∈ I. (3.7.1)

and if (3.7.1) holds the random sequence is homogeneous in time:

P [X(t+ s) = j|X(s) = i) = P (X(t) = j|X(0) = i), ∀t, s ∈ R. (3.7.2)

3.7.2 The Q-matrixThe probability of jumping in small intervals of time h > 0 can be defined by the function

f(h) = P (X(t+ h) = j|X(t) = i), i, j ∈ I (3.7.3)

and with f ′(0) = λij we have

limh→0

P [X(t+ h) = j|X(t) = i]

h= lim

h→0

f(h)− f(0)]

h= f ′(0) = λij. (3.7.4)

For small values of h the equation f(h) can be expressed as

f(h)− f(0) = λij +O(h) (3.7.5)

where O(h) is a function with the property that limh→0O(h)h

= 0 and since for all i 6= j,the above equation can be expressed as,

P [X(t+ h) = j|X(t) = i]− P [X(t) = j|X(t) = i]

h= λij +O(h). (3.7.6)

14

If we apply limh→0 on equation 3.7.6 we obtain f ′(t) = λij, for all j 6= i. If i = j,

P [X(t+ h) = i|X(t) = i] = 1−∑i 6=j

λijh+O(h). (3.7.7)

The probability to go to the same state will be determined by λii = −∑

i 6=j λij whichgives us,

P [X(t+ h) = i|X(t) = i] = 1− λiih+O(h). (3.7.8)

The Q-matrix is now defined as Q = [λij]M∗M , for all i, j ∈ I, and it can be shown thatf ′(t) = λii and the matrix has the following properties:

• λii ≤ 0, ∀i ∈ I.

• λij ≥ 0, ∀i, j ∈ I•∑

j∈I λij = 0, ∀i ∈ I

λij is known as the instantaneous rate from i to j with the assumption that it is constantthrough time.

3.7.3 Exponential time and Kolmogorov equationsAssume a random time V and if the system have not jumped after that time the remainingtime will have the same waiting time distribution as from the beginning. The randomtime V satisfies the memory less Markov property,

P [V > t+ h|V > t,X(t) = i] = P [V > t+ h|X(t) = i] = P [V > h],∀i, j ∈ I, t ≥ 0,

(3.7.9)

with an exponential waiting time distribution P [V > t] = e−λt and instantaneous transi-tion rate λ, it can be shown that

P [V > t+ h|V > t] =e−λ(t+h)

e−λt= e−λh = P [V > h]. (3.7.10)

The Kolmogorov backward and forward equation is,

P ′(t) = QP (t) and P ′(t) = P (t)Q (3.7.11)

Both equations have the unique solution P (t) = eQt where,

eQt =∞∑k=0

Qktk

k!(3.7.12)

15

Substituting by the spectral decomposition of Q = CDC−1, D is the diagonal matrix ofeigenvalues of Q and C is the matrix by eigenvectors of Q.

Qd =

↑ ↑ ↑c1 c2 · · · cd

↓ ↓ ↓

λ1 0

λ2. . .

0 λd

← c1 →← c2 →

...← cd →

(3.7.13)

∞∑k=0

(Qt)k

k!= C

∞∑k=0

Dntn

n!C−1 = CeDtC−1. (3.7.14)

3.7.4 ExampleSuppose that we have a two state matrix Q with the following properties,

Q =

(−λ1 λ1λ2 −λ2

)(3.7.15)

We use the Kolmogorov forward equation to calculate P see equation 3.7.11.

p′11(t) = −λ1p11(t) + λ2p12(t)

p′12(t) = λ1p11(t)− λ2p12(t)

p′21(t) = −λ1p21(t) + λ2p22(t)

p′22(t) = λ1p21(t)− λ2p22(t)

(3.7.16)

From equiation 3.5.4 we know that,

∑i∈I

pi = 1 = p11 + p12, i = 1, 2 ∈ I (3.7.17)

If we combine equation 3.7.16 and 3.7.17 we get,

p11 =λ2

λ1 + λ2− 1

λ1 + λ2p′11(t) (3.7.18)

The solution of equation 3.7.18 can be solved with an exponential e,

p′11(t) = e−(λ1+λ2)t (3.7.19)

16

With the same solutions for the other equations we obtain the following result for p11(t),p12(t), p21(t), p22(t),

p11(t) = λ2λ1+λ2

− 1λ1+λ2

e−(λ1+λ2)t

p12(t) = λ1λ1+λ2

+ 1λ1+λ2

e−(λ1+λ2)t

p21(t) = λ2λ1+λ2

+ 1λ1+λ2

e−(λ1+λ2)t

p22(t) = λ1λ1+λ2

− 1λ1+λ2

e−(λ1+λ2)t

(3.7.20)

If set the time t→∞ we find the stationary states,

( λ2λ1+λ2

λ1λ1+λ2

λ2λ1+λ2

λ1λ1+λ2

)(3.7.21)

The final solution shows that equation 3.5.4 holds,

( λ2λ1+λ2

+ λ1λ1+λ2

λ2λ1+λ2

+ λ1λ1+λ2

)=

(λ2+λ1λ1+λ2λ2+λ1λ1+λ2

)=

(11

). (3.7.22)

3.8 Bayesian estimation testingThe Bayesian Estimation Supersedes the T-test (BEST) algorithm was chosen becauseit is very flexible and have a few other advantages compared to simpler methods such asthe t-test. This section is introduced to show how Bayesian data analysis works and itsapplications(Kruschke 2013).

3.8.1 IntroductionThe BEST-algorithm can incorporate different types of distributions depending on theamount of outliers and the algorithm have a built in function that smooths the data andhandle outliers. Real world problems often have big differences between sample size andvariance which can cause major problems and effect the result but the BEST-algorithmcan handle this and uncertainty in those estimates. It reveals a lot of information aboutthe data such as difference in mean and standard deviation after smoothing, but also howthe data is distributed in relation to outliers. The method can handle null values andkeep the certainty in the estimation high. It shows the effect size which is an measurethat describes the magnitude of the difference between two groups.(Kruschke 2013)

3.8.2 Variable distributionsAs mentioned earlier real world data often contains outliers. These outliers can have asignificant meaning to the result and should be included in the distributions. A nor-mal distribution would have difficulty handling these where the BEST-algorithm usest-distribution with variable width. If the data contains outliers tv → 0 the distribution

17

will have taller tails see figure 3.3. With compact data without outliers the t-distributionwill look more like an normal distribution tv →∞ (see Figure 3.3).

Figure 3.3: T-distribution with different range of tv (Kruschke 2013).

3.8.3 Mean and standard deviationThe mean and standard deviation of the data reveals information of change between twogroups of data. The BEST-algorithm uses the most fitting distribution which will affectthe certainty of both mean and standard deviation. Depending on what distribution thathave been chosen everything is calculated with an 95 % confidence. Every histogramis marked with an 95 % highest density interval (HDI), which shows where the bulk ofmost credible values falls. The definition tells us that every value inside the bulk have anhigher probability density than any value outside the HDI, and the total mass of valuesinside is 95% of the distribution.

To better understand the difference two cases will be presented, one with moderate samplesize and one with an small group of sample size. The first case consider two groups ofpeople N1 = 47 and N2 = 42.

Figure 3.4c shows that 95% HDI falls above zero where 98,8 % of the credible values aregreater than zero. This means that they are credibly different and that it is an significantchange in means between the groups. The second case considered a case of small samplesize with N1 = 8 and N2 = 8 and even if the means are different of the two groups, theposterior distribution reveals great uncertainty in the estimation. This because zero fallswithin the 95% HDI and it cant be sure that an actual change have accoured betweenthe two data sets see figure 3.5c. The standard deviation is calculated in the same wayand presented with an 95% HDI.

3.8.4 Effect sizeThe effect size is a measure of the magnitude of how many standard deviations thatseparates two groups of data. It can be calculated with the difference in mean betweentwo groups (µ1−µ2) divided by the standard deviation (σ) of the population from which

18

(a) Distribution of µ. (b) Distribution of µ.

(c) Distribution of difference in µ.

Figure 3.4: Distributions of µ and difference of µ between group 1 and 2 in case 1(Kruschke 2013).

19

(a) Distribution of µ. (b) Distribution of µ.

(c) Distribution of difference in µ.

Figure 3.5: Distributions of µ and difference of µ between group 1 and 2 in case 2(Kruschke 2013).

20

it is sampled. There is now at least three different calculations with various advantages.They are referred to as Cohen’s d, Glass’s δ and Hedges’ g,

Cohen′s d =µ1 − µ2

SDpooled

=µ1 − µ2√∑

(X1−µ1)2+∑

(X2−µ2)2n1+n2−2

(3.8.1)

Glass δ =µ1 − µ2

σ2(3.8.2)

Hedges g =µ1 − µ2

SD∗pooled=

µ1 − µ2√(n1−1)σ2

1+(n2−1)σ22

n1+n2−2

. (3.8.3)

The only difference in the equations is the method for calculating the standard deviation.If they are roughly the same it is reasonable to expect that they are estimating a com-mon population standard deviation. This would favor Cohen’s d equation and poolingthe standard deviation would be appropriate. If on the other-hand the standard devia-tion differ it wouldn’t be appropriate to pool because of the violation of homogeneity invariance. Then Glass’s δ function could be used where Glass argued that the standarddeviation of the control group is untainted by the effects of the treatment and will there-fore more closely reflect the population. This is in direct relationship with the samplesize where more samples would resemble the population more likely. The last approach,Hedges g formula is recommended if the groups have different sample size and weightedstandard deviation is appropriate. (Ellis 2010)

While it could be appropriate to weight the standard deviation there is a perspectivethat the effect size is merely a re-description of the posterior distribution. Many differentdata sets could have generated the posterior parameter distribution and because of thatthe data should not be used in re-describing the posterior. (Kruschke 2013)

µ1 − µ2√σ21+σ

22

2

(3.8.4)

This form equation 3.8.4 is merely a copy of Hedges g formula without the weighting andit doesn’t change the sign or the magnitude in any greater extent. The distribution ofthe effect size that is greater or less than zero remains unaffected (Kruschke 2013).

One rule of thumb for interpreting the effect size was proposed by Cohen (1988) wherehe said that a ”small” effect size is above 0.2, ”medium” 0.5 and a ”large” above 0.8.However, Cohen did also warn that the rule can differ between fields of study and that theuser should define their own thresholds depending on purpose and area. (Cohen 1988)

3.8.5 Region of Practical EquivalenceThe posterior distribution can be used to make a decision about the credibility of the test.The way to do this is with the 95 % HDI, the range of the ROPE, and the value zero. Ifthe 95 % HDI is fine enough and falls entirely within the ROPE it means that with 95

21

% credibility the values are practically equivalent to each other and the null hypothesiscan be accepted (Kruschke 2013).

(a) Distribution of difference in µ. (b) Distribution of difference in σ.

Figure 3.6: Distributions of difference in µ and σ group 1 and 2 in case 3.

The HDI gets narrower with increased data sets so that sampling noise is easier cancelledout. It is important to notice that if the range of the ROPE is relatively big compared tothe width of the 95 % HDI it can exclude zero and still accept the null value just becauseit is within the range of the ROPE. In practical terms it means that the credible valuesare nonzero but they are so small that there isn’t any practical importance. As thresholdsvary between different fields the range of the ROPE have to be determine specifically toevery area.

For the Bayesian algorithm to accept the null value it is important that both data setslooks similar and have enough samples. This third case contains N1 = 1101 and N2 =1090 samples. Both have similar mean and standard deviation µ1 = −0.000297, µ2 =0.00128, σ1 = 0.986 and σ2 = 0.985. The difference in mean falls within the ROPEwith the 95% HDI (see (3.6a)) and difference in standard deviation includes 100% of thedistribution (see (3.6b)). This means that there is no doubt that both data sets are verysimilar and that no change could be detected between them.

3.8.6 Selecting sample sizeThe ability to detect effects has a direct correlation with the number of samples. Inmany cases it is fair to say that the success or failure of a project to reach an statisticalsignificant result hinges on it sample size. Let say if simulations was running to detectan particular effect size and the simulation is based on to to few samples, statisticalsignificance could occur randomly. E.g. if you expect the difference between two groupsto be equivalent to an effect size of d = 0.20, and you wish to have at least an 85% chanceof detecting this difference, you will need at least 900 participants in your sample. As thiseffect size relates to differences between groups, the implication is that you will need aminimum of 450 participants in each group. If you wish to further reduce the possibilityof overlooking real effects by increasing power to 0.95%, you will need a minimum of1302 participants or 651 in each group see table 3.2. So it is usefull to have an idea ofthe sensitivity of the research design where the risk only can be reduced by increasingsamples. Minimum sample sizes for detecting a statistically significant difference betweentwo groups is presented in table 3.2.(Ellis 2010)

22

Table 3.2: Minimum sample sizes for different effect sizes and power levels. Note: Thesample sizes reported for d are combined N1 +N2 (Ellis 2010).

Powerd 0.75 0.80 0.85 0.90 0.95

0.10 2779 3142 3594 4205 52000.20 696 787 900 1053 13020.30 311 351 401 469 5800.40 176 199 227 265 3270.50 113 128 146 171 210

23

Chapter 4

Work Flow, Technology and Data

This chapter will explain the work flow, the technologies used and the data behind ourresults and conclusions. It will also give the reader a better understanding of the resultsand connecting it with the theory. The work flow describes the process step by stepand gives a detailed description of what every step involved. The technology section willgive a short introduction to the technologies used in this thesis and state why they werechosen. The last section of this chapter will give a description of the data which theresults are based upon.

4.1 Work FlowThe project can be divided into two main steps:

Step 1: Gather and structure data:

Identify events related to UERC

Identify relevant event fields

Pre-processing of the data

Step 2: Analyse the data:

{Discrete-time analysis

Continuous-time analysis

The first step will be to go through the data and find which events that can be useful.When the desired events have been identified the ASN.1 parser will extract them fromthe raw data and create call sequences. The second step is to apply our theory to thegathered data and try to get as much information out of it as possible.

4.1.1 Identify events and create call sequencesA major part of the thesis have been trying to understand the WCDMA standard, collectthe right type of data and merge to consistent sequences of events. The WCDMA stan-dard is complex and has increased in complexity to cater for more advanced use-casesover time when more and more advanced end user terminals has been introduced intothe market. For Ericsson to maintain a high performance in the network new functionshave been developed and released in software updates. This increased complexity of thenetwork has made the collection of call sequences very difficult and for some parts almost

24

impossible.

Knowledge Acquisition:

• Reading up on related work

• Reading internal Ericsson manuals

• Collecting information on the internal Ericsson web

• Attending presentations at Ericsson

• Several meetings with employees at Ericsson

4.1.2 Pre-processing of the dataBefore the data can be used it needs to be pre-processed and structured in a way to makeit compatible with our analysis-tools. The data is originally in a binary format but thereexist a decoder that turns it into ASN.1 formatted file. As there are not any suitableopen source decoders available for ASN.1 we need to construct our own. The goal of thepre-processing is to get two comma seperated files (csv) where

1. Each line represent a sequence of UERC ids that all belongs to the same UE andwas recorded during the same session

2. Each line is a switch with source and target UERC and a timestamp

The holding time is then calculated with the timestamp for entering and leaving theUERC. The csv format is very practical when working with large data sets and manysoftware supports the format.

4.1.3 Discrete-time analysisThe discrete-time analysis only considers switches between UERCs, meaning that thetime it stayed in the UERC is not taken into account. The sequence analysis for discrete-time follows the following steps:

1. Construct n-grams for n=1, 2, 3, . . .

2. Create P-matrix for DTMP based on the 2-gram

3. Draw Markov chain with P-matrix

4. Compute and draw normalized sequences

5. Do steps 1-4 for second part of data

6. Compare the P-matrices with chi-square test

7. Reject or retain the null hypothesis

The Markov chains are drawn with the R library igraph and the n-gram program fromthe library tau.

4.1.4 Continuous-time analysisIn continuous-time it is the holding time that is analysed. This would be the actualtime each UE spends in every UERC before moving on to a new one. The holdingtime is both deterministically specified in the system and by the behaviour of the user.

25

These holding times have been analysed with the continuous-time Markov Process andthe BEST-algorithm.

Material have been collected in databases with high-ranking publications, the databaseswhich provided the BEST information was:

• IEEE Xplore Digital Library

• ACM Digital Library

The programing environment used to develop the CTMP is MATLAB. The sequenceanalysis for continues time follows the steps

1. Construct the Q matrix

2. Use the Kolmogorov equation to calculate P

3. Solve the Kolmogorov equation with an exponential e

4. Plot the probability distribution in time and evaluate

The BEST-algorithm on the other hand was developed in R and came as a ready to useproduct in an R library called BEST. This specific library needed the following packagesto be installed:

• JAGS

• rjags

To be able to define the range of the ROPE for the effect size, data from part one wasshopped up in batches of 2600 samples. The number of samples was chosen in relationto the simulation time and maximum chance to detect a statistical significant difference(see Table 3.2). These batches of data was compared to each other and because part onedid not include any known changes the corresponding magnitude of the effect size couldbe treated as no change at all.

4.2 TechnologiesThis project requires the use of several different programs and technologies on boththe Linux and Windows environment. Some of them have been developed internally atEricsson but the ones most used are either an open-source or a commercial software. Forthe data gathering and pre-processing a Linux lab computer with 8 processors will be usedfor its superior computational power. The data used in this project was small enough toanalyse on a standard PC, otherwise it would have been more efficient to do everythingon the LAB computer. The primary technologies are the programming language Python,the commercial high-level language and environment MATLAB and the free high-levellanguage and environment R.

4.2.1 PythonPython is a powerful and versatile programming language that can be used for bothobject oriented programming as well as functional programming. It has a rich standardlibrary that supports many of the most common programming tasks such as connectionto web servers, searching text and more. One of its main features is that its easy to read

26

because Python does not end each line by a semicolon or use brackets to define codesections, instead, Python uses a new line and indents. (Holden 2014)

Python is an excellent scripting language and has large support for searching in text.That, and the ability to mix object oriented programming with functional, makes Pythonthe perfect language to write the ASN.1 parser in.

4.2.2 MATLABMATLAB is an abbreviation of MATrix LABoratory and as the name suggests its partic-ulary good at manipulating matrices. It is a high-level language developed by Mathworksthat requires a license to use. By offering it for free to Universities and similar institu-tions it has become widely used as an research and learning tool. Its primary functionis numerical calculations but there exist several toolboxes that allows for more specificapplications, such as the Statistal Toobox which includes many pre-written methods fromdata mining and machine learning. (The MathWorks 2014)

At Ericsson, MATLAB is a common tool for analysing the GPEH data, mostly for drawinggraphs based on certain KPIs. With an extensive toolbox for Statistics and the closeconnection to GPEH data it was natural that MATLAB would play a part on this project.

4.2.3 RR is used for statistical computing and graphics. It is a free, high-level programminglanguage and it is very popular within data mining and machine learning. R is a GNUProject 1 and it is available on UNIX, Windows and MacOS. (R-project n.d.)

R was specifically constructed for solving problems like the ones this project faces. It isa widely used tool for data mining and it contains packages for calculating η-grams andconstructing Markov chains.

4.3 GPEH Test DataThe offline test data is from a major sport event in Sweden. The data is often parsed fromGPEH to a MATLAB file. The benefit of collecting data from sport events is becauseusers tend to stay in one place for a long time. This means that a very high amount ofusers are connected to a few cells for a longer time and its easier to follow their mobiletraffic. This is very critical because users that move around constantly change RBSsand this would increase the difficulty to keep track of the right user. There is also abenefit in having a high amount of users in one concentrated place, live stress tests ofthe network can be performed. This is important to analyse and be able to understandhow the network could perform in peak hours. As most games have period breaks theinteresting part of the the analysis happens during these breaks where most users startto use their mobile phones. The network operator and Ericsson usually have an agendafor every recording and changes settings in the network after half-time. This is done toevaluate and tweak the system for better performance.

1A GNU Project means that its free software, available under Free Software Foundation’s GNUGeneral Public License.

27

The two data sets used for comparison in this report actually comes from the same sportevent where some settings were changed in a period break. The data will be divided intotwo parts; where part 1 will consist of the data before the change and part 2 after thechange. Methods from the sequence analysis part of the theory will be applied to the twodata sets in order to compare them with hope of identifying the changed settings.

28

Chapter 5

Results

The results chapter will cover the findings regarding the investigation of the possible useof UERTT, the identification of the events needed to create UERC sequences and anexplanation of how the ASN.1 parser that extracts the events work. The chapter willalso show the results from analysing two different GPEH-files in order to find changesregarding UERC switches.

5.1 User Equipment Real-Time TraceThe first task was to determine whether or not UERTT could be used to analyse thenetwork with data mining and machine learning methods. The UERTT works by trackinga certain UE in the network on the OSS and then stream the events connected to thatUE to a port that can be listened to on another machine.

5.2 Detecting Events and SequencesThe first step in creating UERC sequences was to find which events contained relevantinformation and to assign each event to a specific UE. This proved a lot harder than firstimagined. Some events are difficult to match to a certain UE since they do not carryan IMSI number, they only contain the fields rnc-module-id and ue-context to identifywhich UE it should belong to.

Figure 5.1 shows the logic behind an RNC and how it keeps track of all the UEs andtheir events. Every RNC contains several modules which in their turn handles severaldifferent UEs. Every UE have a corresponding list of events which, among other, tells

RNC

Module 1Module 2...Module m

UE 1UE 2...UE n

[Event 1, Event 2, . . . , Event k]

Figure 5.1: The logic behind a RNC, its modules and UEs, and their assiciated events.

29

in which order the UE has moved between different UERCs. The events that contain anIMSI number can easily be matched to a specific UE but the ones that do not containit have to matched by their rnc-module-id and ue-context. At every instant of time, theue-context is unique for every module but it is however reused which means that whena connection is dropped for one UE a new one will take its place and have the sameue-context as the previous one. This problem was solved by finding start and end eventswhich determines when a new call sequence starts and ends respectively.

Table 5.1: The events with id and description that is needed to create UERC sequences.

Event id Name Description415 internal-rab-establishment Establishing a connection to a RAB416 internal-rab-release Releasing a connection to a RAB387 internal-channel-switching The RAB connection has been reconfigured

(the UE switched to another UERC)19 rrc-connection-release The connection with the RNC has been re-

leased (UE has gone to idle)438 internal-system-release One or several RABs, or a standalone RNC,

have been released but the release was not anormal one

After consulting several employees at Ericsson and by analysing the events by hand fivedifferent events could be identified needed to create the UERC sequences that were uniquefor every UE. The identified events can be seen in Table 5.1 with their id, name and ashort description.

A major part of the identification of the desired events was by examining which fieldsthey contained. As mention in section 2.2, there are some fields in the events that arerequired in all events but most events also have its unique fields.

Table 5.2: Relevant fields in the events.

Field name Descriptiontimestamp Date and time for when the event took placeevent-id Unique id for every eventrnc-module-id Id for the module that handled the eventue-context Id for the UE assigned by the modulecell-id The cell (RBS) that handles the UEsource UERC source of a channel switchtarget UERC target of a channel switchimsi Unique id for the UEexception Integer that tells if the event was successful or not

Table 5.2 displays the relevant field names. The first five fields; timestamp, event-id, rnc-module-id, ue-context-id and cell-id are all required fields for an event and also neededto create UERC sequences. The following fields in the table only accompany some of theevents, e.g. the fields source, target and exception are related to events regarding UERC

30

so they are not present in the event rrc-connection-release and the imsi number is onlypresent in the internal-channel-switching event.

5.2.1 ASN.1 ParserBased on the identified events required to create UERC sequences the ASN.1 parser couldbe developed to extract the events and their relevant fields from the large GPEH files.The parser was written in Python with ASN.1 formatted data as input (GPEH-files andUERTT from port) and two csv-files as output.

Listing 5.1: Pseudo code for the ASN.1 parser.

rnc = RadioNetworkControl ler ( )for l i n e in input do :

i f l i n e has ’ event id ’ do :ev en t i d = l i n e . g e t In t ( )

else :continue

i f ev en t i d not in wantedEventList do :continue

module id = l i n e . nextLine ( ) . g e t In t ( )timestamp = l i n e . nextLine ( ) . g e t In t ( )c e l l i d = l i n e . nextLine ( ) . g e t In t ( )i f l i n e . nextLine ( ) has ’ source ’ do :

soource = l i n e . g e t In t ( )t a r g e t = l i n e . nextLine ( ) . g e t In t ( )ims i = l i n e . nextLine ( ) . g e t In t ( )except ion = l i n e . nextLine ( ) . g e t In t ( )event = SwitchEvent ( event id , timestamp , c e l l i d )event . se tSwitch ( source , t a r g e t )event . se tExcept ion ( except ion )event . s e t Ims i ( ims i )

else :event = Event ( event id , timestamp , c e l l i d )

rnc . addEvent ( module id , ue id , event )endrnc . t o F i l e ( )

Listing 5.1 shows the main traits of the ASN.1 parser in pseudo code. Basically, it readsevery line separately in order and if it finds an event with an id that matches any of theevents in Table 5.1 the parser will continue to read the required fields and add that eventto the RadioNetworkController object.

In Figure 5.2 the work flow and object classes of the ASN.1 parser can be seen. Thework flow demonstrates how the parser is built from a higher perspective, with inputand output specified. The object classes are the implemented Python classes and thefigure also shows their dependencies towards each other. The parser is inspired by theRNC logic presented in Figure 5.1 when it handles the event. The main script (shown inthe parser flow under ”ASN1 Parser”) implements the pseudo code in Listing 5.1 and itsprimary function is to add events to the rnc object which in turn will assign them to therelevant module and then session (unique UE sequence).

31

Figure 5.2: Workflow and object classes of the ASN.1 parser.

5.3 n-grams and Markov ChainsFor both part of the data, n-grams were calculated for n=1,2 and 3, e.i. the unigrams,bigrams and trigrams respectively. The unigrams is basically just a list with how manytimes a certain UERC is visited while the bigrams tells how many times a switch betweentwo UERCs has happened. The trigrams adds one more dimension and here it is possibleto see which UERC is most likely to be next after a certain switch. Or the other way,given a certain state which switch is most likely to happen.

5.3.1 UnigramsThe unigram algorithm will count every occurrence of a specific UERC in all sequences.Table 5.3 shows the results for both part 1 and part 2 of the data sets. The percentageshow the proportion of a specific UERC occurrence compared to total number of counts.Although, the percentages differ some between the data sets the order of most commonUERC is the same. It appears that there are three dominant states; UERC 4, 21 and25 with an occurrence of over 93 % and 91 % respectively. Comparing the individualpercentages for these three dominants in the two data sets shows that they are almostthe same for UERC 25 while UERC 4 occurs less frequently and 21 more frequently inthe second part.

32

Table 5.3: Unigrams for data part 1 and part 2.

Part 1 Part 2UERC Counts % Counts %

4 88195 44.5 61923 41.2221 51078 25.77 40821 27.1825 45473 22.94 34252 22.81 5437 2.74 5119 3.410 4057 2.05 4402 2.9353 1511 0.76 1281 0.859 932 0.47 892 0.59

123 668 0.34 670 0.4515 400 0.2 481 0.322 252 0.13 169 0.11

438 115 0.06 133 0.0962 53 0.03 46 0.03113 14 0.01 3 069 4 0 6 052 3 0 4 067 N/A N/A 3 0456 2 0 2 019 N/A N/A 1 010 1 0 N/A N/A

It can also be seen in Table 5.3 that some UERC only shows up in one part and not theother. There are also several that occur so few times that their occurrence percentage isrounded to zero (less than 0.005 % occurrence).

5.3.2 BigramsThe tables for the bigrams are a lot longer than for the unigrams so they have been placedin the appendix and can be seen in Appendix B where the bigrams for part 1 is in Table7.3 and part 2 in Table 7.3. Due to the large number of bigrams a visual analysis is hard.As mentioned in Section 3.4, a bigram is equivalent to a Markov process of order 1. Thiswill be used to draw Markov chains for both part 1 and part 2 as well as comparing theirtransition matrices. The most frequent switches will also be analysed by drawing theirMarkov chains based on the so called normalized sequences.

Markov Chains

The Markov chains are based on the bigrams and represents switches between the differentUERCs in the network. Figure 5.3 shows the Markov chain for the first part of the dataset. The background grouping tells if the UERC is for voice (CS, sky blue) or data(PS, light red), or both. The arrows between the states represents occurred switches andarrows that go back to the same state are non-fatal errors (e.g. a UE tried to accessUERC 25 but it was full so it had to stay on its current UERC). The green state (SRB)represent UERC 1 which is the starting state of a sequences and the orange state (idle)

33

represent UERC 0, the end state. All other UERCs are represented by a blue coloredstate. The red states does not represent an actual UERC id but rather the event idassociated with the errors:

438: ’internal-system-release’ - indicates an error that was so severe the session had tobe terminated

456: ’internal-call-setup-fail’ - indicates a failed RNC connection setupMarkov chain part 1

421

251

53

123

9

15

2

62

113

52

69

10

0

438

456

CSPS

SRBUERCIdleError

Figure 5.3: Markov chain for part 1 data.Markov chain part 2

214

25 1

53

123

9

15

2

62

438

52

69

113 19

670

456

CSPS

SRBUERCIdleError

Figure 5.4: Markov chain for part 2 data.

Figure 5.4 shows the Markov chain based on the second part of the data. It follows thesame structure and at first glance they may appear quite similar. On a closer inspectionits clear that they differ on some vital points; there is some difference in observed UERCs(same as for unigrams) and some connections have been added/removed.

Transition Matrices

The transition matrices for the two parts can be calculated from the bigrams. A transitionmatrix can be seen as a summarizing of the bigram, displaying the observed probability

34

to jump to a specific UERC given you are in another. Figure 5.5 shows the transitionmatrices for both part 1 and part 2. It also displays the results from the chi-squareanalysis, which transitions that accepted or rejected the null hypothesis as well as thechi-square value for every transition. The matrice colors are normalized meaning thatthe colors depend on the value, a higher value will give a darker color and a value closeto zero will be almost invisible. This has been done to magnify the results and importantelements in the matrices.

(a) Transition matrix for part 1. (b) Transition matrix for part 2.

Figure 5.5: Transition matrices with source and target states.

(a) Null hypothesis test. (b) Chi-square values.

Figure 5.6: Matrices from the Chi-square analysis.

The matrices in Figure 5.5 represent the transition matrices for part 1 and part 2. Everyrow in a transition matrix sums to 1 which means that the color of every row will ’sum’to red. If a row contains many light red elements it implies that the source UERC doesnot have a dominant switching target.

35

Figure 5.6a shows if the null hypothesis was rejected or accepted for each element. Thetest was performed using the Chi-square method on each element from the transitionmatrices. A dark red color means that the test was rejected, a light red means it passedand a no color (white) means that the test could not be conducted (one or both had azero in the element).

The last matrix, in Figure 5.6b displays the Chi-square value, a measurement of thechange. Same as for the other matrices, the color is normalized meaning that it willhighlight large values and hide small ones, i.e. highlighting big changes and hiding small.

Normalized Sequences - Bigrams

These normalized sequences are based on the bigrams in Table 7.2 and Table 7.3 fromAppendix B. They represent the most common UERC switches that together make upmore than 90 % of the total number of switches. There are five frequent UERC ids; 0, 1,4, 21 and 25. The normalized sequences were used to draw the Markov chains in Figure5.7.

1

25

4 21

0

1.0

0.95

0.05

0.47

0.52

0.011.0

(a) Markov chain part 1.

1

25

4 21

0

1.0

0.76

0.08

0.160.48

0.50

0.021.0

(b) Markov chain part 2.

1 SRB

25 UEL/HS

4 FACH

21 URA

0 IDLE

Figure 5.7: Markov chains based on normalized bigrams.

There are some changes in the transition probabilities between 5.7a and 5.7b but themost obvious change is that the arrow (switch) between UERC 25 and 21 is not presentin the first part. As a consequence, the probability to switch from UERC 25 to 4 in thesecond part decreased drastically since it now had another way to go.

5.3.3 TrigramsThe tables for the trigrams can be seen in Appendix C. The trigrams for part 1 is inTable 7.4 and part 2 in Table 7.5. The trigram tables are very long and contain manysequences that only occurs a couple of times, much too few to be able to compare anddraw any conclusions from. For the most common sequences there are however a lot ofdata that can be analysed.

36

Normalized Sequences - Trigrams

For the trigrams, the normalized sequences are based on Table 7.4 and Table 7.5 inAppendix C. They are constructed under the same premises as the bigrams, consisting ofover 90 % of the observed trigrams. The Markov chains in Figure 5.8 are a bit differentfrom the a normal Markov chain since it is of the second order and depends on previousstate. For every state in the chain, there is only one other state that leads there. Thus,the probabilities of going to one state are depending on both where you are and whereyou came from, e.i. the probability to go from UERC 25 to 4 is 58 % if the previousstate was UERC 1 and 100 % if the previous state was UERC 4 (for the Markov chainin Figure 5.8a).

1

250

4 21

25 4

1.0

0.42

0.580.02

0.36

0.62

1.01.0

0.41

0.59

(a) Markov part 1.

1

250

4 21

25 4

1.0

0.56

0.29

0.15

0.02

0.38

0.60

1.00.840.16

0.58

0.42

(b) Markov part 2.

Figure 5.8: Markov process based on trigrams.

There are two more possible paths for the second part (Figure 5.8b) than for the firstpart (Figure 5.8a). The added sequences are 1−25−21 and 4−25−21 with a transitionprobability of 15 and 16 % respectivielty.

5.4 BESTThis section will primarily focus on UERC 4, 21 and 25 due to the call sequence analysisin table 5.3. The table shows that more than 90% of the visits in both part 1 and 2 wentthrough these states and that the others have to few samples to maintain an high powerlevel see table 3.2. UERC 1 have enough samples but contain to many deterministicallyset values and UERC 0 is an end state and will therefor not be evaluated. If the effect sizecan detect major changes a deeper analysis will be performed for that UERC. Figurespresented in this section are based on the BEST algorithm and will show mean (µ),standard deviation (σ) and effect size.

37

5.4.1 Define range of ROPE effect sizeTo measure the range of ROPE for the effect size in UERC 4, 21 and 25 data in part onewas chopped up in batches of 2600 samples. Five simulations was done for every UERC(see Table 5.4).

Table 5.4: Range of effect size.

UERC range4 21 25

-0.063 0.054 -0.016 0.146 -0.023 0.139-0.075 0.046 -0.023 0.132 -0.042 0.119-0.068 0.055 -0.010 0.162 -0.032 0.142-0.071 0.050 -0.020 0.147 -0.022 0.136-0.069 0.049 -0.009 0.149 -0.030 0.143

The result in Table 5.4 shows that an range for the effect size between [−0.20.2] would fitas limits for every UERC. This range is now used to accept or reject the null hypothesis.

5.4.2 UERC 4 - (FACH) and 21 - (URA)The BEST algorithm didn’t detect any major changes for UERC 4 and 21 which Figure5.9 and 5.10 shows. UERC 21 almost have twice as big effect size compared to state 4and the distribution is wider. The range for the 95 % distribution is 0.0421 compared toUERC 4 that have 0.0248. Both simulations had an very high amount of samples, whereUERC 4 had n1 = 80943 and n2 = 57092 and 21 had n1 = 43255 respectively n2 = 34225samples.

Figure 5.9: Distribution of effect size UERC 4.

5.4.3 UERC 25 - (EUL/HS)The distribution for the effect size is far away from the ROPE limits and the null hy-pothesis is rejected. The magnitude of change is approximately 1.4 standard deviationsshown in Figure 5.11.

There is an significant change in mean for UERC 25 with an increase of 2.08, whichis equivalent to 33 %. The data also appears to be more spread out in part two, thedistribution is wider and more uncertain which can be seen in Figure 5.12b and 5.13a.The simulation is based on n1 = 43271 and n2 = 30234.

38

Figure 5.10: Distribution of effect size UERC 21.

Figure 5.11: Distribution of effect size UERC 25.

5.5 Continous time Markov processThis section will focus on UERC 25 due to the findings in 5.4.3. The continuous timeMarkov process have an exponential distribution and Figure 7.1 in appendix D show allholding times for UERC 25 plotted against an exponential distribution. Most of the datacan be approximated except the outliers, 96 % of the data is covered.

The components used to construct the Q matrix was mean of holding time µi for eachstate and the P matrix from discreet time see Figure 5.5a and 5.5b. The transition ratesfrom the P matrix was used to weight µi for each state. Equation 5.5.1 is an definition

(a) Distribution of µ for part 1. (b) Distribution of µ for part 2.

Figure 5.12: Distributions of µ UERC 25.

39

(a) Distribution of σ for part 1. (b) Distribution of σ for part 2.

Figure 5.13: Distributions σ UERC 25.

of the Q matrix.

Q = µi ∗ pij =

−µ1 µ1 ∗ p12 · · · µ1 ∗ p1n

µ2 ∗ p21 −µ2 · · · µ2 ∗ p2n...

.... . .

...µm ∗ pm1 µm ∗ pm,2 · · · −µm

(5.5.1)

The properties for the Q matrix holds (see theory part 3.7.2), every row sums up to zero,diagonal element are less than zero and all the other elements are above zero.

When the construction of the Q matrix worked with the theory every state was plottedin time to see how the transition rates changed. As mentioned earlier will this thesismake an deeper evaluation of UERC 25 and Figure 5.14 shows the probability densityfrom zero up to fifteen seconds. The mean holding time for part one was µ25 = 9.19 andµ25 = 12.98 for part two. All transition rates sums up to 100 %. For the first 15 secondsits an higher probability to stay in the source state. The intersection between UERC 21and 4 happens earlier in the second part compared to the first.

40

(a) UERC 25 part 1. (b) UERC 25 part 2.

Figure 5.14: Probability to go to another state from UERC 25 over time.

41

Chapter 6

Discussion

In short, this thesis project describes a method to create user specific sequences of eventsfrom ASN.1 formatted data (GPEH or UERTT) in a mobile network. The thesis alsodemonstrates a methodology to detect changes between date sets based on these se-quences. The derived methodology to detect changes between data sets was implementedon two GPEH-files where actual changes had been made to the network. The resultsfrom the analysis was that two major changes could be identified in the second part ofthe data. The changes was:

1. Switching from UERC 25 to UERC 21 had been enabled

2. The holding time in UERC 25 had been increased

After verifying with employees at Ericsson, it could be concluded that the findings areconsistent with actual changes to the network settings.

6.1 ASN.1 parserThe ASN.1 parser fulfill its purpose but its very time consuming to parse the large GPEH-files. The interesting part is rather how it works; which event are relevant and which fieldsneed to be stored in order to create a sequence of events specific to a unique UE. Althoughit works acceptable well, the methodology is not perfect and there are several holes inthe sequences after the parser which can not be explained. It appears that the problemis within the GPEH-files rather then with the constructed parser, a theory supported bythe same findings using other methods.

6.2 Sequence AnalysisExcept constructing UERC sequences, the main goal of the thesis project was to anal-yse the sequences and compare two different data sets in order to find any statisticaldifferences between them. The analysis was performed by looking at the sequences indiscrete-time and finding patterns as well as analysing the holding times of some of themore frequent UERCs.

42

6.2.1 UERC Switching PatternsPatterns in the sequences was identified by the n-gram algorithm which proved to be verysuitable for this purpose. Based on the unigrams, bigrams and trigrams, Markov chains,transition matrices and normalized sequences could be constructed and used to identifychanges in the sequences. The unigrams were only analysed by inspection mostly becausethey were not believed to be able to contribute anything new.

The Markov chains which were based on the bigrams serves as an illustrative method tosee how different UERCs are connected to each other but their complexity makes it hardto draw any conclusions from. Although there are some noticeable differences in the twoMarkov chains, such as new states, the bigrams show that there are not enough dataprovided for these states to draw any conclusions from.

Analysing the transition matrices with the Chi-square method based on the null hypoth-esis (see Figure 5.6a) reveals many switches that rejects the null hypothesis. With 99% confidence it could say that there have been a significant change in these switches.However, there is a difference between a significant change and a practical change. Whenlooking at the Chi-square values in Figure 5.6b it reveals that the changes are of verydifferent magnitude. The biggest changes, by far, were in the switches 25 to 4 and 25 to21.

The normalized sequence analysis was introduced to provide a more standardized struc-ture of the network and find the essential behaviour. It showed that just five states couldrepresent more than 90 % of the UERCs being used and the model clearly shows howswitches from UERC 25 to 4 are decreased in favor of switches to 21.

The same conclusion can be drawn based on the trigrams and due to the extra dimensionit seems like the enable switch from 25 to 21 does not depend on where the UE camefrome.

6.2.2 UERC Holding TimesThe continuous time analysis proceeded with three different states, the states that stoodout and accounted for 93.21 % of all the visits in part one and 91.2 % in part two wasUERC 4, 21 and 25. As the goal for the thesis is to find a statistical change between twodata sets, determining the threshold for an actual change is critical. This limit have totake into account both deterministic values in the system as well as values depending onthe user.

Theoretical and practical changes can sometimes differ where an small and significantchange not always have an practical importance. The limit for an practical change wasstated throughout several simulations where the range of the ROPE ended up going from−0.2 to 0.2 for effect size. An closer look at table 5.4 shows that an range from −0.15 to0.15 also could have worked. This thesis used the same limit for every state, but UERC4 would have fitted in between −0.1 to 0.1. The rule of thumb mentioned earlier in thetheory chapter treated an effect size of −0.2 to 0.2 as small (Cohen 1988). And even ifthat rule was in the same area, no major conclusion could be done and no efforts wasmade to define a small, medium or large change, this due to the complexity to actuallymeasure this.

43

Both UERC 4 and 21 accepted the null hypothesis and no practical change could bedetected, 100 % of data is within the ROPE limit. UERC 25 on the other hand rejectedthe null hypothesis, 100 % of the data was outside the ROPE limit. The effect sizemeasured an change of approximately 1.4 standard deviations which was seven timesbigger than the limit. Clearly have something happened, users stay longer in state 25.The threshold for maximum occupation time has changed, it is higher and the magnitudeof the change is calculated with an 95 % probability to be between 2.05 - 2.12 secondswhich Figure 7.6a shows. The increased time have to be deterministic, meaning that thesystem have changed settings. It is still the same amount of users in the network, peopleusually stay the whole game.

As UERC 25 had an major change in the holding time, it could be interesting to see whathappens with the state transition probabilities over time. Figure 5.14 shows how userstend to stay longer in the source state before moving on to another one. This correspondswell with what was discovered using the BEST algorithm. Comparing the mean holdingtime µ25 = 9.19 and µ25 = 12.98 in part one and two with the transition rates in Figure5.14a and 5.14b shows that they both have an probability of approximately 50 % to stillbeing in in the source state which makes perfect sense. The BEST algorithm uses at-distribution, comparing mean shows an big difference where BEST calculated a meanof 6.26 (6.24 - 6.28) compared to 9.19 and 8.34 (8.32 - 8.37) compared to 12.98 seconds.Live data contains outliers and the transition rates calculated with the Markov processcan therefore include some errors.

44

Chapter 7

Conclusion

The major findings in this thesis are not the identified changes in the two different datasets - they were already known. That part is a proof of concept, to demonstrate thatsequence analysis can be applied to find changes in GPEH data in general and UERCsequences in particular. The important parts are the identification of the events neededto create UERC sequences, the methodology to put the events together in a unique UEsequence, and the theory for analysing these sequences. This thesis only focused onthe UERC related events but the methodology could easily be widened to include otherevents.

7.1 Pre-processingThe pre-processing of the data proved to be a lot more difficult than first imagined andin the end it had taken up half of the entire thesis work. Partly because the initial planwas to only use UERTT data but when this proved impractical a decision was made tochange to GPEH data instead. It also proved hard to find when a new UE is assignedthe same ue-context id as a previous one. There were also a problem with missing events(the UERC switches did not make sense, e.g. an UE going to UERC 4 but in next eventit said it is in UERC 21) in the analysed GPEH files and a quick study showed thatapproximately every tenth event was missing, leading to many sequences being thrownaway due to the holes, over half of them actually. This was considered too many to beignored and it was decided to manually add the event presumed missing in order to getmore sequences to analyse.

The missing events could be due to an error in our method, e.g. a missed event thatwould have explained the gap, it could also be specific to these GPEH-files or due tothe logging in the OSS. A lot of time was spent trying to explain the missing events byboth us as well as employees at Ericsson but no conclusions could be drawn to why therewere missing events. In order to continue with the thesis project this question were leftunanswered. Hopefully, the manually added events will not affect the analysis part sincethey are discarded in the BEST and CTMP analysis and for the sequences analysis everyadded switch were under a ’best guess assumption’, e.i. based on logic and experiencefrom Ericsson’s employees.

45

7.2 Sequence AnalysisBayesian statistics proved to be a very efficient and effective tool comparing two differentdata sets with holding times. Outliers and different sample sizes was handled in ancredible way with very extensive and detailed results. For further research could it beinteresting to try different distributions, the t-distribution can handle outliers which isvery good but it can also contain negative values. The BEST algorithm that is usedin this thesis could handle both the t-distribution and log-normal distribution. As thelog-normal distribution never include negative values it could be interesting to try thatand see how the results can differ. Analysing UERC sequences with the BEST algorithmis very time consuming, these sequences tend to blow up and become very large. The sizewas between 30 - 80 000 samples and it is strongly recommended to use a cluster withmultiple cores to be able to shrink the simulation time.

The continuous time Markov process showed some interesting results of the transitionrates but they have to be treated with some skepticism. The exponential distributioncant handle outliers which makes the results very uncertain. The Markov process cantchange the distribution because it is a part of the solution, so an recommendation forfurther research would be to find another algorithm that can track transition rates intime with different distributions.

46

Bibliography

Chung, K. L. (2000), ”A Course in Probability Theory, Third Edition”, Academic Press,San Diego, USA.

Cohen, J. (1988), ”Statistical Power Analysis for the Behaviour Science”, Lawrence Erl-baum Associates, Hove and London.

Ellis, P. D. (2010), ”The essential guide to effect sizes : statistical power, meta-analysis,and the interpretation of research results”, University press Cambridge, CambridgeUnited Kingdom.

Eri (2002), WCDMA RAN Protocols and Procedures, r1a edn.

Ericsson (2014), ‘WCDMA Radio Access Network ’, http://www.ericsson.com/

ourportfolio/products/wcdma-radio-access-network. [Online; accessed 2014-05-08].

Google (2012), ‘Protocol Buffers Overview ’, https://developers.google.com/

protocol-buffers/docs/overview. [Online; accessed 2014-05-13].

Holden, S. (2014), ‘BeginnersGuide/Overview ’, https://wiki.python.org/moin/

BeginnersGuide/Overview. [Online; accessed 2014-05-04].

Janssen, J. & Manca, R. (2006), ”Applied Semi-Markov Processes”, Springer, New York,USA.

Kruschke, J. K. (2013), ‘”Bayesian Estimation Supersedes the t Test”’, Journal of Ex-perimental Psychology 142(2), 573–603.

Manning, C. D. & Schutze, H. (1999), ‘”Foundations of statistical natural language pro-cessing”’, Ed. Hinrich Schutze. MIT press .

McClean, P. (2000), ‘The Chi-Square Test ’, http://www.ndsu.edu/pubweb/~mcclean/plsc431/mendel/mendel4.htm. [Online; accessed 2014-05-19].

Pedrini, L. (2013), ‘What is RRC and RAB ’, http://www.telecomhall.com/

what-is-rrc-and-rab.aspx. [Online; accessed 2014-04-28].

R-project (n.d.), ‘What is R? ’, http://www.r-project.org/index.html. [Online; ac-cessed 2014-05-04].

47

The MathWorks, I. (2014), ‘Key Features ’, http://www.mathworks.se/products/

matlab/description1.html. [Online; accessed 2014-05-04].

Trek, S. (2014), ‘What is Hypothesis Testing? ’, http://stattrek.com/

hypothesis-test/hypothesis-testing.aspx. [Online; accessed 2014-05-20].

Union, I. T. (2014), ‘Introduction to ASN.1 ’, http://www.itu.int/en/ITU-T/asn1/

Pages/introduction.aspx. [Online; accessed 2014-03-24].

48

Appendix A

Table 7.1: Table of RAB combinations and description.

UERC ID Description0 Idle1 SRB (13.6/13.6)2 Conv. CS speech 12.23 Conv. CS unkn (64/64)4 PS Interactive (RACH/FACH)5 PS Interactive (64/64)6 PS Interactive (64/128)7 PS Interactive (64/384)8 Stream. CS unkn. (57.6/57.6)9 Conv. CS speech 12.2 + PS Interactive (0/0)10 Conv. CS speech 12.2 + PS Interactive (64/64)11 SRB (13.6/13.6), pre-configured12 Conv. CS speech 12.2 , pre-configured13 Stream. PS (16/64) + PS Interactive (8/8)14 Conv. CS unkn (64/64) + PS Interactive (8/8)15 PS Interactive (64/HS)16 PS Interactive (384/HS)17 Stream. PS (16/128) + PS Interactive (8/8)18 PS Interactive (128/128)19 Conv. CS speech 12.2 + PS Interactive (64/HS)20 Conv. CS speech 12.2 + PS Interactive (384/HS)21 PS Interactive (URA/URA)22 Stream. PS (128/16) + Interact. PS (8/8)23 Conv. CS speech 12.2 + Stream. PS (128/16) + Interact. PS (8/8)24 Conv. CS speech 12.2 + Stream. PS (16/128) + PS Interactive

(8/8)25 PS Interactive (EUL/HS)26 2* PS Interactive (64/64)27 Conv. CS speech 12.2 + 2* PS Interactive (64/64)28 PS Interactive (128/64)29 PS Interactive (384/64)30 PS Interactive (384/128)31 PS Interactive (128/384)

49

UERC ID Description32 PS Interactive (384/384)33 Conv. CS speech 7.9534 Conv. CS speech 5.935 Conv. CS speech 4.7536 Conv. CS speech 12.2 + PS Interactive (64/128)37 Conv. CS speech 12.2 + PS Interactive (128/64)38 Conv. CS speech 12.2 + PS Interactive (64/384)39 2* PS Interactive (64/128)40 Conv. CS Speech (12.65, 8.85, 6.60)41 Conv. CS Speech (12.65, 8.85, 6.60), preconfigured42 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (0/0)43 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (64/64)44 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (64/128)45 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (128/64)46 Stream. PS (128/HS) + PS Interactive (8/HS)47 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (64/HS)48 Conv. CS Speech (12.65, 8.85, 6.60) + PS Interactive (384/HS)49 Conv. CS Speech 12.2 + Stream. PS (128/HS) + PS Interactive

(8/HS)50 Conv. CS Speech (12.65, 8.85, 6.60) + Stream. PS (16/128) + PS

Interactive (8/8)51 Conv. CS Speech (12.65, 8.85, 6.60) + 2* PS Interactive (64/64)52 PS Interactive (128/HS)53 PS Interactive (16/HS)54 2* PS Interactive (64/HS)55 2* PS Interactive (128/HS)56 2* PS Interactive (384/HS)57 Conv. CS Speech 12.2 + 2* PS Interactive (64/HS)58 Conv. CS Speech 12.2 + 2* PS Interactive (128/HS)59 Conv. CS Speech 12.2 + 2* PS Interactive (384/HS)60 Conv. CS Speech 12.2 + PS Interactive (128/HS)61 Conv. CS Speech 12.2 + 3* PS Interactive (64/HS)62 2* PS Interactive (EUL/HS)63 Stream. PS (16/HS) + 2* PS Interactive (64/HS)64 Conv. CS Speech 12.2 + Stream. PS (16/HS) + 2*PS Interactive

(64/HS)65 Conv CS Speech 12.2 + Stream. PS (128/HS) + 2* PS Interactive

(64/HS)66 3* PS Interactive (64/HS)67 PS Interactive (16/16)68 PS Interactive (16/64)69 PS Interactive (64/16)71 Conv. CS Speech 12.2 + 3* PS Interactive (64/64)72 Stream. PS (16/HS) + PS Interactive (8/HS)73 Stream. PS (32/HS) + PS Interactive (8/HS)

50

UERC ID Description74 3* PS Interactive (64/64)75 Stream. PS (128/HS) + 2*PS Interactive (64/HS)76 Conv. CS Speech 12.2 + 2* PS Interactive (128/128)77 Conv. CS Speech 12.2 + Stream. PS (16/HS) + PS Interactive

(8/HS)78 Conv. CS Speech 12.2 + Stream. PS (32/HS) + PS Interactive

(8/HS)79 CS Conv. CS speech (5.9, 4.75)80 Conv. CS speech (5.9, 4.75) + PS Interactive (0/0)94 SRB (3.4/3.4)95 SRB (3.4/3.4), preconfigured113 Conv. CS speech 12.2 + PS Interactive (16/HS)123 Conv CS Speech 12.2 + PS Interactive (EUL/HS)124 Conv CS Speech 12.2 + 2* PS Interactive (EUL/HS)125 Conv CS Speech 12.2 + 3* PS Interactive (EUL/HS)128 3* PS Interactive (EUL/HS)176 Conv. CS speech (5.9, 4.75) + PS Interactive (EUL/HS)

51

Appendix B

Table 7.2: Bigram part 1. Table 7.3: Bigram part 2.

Sequence Counts % Sequence Counts %

4-21 44799 23.92 21-4 35060 25.0221-4 44050 23.52 4-21 30370 21.6725-4 42522 22.7 4-25 29453 21.024-25 40577 21.66 25-4 25735 18.371-25 4693 2.51 25-21 5393 3.8525-0 2031 1.08 1-25 4538 3.244-0 1284 0.69 25-0 2758 1.974-53 1048 0.56 4-0 1027 0.7353-4 989 0.53 4-53 712 0.5125-21 695 0.37 53-4 709 0.51123-9 562 0.3 123-9 594 0.429-123 483 0.26 9-123 486 0.359-4 430 0.23 9-4 373 0.271-53 410 0.22 1-53 362 0.264-9 356 0.19 4-9 295 0.21

53-15 278 0.15 53-0 266 0.1921-0 245 0.13 53-15 248 0.181-2 239 0.13 15-15 233 0.1753-0 237 0.13 15-53 199 0.1415-4 182 0.1 1-2 159 0.112-123 128 0.07 21-0 155 0.1115-15 122 0.07 25-25 149 0.112-0 116 0.06 25-123 102 0.07

123-25 96 0.05 2-0 86 0.061-0 93 0.05 2-123 80 0.064-1 60 0.03 21-438 75 0.05

21-438 58 0.03 123-25 64 0.0525-123 57 0.03 25-438 48 0.0325-1 53 0.03 62-25 46 0.0325-25 53 0.03 53-21 45 0.0315-53 52 0.03 438-21 39 0.03

52

62-25 52 0.03 25-1 36 0.0315-0 41 0.02 9-0 26 0.02

25-438 39 0.02 25-62 25 0.024-62 34 0.02 15-4 23 0.024-4 20 0.01 4-1 23 0.02

25-62 19 0.01 15-0 21 0.014-438 17 0.01 4-62 21 0.01113-9 13 0.01 4-4 19 0.019-113 10 0.01 123-2 7 0123-2 9 0 53-53 7 02-1 4 0 25-69 6 0

25-69 4 0 15-52 4 053-21 4 0 52-0 4 09-2 4 0 4-438 3 0

15-52 3 0 53-438 3 02-113 3 0 69-4 3 052-0 3 0 9-2 3 09-0 3 0 1-438 2 0

1-456 2 0 1-456 2 069-25 2 0 123-0 2 069-4 2 0 123-123 2 010-0 1 0 2-1 2 0

113-53 1 0 69-67 2 0123-0 1 0 9-113 2 02-9 1 0 113-19 1 053-1 1 0 113-53 1 0

53-113 1 0 113-9 1 053-438 1 0 123-21 1 062-0 1 0 15-438 1 09-10 1 0 19-9 1 0

9-21 1 0 2-9 1 0

53-1 1 053-113 1 053-67 1 067-0 1 067-25 1 067-4 1 069-25 1 09-21 1 0

9-438 1 0

53

Appendix C

Table 7.4: Trigram part 1. Table 7.5: Trigram part 2.

Sequence Counts % Sequence Counts %

4-21-4 40101 22.12 4-21-4 27184 20.014-25-4 39722 21.91 4-25-4 24376 17.9525-4-21 26139 14.42 21-4-25 19640 14.4621-4-25 25066 13.82 25-4-21 15418 11.3521-4-21 17757 9.79 21-4-21 14275 10.5125-4-25 15409 8.5 25-4-25 9723 7.161-25-4 2660 1.47 4-25-21 4678 3.441-25-0 1948 1.07 25-21-4 4578 3.3721-1-25 1540 0.85 1-25-0 2515 1.8525-4-0 889 0.49 1-25-4 1279 0.944-53-4 805 0.44 1-25-21 682 0.54-25-21 661 0.36 25-4-0 542 0.421-4-53 541 0.3 4-53-4 480 0.3553-4-21 478 0.26 21-4-0 439 0.3253-4-53 427 0.24 9-123-9 426 0.319-123-9 408 0.23 21-4-53 420 0.319-4-21 309 0.17 53-4-21 391 0.2921-4-0 308 0.17 123-9-123 332 0.2421-4-9 290 0.16 53-4-53 281 0.21123-9-4 283 0.16 9-4-21 257 0.19

123-9-123 273 0.15 123-9-4 245 0.1825-21-4 271 0.15 21-4-9 240 0.1825-21-1 239 0.13 1-53-0 235 0.174-21-0 220 0.12 4-25-0 234 0.174-53-15 214 0.12 4-53-15 175 0.131-53-0 212 0.12 15-53-4 170 0.134-9-123 209 0.12 15-15-15 167 0.1253-15-4 142 0.08 4-9-123 154 0.114-9-4 141 0.08 53-15-53 139 0.11-53-4 137 0.08 21-0-1 128 0.0921-1-53 122 0.07 4-9-4 127 0.09

54

1-2-123 119 0.07 0-1-2 117 0.091-2-0 114 0.06 21-1-53 101 0.07

2-123-9 103 0.06 4-21-0 95 0.0715-4-21 99 0.05 25-123-9 92 0.072-0-1 99 0.05 25-25-25 90 0.079-4-25 99 0.05 9-4-25 84 0.06

123-25-4 91 0.05 1-2-0 82 0.064-25-0 81 0.04 438-1-25 82 0.0615-4-53 79 0.04 4-25-123 76 0.0653-4-0 79 0.04 2-0-1 75 0.06

9-123-25 73 0.04 1-2-123 74 0.0521-21-1 72 0.04 2-123-9 74 0.0521-1-2 71 0.04 0-21-21 72 0.05

438-1-25 65 0.04 53-15-15 66 0.051-53-15 59 0.03 15-15-53 60 0.044-21-438 56 0.03 1-53-4 59 0.0425-123-9 51 0.03 1-53-15 54 0.0453-15-15 50 0.03 4-21-438 54 0.040-21-21 48 0.03 9-123-25 54 0.0425-4-9 48 0.03 21-438-1 53 0.044-1-0 48 0.03 25-21-0 50 0.04

15-53-4 46 0.03 4-25-25 47 0.034-25-123 46 0.03 1-0-1 45 0.0325-1-0 45 0.02 53-0-21 40 0.0321-0-21 43 0.02 53-21-4 39 0.0353-15-53 43 0.02 123-25-4 37 0.0353-15-0 41 0.02 21-1-2 37 0.0315-15-4 40 0.02 25-4-9 37 0.0362-25-4 39 0.02 53-4-0 36 0.0321-4-1 37 0.02 25-1-0 35 0.03

21-438-1 35 0.02 438-21-4 35 0.0315-0-1 34 0.02 25-438-1 33 0.024-62-25 34 0.02 25-25-438 32 0.02438-21-4 34 0.02 4-53-21 29 0.021-25-21 33 0.02 21-0-21 27 0.0225-438-1 32 0.02 4-53-0 27 0.021-25-1 29 0.02 62-25-4 26 0.0253-0-21 28 0.02 25-62-25 25 0.0221-4-62 26 0.01 1-25-123 22 0.024-53-0 25 0.01 21-438-21 22 0.02

25-25-438 24 0.01 123-25-21 21 0.024-25-25 24 0.01 4-62-25 21 0.02

21-438-21 23 0.01 53-15-0 21 0.021-0-21 20 0.01 4-1-0 20 0.014-25-1 20 0.01 9-0-1 20 0.01

25-25-25 19 0.01 1-25-1 19 0.01

55

25-4-1 19 0.01 15-0-1 19 0.0125-62-25 18 0.01 15-53-15 18 0.012-0-21 17 0.01 21-4-62 18 0.01

2-123-25 17 0.01 9-4-9 18 0.019-4-9 17 0.01 25-25-4 17 0.014-4-21 16 0.01 4-4-21 17 0.01

4-25-438 15 0.01 53-15-4 17 0.0121-21-0 13 0.01 4-25-1 16 0.0121-4-438 13 0.01 123-9-0 15 0.0121-4-4 12 0.01 25-438-21 15 0.0125-25-4 10 0.01 21-4-1 14 0.014-438-1 10 0.01 4-25-438 13 0.01438-1-53 10 0.01 4-25-62 13 0.011-25-25 9 0 15-4-21 12 0.0115-15-53 9 0 21-4-4 12 0.0162-25-62 9 0 1-0-21 11 0.019-113-9 9 0 1-25-25 11 0.011-25-123 8 0 15-4-53 11 0.012-123-2 8 0 2-0-21 11 0.0125-4-62 8 0 25-21-438 10 0.014-1-25 8 0 4-9-0 10 0.01

123-2-123 7 0 9-4-0 10 0.0115-0-21 7 0 1-53-21 9 0.0125-1-25 7 0 62-25-62 9 0.0125-4-4 7 0 25-123-25 8 0.01

25-438-21 7 0 438-1-53 8 0.014-438-21 7 0 25-4-1 7 0.01113-9-113 6 0 0-21-438 6 0113-9-4 6 0 15-15-4 6 0

25-123-25 6 0 15-53-21 6 025-21-0 6 0 21-21-0 6 04-25-62 6 0 62-25-21 6 00-21-0 5 0 9-0-21 6 0

15-53-15 5 0 25-25-21 5 01-2-1 4 0 53-53-53 5 0

1-25-62 4 0 62-25-0 5 015-4-0 4 0 0-21-0 4 02-1-25 4 0 1-25-69 4 04-1-2 4 0 123-2-123 4 0

4-9-113 4 0 123-25-123 4 053-4-1 4 0 15-52-0 4 062-25-1 4 0 25-4-4 4 0

123-25-123 3 0 438-1-2 4 0123-9-2 3 0 438-21-21 4 015-52-0 3 0 53-15-52 4 02-113-9 3 0 9-123-2 4 0

56

25-4-438 3 0 1-25-438 3 04-4-25 3 0 1-25-62 3 0438-1-2 3 0 123-2-0 3 052-0-1 3 0 15-53-0 3 053-21-4 3 0 2-123-2 3 09-0-1 3 0 25-4-62 3 09-4-0 3 0 25-69-4 3 0

1-2-113 2 0 52-0-21 3 01-25-69 2 0 53-21-21 3 0123-9-0 2 0 53-21-438 3 021-1-456 2 0 53-438-1 3 025-69-25 2 0 69-4-25 3 025-69-4 2 0 9-4-4 3 04-25-69 2 0 0-1-456 2 04-53-21 2 0 1-2-1 2 0

438-21-21 2 0 1-438-1 2 053-15-52 2 0 1-456-1 2 069-25-0 2 0 1-53-438 2 09-2-123 2 0 123-0-1 2 00-0-1 1 0 123-123-9 2 0

1-456-1 1 0 123-25-0 2 01-456-21 1 0 15-0-21 2 01-53-1 1 0 2-1-25 2 01-53-21 1 0 2-123-25 2 010-0-21 1 0 21-21-438 2 0113-53-4 1 0 21-4-438 2 0113-9-0 1 0 25-25-0 2 0123-0-21 1 0 25-25-69 2 0123-2-0 1 0 25-69-67 2 0123-2-9 1 0 4-1-25 2 0

123-25-21 1 0 4-4-25 2 0123-25-25 1 0 4-438-1 2 0123-9-21 1 0 4-9-2 2 015-15-52 1 0 456-1-25 2 015-53-21 1 0 9-2-123 2 02-9-123 1 0 0-1-438 1 0

21-21-438 1 0 1-2-9 1 025-1-2 1 0 1-53-1 1 0

25-21-438 1 0 1-53-113 1 025-62-0 1 0 1-53-67 1 04-0-0 1 0 113-19-9 1 04-4-4 1 0 113-53-0 1 0

4-53-113 1 0 113-9-4 1 04-53-438 1 0 123-21-4 1 04-9-10 1 0 123-9-2 1 04-9-2 1 0 123-9-438 1 0

57

438-21-0 1 0 15-438-21 1 0456-1-25 1 0 15-53-438 1 0456-21-4 1 0 15-53-53 1 053-1-25 1 0 19-9-113 1 053-113-9 1 0 2-123-0 1 053-21-1 1 0 2-9-0 1 053-4-9 1 0 21-1-438 1 0

53-438-1 1 0 25-1-25 1 062-0-1 1 0 25-123-123 1 069-4-0 1 0 25-123-21 1 069-4-21 1 0 25-25-1 1 09-10-0 1 0 25-4-438 1 0

9-113-53 1 0 25-69-25 1 09-123-0 1 0 4-1-2 1 09-123-2 1 0 4-438-21 1 09-2-0 1 0 4-53-53 1 0

9-2-113 1 0 4-9-113 1 09-21-4 1 0 4-9-21 1 09-4-438 1 0 52-0-1 1 0

9-4-53 1 0 53-1-0 1 0

53-113-53 1 053-15-438 1 0

53-4-1 1 053-53-15 1 053-53-21 1 053-67-0 1 067-0-1 1 0

67-25-21 1 067-4-25 1 069-25-25 1 069-67-25 1 069-67-4 1 09-113-19 1 09-113-9 1 09-123-0 1 0

9-123-123 1 09-2-0 1 09-21-4 1 09-4-1 1 0

9-438-1 1 0

58

Appendix D

Figure 7.1: Raw data plottad against an expontial distribution.

59

(a) Distribution of µ for part 1. (b) Distribution of σ for part 1.

(c) Distribution of µ for part 2. (d) Distribution of σ for part 2.

Figure 7.2: Distributions of µ and σ part 1 and 2 UERC 4.

(a) Distribution of diffrence in µ part 1 and 2.

(b) Distribution of difference in σ part 1 and 2.

Figure 7.3: Distributions of difference in µ and σ UERC 4.

60

(a) Distribution of µ for part 1. (b) Distribution of σ for part 1.

(c) Distribution of µ for part 2. (d) Distribution of σ for part 2.

Figure 7.4: Distributions of µ and σ part 1 and 2 UERC 21.

(a) Distribution of diffrence in µ part 1 and 2.

(b) Distribution of difference in σ part 1 and 2.

Figure 7.5: Distributions of difference in µ and σ UERC 21.

61

(a) Distribution of diffrence in µ part 1 and 2.

(b) Distribution of difference in σ part 1 and 2.

Figure 7.6: Distributions of difference in µ and σ EUL/HS.

62