SQI Solution for Reference_20110330

8/2/2019 SQI Solution for Reference_20110330

http://slidepdf.com/reader/full/sqi-solution-for-reference20110330 1/15

Transition from KPI to KQI

HUAWEI TECHNOLOGIES CO., LTD.

-- Huawei Service Quality Improvement Solution



Content

What is Service Quality Improvement

Case Study



Technology Trend

Video/Voice Messaging

Email Video telephony

DeviceUser

MultimediaApps

Home

Subway

Office

Cafe Working

Watching TV

Learning

Driving

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 3

End-userExperience

SMS

LocationBased Services

IMS

Mobile TVPTT

Social Networking

User culture drives Technology Trend

IP Transformation





What is Service Quality Improvement about? Service Quality Improvement

Is about End-To-End

Visualization of real end-user experience

Improve application service quality

Root cause analysis of service quality issues


Slow to webSlow to web

……Poor voice qualityPoor voice quality

No serviceNo service

User Experience

Customer Hotline

Network KPI

……

RRC Setup Success

Rate

RRC Setup Success

Rate

Call Setup Success

rate

Call Setup Success

rateCall Drop RateCall Drop Rate

≠



From KPI to Service Quality ImprovementAs is To be

Good KPI, but Poor User Experience Build SQI System, Improve End-User Experience

CTO’s Experience:

• Full signal bar, but difficult to call out

• Poor voice quality, One-way audio

• Connecting to internet, but can’t browse

Level 1 QoE: Quality of Experience, ranking from 1-5;

Level 2 KQI: Key Quality Index, measure end user experiences;

Level 3 Measurement KQI: Measurable, Manageable,Assessable, Improvable KQIs;


internet

Cell KPI Value

RAN Access Success

Rate>99%

Call Drop Rate <0.3%Cell Average KPI is Good

Sample DT KPI Value

Coverage >99%

Voice Quality (MOS) 3.6Sample DT KPI is Good

Good KPI ≠ Good ServiceQuality

√

√

X QoE KQI Measurement KQI

WEB pageaccesssatisfaction

WEB page accesssuccess ratio

E2E access success ratio

DT access success rate

WEB page access delay E2E page access delay

DT page Access delay



From Network layer to User layerAs is To be

Indicator Care ---Monitoring Network KPI VIP Care ---Quick Response to User Complaint

KPI Normal99.20 99.25 99.30 99.30

9 6 . 0 0

9 6 . 0 09 6 . 0 0

9 6 . 0 0

9 7 . 0 0

9 7 . 0 09 7 . 0 0

9 7 . 0 0

9 8 . 0 0

9 8 . 0 09 8 . 0 0

9 8 . 0 0

9 9 . 0 0

9 9 . 0 09 9 . 0 0

9 9 . 0 0

1 0 0 . 0 0

1 0 0 . 0 01 0 0 . 0 0

1 0 0 . 0 0

Week 39 Week 40 Week 41 Week 42

Call Setup Success Ratio-NCR[%]

Call Set-up Success Rate


10:00 am 10:30 am 11:00 am 11:10 am

Call information

Subscriber

Access/release

CS service

PS service

HandoverSignaling

Call recordCause types:

Normal

Access failure

Handover failure

Call drop

Web refresh delay

Other exceptions

10:00 am

Several Key users’ worse experience will notindicate from KPI, then complaint exists…

…..

18:00 pm

At same time:



Huawei SQI Capability - Tools & Platform

Web offline ratio…

Retainability

E2E accessduration

E2E accesscompletion rate

Accessibility

SpeechQuality

Integrity

VIP&VAP user perceptionUser perception

of the entire network

Service QualityKQI System

-One-way Audio

-Echo

-Time delay

-Chopping

-Noise

-Crosstalk


Assessmenttool set

M2000(traffic/log)

NastarPRS MRNastar

Monitoring (tool) Assessment & Delimiting & Identification (Service)

E2E Active/passivejoint testing

U2520

UMGBSC

IP

UMG BSCAbis A Abis AMSCMSCBTSBTS BTSBTS

Tool＋＋＋＋Service

EquipmentcapabilityResource-based refined tracking User-based refined trackingEnd-to-End Network



Huawei SQI Capability – User Experience Research Team

Special Experts Biography

Dr. Dong The User Experience Principle at Huawei.Ph.D from Purdue University in the USA.More than 15 years of research and Design experience in top companies, such as IBM and eBay. He is now in charge

of the user research and consumer programs in the Department of User Centered Design.

Dr. Wan Ph.D. degree in psychology from the Zhejiang University, Zhejiang, China.More than 6-year research experiences on human perception and mental processing.Expert of QoE modeling- establishing quantitative relation between KQIs and QoE.


Dr. Cui Ph.D in Cognitive Psychology from the Zhejiang University, China.Senior user researcher at UCD center, Huawei R&D.Expert of QoE research ， User Experience Design and Human-Computer Interaction.

Tina Zhang Tingting ， received her master of Arts (Sociology) in Nanjing University, accumulated wealth of experience inuser research. She organized and participated in lots of large sample social research and focused on quantitativeanalysis.

From Technologies to Research in End User Experience & Human Perception



Q1: How to Establish SQI System?

Step 5：：：： KQI Monitoring1.Precision：

：：

： KQIs for use of each specific service

2.Time Span: 2~4 weeks, 7*24 hours

3.Sample Size ：

：：

： About 20,000

4.Sample Structure ：

：：

： User groups of different user

behavior, and from different regions.

Step5：：：： QoE Survey1.Method ：

：：

： Phone survey

2.Questionnaire： Satisfaction of the quality of Service,

including QoE and sub-QoE.

3.Performer： Call center (Trained by QoE Experts)

4.Sample Size: 900~1,000 users from those monitored

Example

Project Kick-off Tool &

Service Setup

KQI Data

Collection

KQI Monitoring

& QoE Survey

KQI Analysis &

Optimization

QoE – KQI

ModelingProject End


QoE score fromphone survey

Mapping KQI to QoE

Subjective

. o en a s : g annoy e en users, ow o con ro

KQI monitored QoE surveyed

Data fromKQIsmonitoring

Objective

5 5

6

Step6：：：： QoE-KQI ModelingMethod ：

：：

： Regression analysis+ QoE = f(KQI1, KQI2, KQI3 ……)



Q2: How to Measure QoE?

QoEQoEQoEQoE

Category KQI nameMapping

WeightKQI value

QoEscore

Accessibility

E2E access completion rate(Passive Test)

≥ 9 9 % 5

4 0 %

9 5 % ~ 9 9 %

9 5 % ~ 9 9 % 9 5 % ~ 9 9 %

9 5 % ~ 9 9 % 4

44

4

8 0 % ~ 9 5 % 3

7 0 % ~ 8 0 % 2

< 7 0 % 1

Front page access delay

≥ 5 1 2 k b p s 5

1 0 %

2 5 6 ~ 5 1 2 k b p s 4

1 2 8 ~ 2 5 6 k b p s

1 2 8 ~ 2 5 6 k b p s1 2 8 ~ 2 5 6 k b p s

1 2 8 ~ 2 5 6 k b p s 3

33

3

6 4 ~ 1 2 8 k b p s 2

< 6 4 k b p s 1

≤ 0 . 1 % 5

QOE score QoE grade User perception

≥ 4.5 Excellent Very satisfied

4~4.5 Very good Satisfied

3.5~4 Good Neutral

3~3.5 Fair Very unsatisfied

< 3 Poor Extremely unsatisfied

Example


• Initialized with default weight value & benchmark forKQIs , yet need to get agreement with CamGSM first.

4444 **** 40% + 340% + 340% + 340% + 3****10% + 4*10% + 3*25% + 4*15%10% + 4*10% + 3*25% + 4*15%10% + 4*10% + 3*25% + 4*15%10% + 4*10% + 3*25% + 4*15%

3.65 (Good)Retainability Web browsing data transfercut-off ratio

≤ 0 . 1 % 5

1 0 %

0 . 1 % ~ 0 . 5 %

0 . 1 % ~ 0 . 5 % 0 . 1 % ~ 0 . 5 %

0 . 1 % ~ 0 . 5 % 4

44

4

0 . 5 % ~ 1 % 3

1 % ~ 2 % 2

> 2 % 1

Integrity

Web page refresh successratio

≥ 9 9 % 5

2 5 %

9 5 % ~ 9 9 % 4

8 0 % ~ 9 5 %

8 0 % ~ 9 5 % 8 0 % ~ 9 5 %

8 0 % ~ 9 5 % 3

33

3

7 0 % ~ 8 0 % 2

< 7 0 % 1

Web page refresh delay

≥ 5 1 2 k b p s 5

1 5 %

2 5 6 ~ 5 1 2 k b p s

2 5 6 ~ 5 1 2 k b p s2 5 6 ~ 5 1 2 k b p s

2 5 6 ~ 5 1 2 k b p s 4

44

4

1 2 8 ~ 2 5 6 k b p s 3

6 4 ~ 1 2 8 k b p s 2

< 6 4 k b p s 1



Content

What is Service Quality Assurance

Case Study



Data Service Quality Improvement – China Mobile

Customer Needs

Measurable – Establish WAP/MMS/FTP/Stream data service measurement system,

which is tailored for China Mobile Hangzhou;

Visible – Quality monitoring of WAP/MMS/Stream data services;

Manageable – Establish methodology to identify key factors and improve KQI;

Improvement 3G Quality – Improve 3G network quality. (E.g. WEB front page

access delay/Unstable MMS downloading speed/Frequent 3G complaints etc)


Establishment - China Mobile Hangzhou SQI system

through partnership

Analysis & Optimization – - Data Service Quality Analysis

- User complaints Analysis

- 2G/3G Hotspot Analysis

- User Behavior Analysis

- P2P Service Analysis & Restriction

- PS Core Network Parameter Audit & Resource Evaluation

Hangzhou, China Mobile

Solutions Delivery Period: Aug.2010 – Dec.2010

Organization: An End-to-End project teamincluding: Wireless / PS / SQI PDT team / NastarTool Team

Tools: - Nastar- Wireless tool set- Probe /CDS- U2520



Data Service Quality Improvement – China Mobile

•• TOP region’s WEBTOP region’s WEB Delay comparisonDelay comparison

Connectionaccess delay

Front Page Access Delay – Before & After (s)

BF AF BF BF BFAF AF AF

Refreshdelay

Page Refresh Delay – Before & After (s)



•• TOP region’s WEBTOP region’s WEB access failure comparisonaccess failure comparison

Accessfailure rate

TOP Cell Access Failures – Before & After


Values:

- Established customized SQI solution for Hangzhou,

China Mobile.

- New O&M methodology for long-term benefits;

- WEB Data Service Quality was remarkably Improved;

- Joint Benefits Analysis is undertaken.



Thank youwww.huawei.com

Documents

SQI Solution for Reference_20110330