Tracking and Predicting End-to-End Quality in Wireless Community Networks (WCN)

4th Int. Workshop on Community Networks and Bottom-up-Broadband,

CNBuB 2015

August 26th, 2015. Rome, Italy

Pere Millán1, C. Molina1, E. Dimogerontakis2, L. Navarro2, R. Meseguer2, B. Braem3, C. Blondia3

 1Universitat Rovira i Virgili, Tarragona, Spain

2Universitat Politècnica de Catalunya, Barcelona, Spain3University of Antwerp – iMinds – MOSAIC, Antwerpen, België

• Motivation

• [Link/End-to-End Quality] Prediction

in [Wireless] Networks

• End-to-End Quality Prediction

in Wireless Community Networks

• Experimental Methodology & Results

• Conclusions & Future Work

OLSROutlineOutline

2

Motivation3

MotivationMotivationCommunity networks create measurable social impact

providing the right and opportunity of communication

4

• Community networks: – Large, heterogeneous, dynamic and decentralized structures

• Some challenges:– What is the effect of the asymmetrical features and

unreliability of wireless communications on network performance and routing protocols?

– Link quality tracking is a key method to applywhen routing packets through an unreliable network.

– Routing algorithms should avoid weak links whenever possible and as soon as possible.

MotivationMotivation

5

• Link quality (LQ) estimation/prediction approach increases the improvements in routing performance achieved through link quality tracking.

– RT metrics do not provide enough information to detect degradation/activation of a link at the right moment.

– Prediction techniques are needed to foresee link quality changes in advance and take the appropriate measures.

MotivationMotivation

6

• End-to-End Quality (EtEQ) or Path Quality extends the Link Quality (LQ) concept to the full communication path (sender-receiver)and is computed based on the quality (ETX) of the individual links that conform the communication path.

MotivationMotivation

7

Main contributions:• A detailed analysis of path properties and path ETX behavior

in wireless community networks (WCN):EtEQ prediction is possible and meaningful.

• Use of time series analysis to estimate EtEQ in the routing layer for real-world WCN.

• Clear evidence that EtEQ values computed through time series algorithms can make accurate predictions in WCN.

• A detailed analysis of prediction accuracy for the next stepconsidering time of day and some steps ahead in future.

In this work we present an analysis of End-to-End Quality tracking and predictionand differences with our previous LQ analysis

8

Prediction in WCN9

• FunkFeuer uses OLSR Routing Protocol:– Prediction of path changes can improve local node routing decisions,

since it can provide the node with an estimation about the future local and remote events.

– OLSR uses ETX to choose the next hop.

• ETX Metric:– Link: number of expected transmissions to send a packet over the link.– Path: sum of ETX values of links that form the path (≥ path Hops).

• Path ETX prediction:– Will allow more efficient routing decisions in an unstable environment.

BackgroundBackground

10

• Goals of Network Prediction:• Routing Traffic Reduction: OLSRp, Kinetic Multipoint Relaying (KMPR).

• Energy Efficient Routing: LPR, MDR, E-DSR routing protocol.

• Link Quality prediction.

• Link Quality Prediction in WCN:• LQ tracking: select higher quality links max. delivery rate & min. traffic congest.

• LQ prediction: determine beforehand which links are more likely to change behavior.

• Idea: The routing layer can make better decisions at the appropriate moment.

• Link Quality Estimators (LQE) metrics:• Measure quality of links between nodes based on physical or logical metrics.• Physical metrics focus on the received signal quality: LQI, SNR, RSSI.

• Logical metrics focus on % of lost packets: RNP, ETX, PSR.

• To select the more suitable neighbor nodes when making routing decisions.

Network PredictionNetwork Prediction

11

• Compute Path Quality as aggregation of Link values:– MARA [28], EER [29], ETOP [30], EED/WEED [31].

• Other relevant works:– MetricMap [32]: uses a learning-enabled method for LQ assessment.

Also uses time series analysis to improve the routing protocol.– Maccari & Cigno [33]: quality of routes & techniques to select MPR nodes.– Cunha et al. [34]: detects path changes (NN4) and then remaps (DTRACK).– Millan et al. [13]: behaviour of LQ prediction in WCN, using Time-Series,

several learning algorithms, to accurately predict future values.

End-to-End QualityEnd-to-End Quality

12

• Funkfeuer WCN (Austria):– 2.000+ links, OLSR-NG routing protocol.

• Open data set (Confine Project):– OLSR info, 404 nodes, 7 days, av. degree: 3.5, diameter: 18.– 1.032 links with variations in LQ (higher prediction accuracy using all links).

• 4 Prediction Algorithms:– Support Vector Machines (SVM), k-Nearest Neighbors (kNN),

Regression Trees (RT), Rule-Based Regression (RBR).

• Time Series Analysis & Forecasting:– Training and test sets validation approach.– Weka: machine learning/data mining approach to model time series,

encodes time dependency via additional input fields (“lagged” variables).

• Metrics and Plots: • Mean Absolute Error (MAE). MAE = sum(abs(predicted - actual)) / N• Boxplots: classic representations of a statistical distribution of values.

Experimental MethodologyExperimental Methodology

13

Results

14

Comparison of TS learning algorithmsComparison of TS learning algorithmsTime series analysis

can be used to predict future End-to-End quality values?

4 classification algorithms:• Support Vector Machines (SVM)• k-Nearest Neighbors (KNN)• Regression Trees (RT)• Rule-Based Regression (RBR)

15

Data sets:

• Training: 2016 instances (7 days)

• Test: 288 instances (1 day)

Lag window: last 12 instances

BEST

WORST

High percentage of success

Learning algorithms: error variabilityLearning algorithms: error variability

16

3 of 4 algorithms (RT, RBR, SVM) achieved a similar accuracy

for most of the linksSome outliers

have high errors

… that increase the average values

t-test result: RBR is a good candidate

to make predictions

EtEQ Prediction with RBR algorithmEtEQ Prediction with RBR algorithm

How can we reach a satisfactory level of prediction?

17

Same Data sets:

• Training: 2016 instances (7 days)

• Test: 288 instances (1 day)

Lag window: last 12 instances

Some hop pathshave high dispersion

… but we successfully predict a big percentage of fluctuations

EtEQ Prediction with RBR: accuracyEtEQ Prediction with RBR: accuracy

Real and predicted values are very close

18

Deviation remains below 0.5 throughout the whole prediction

Why ETX error has an increasing trend?

(8:00 – 8:30 am)

EtEQ Prediction accuracy (day/night)EtEQ Prediction accuracy (day/night)

Future work: 2 different predictors (day/night)?

19

Less deviations/errors

Data sets:

• Training: 2016 instances (7 days)

• Test: 144 instances (1/2 day)

Lag window: last 12 instances

Day (12 am – 12 pm)

Night (12 pm – 12 am)

More deviations/errors

Prediction of some steps aheadPrediction of some steps ahead

Time series analysis and prediction can be used to predict the value of EtEQsome time steps ahead into the future?

20

Good results for the majority of steps ahead

Average MAE grows very slowly

We could predict successfully the EtEQ several steps ahead in time.

21

ConclusionsFuture Work

• Time series analysis is a promising approach to accurately predict LQs in Community Networks– Routing protocol performance can be improved by providing

information to make appropriate and timely decisions to maximize the delivery rate and minimize traffic congestion.

• All algorithms achieved high percentages of success with average MAE per link between 2.4% and 5% when predicting the next value of EtEQ, being the Rule-Based Regression the best one.– RBR prediction shows an average absolute error less than 1.

• The error variability is similar for 3 of the algorithms: RT, RBR, SVM.– kNN performs worse due to outliers with larger errors.

OLSREtEQ Prediction: ConclusionsEtEQ Prediction: Conclusions

22

OLSRFuture WorkFuture Work

• To extend this analysis to other community networks to evaluate if the observed behavior can be generalized.

• To identify which paths contribute most to the errors in the EtEQ prediction and to understand what factors make it more difficult to predict them.

• To study the impact of errors in routing decisions.

• To study a solution with 2 different predictors: Day/Night.

• To improve the prediction process by discarding those paths whose relation between EtEQ and prediction accuracy is above a certain threshold.

23

Thanks for Your Attention…Questions?

4th Int. Workshop on Community Networks and Bottom-up-Broadband,

(CNBuB 2015) August 26th, 2015. Rome, Italy