Qasim M. Chaudhari and Dr. Erchin Serpedin Department of Electrical and Computer Engineering

1

Estimation of Clock Parameters and Performance Benchmarks for

Synchronization in Wireless Sensor Networks

Qasim M. Chaudhari and Dr. Erchin Serpedin

Department of Electrical and Computer EngineeringTexas A&M University, College Station, TX.

2

Outline

1. Wireless sensor networks

2. Related work

3. Clock model

4. A Sender-Receiver protocol

5. Clock offset estimation

6. Clock offset and skew estimation

7. Simplified schemes

8. Best Linear Unbiased Estimation – Order Statistics

9. Minimum Variance Unbiased Estimation

10. Minimum Mean Square Error estimation

3

11. Clock synchronization of inactive nodes

12. Clock offset and skew estimation in a Receiver-Receiver protocol

13. Conclusions

14. Future research directions

4

Wireless Sensor Networks

Gateway

Internet

Server

Wireless Terminal

S

S

D

Source

Destination

D

S

5

Introduction

Small scale sensor nodes Limited power Harsh environmental conditions Communication failures Node failures Dynamic network topology Mobility of nodes

6

Applications Monitoring

Environment and habitat Military surveillance Security Traffic

Controlling and tracking Industrial processes Fire Detection Object tracking

7

Main Challenges

8

Importance of time synchronization

Time synchronization in WSNs is important for Efficient duty cycling Localization and location-based monitoring Data fusion Distributed beamforming and target tracking Security protocols Network scheduling and routing, TDMA

9

Constraints Limited hardware

Reduced computational power Low memory

Limited energy Communication vs. computation RF energy required to transmit 1 bit over 100 meters is equivalent to execution

of 3 million instructions [Pottie 00]

Traditional clock synchronization techniques Communication comes for free Computational resources are powerful

Examples: NTP is energy expensive, GPS is cost expensive

10

Related Work

Reference Broadcast Synchronization (RBS) [Elson 02] Conventional receiver-receiver protocol Reduces nondeterministic delays Conserves energy via post facto synchronization

Timing synch Protocol for Sensor Networks (TPSN) [Ganeriwal 03] Conventional sender-receiver protocol Two operational phases: Level Discovery and Synchronization

Time Diffusion Protocol (TDP) [Su 05] Achieves a network-wide equilibrium time using an iterative, weighted

averaging technique based on diffusion of timing messages

11

Related Work

Analysis of a sender-receiver model [Ghaffar 02] For known fixed delays, maximum likelihood estimator for clock offset

does not exist Five algorithms: median round delay, minimum round delay, minimum

link delay, median phase, average phase. Minimum link delay algorithm has the lowest variance

Maximum likelihood clock offset estimator for unknown fixed delays [Jeske 05]

12

Clock Model

A computer clock consists of two components Frequency source Means of accumulating timing events

Practical clocks are set with limited precision Frequency sources run at slightly different rates Frequency of a crystal oscillator varies due to

Initial manufacturing tolerance Temperature, pressure Aging

13

Clock Model

A general clock model can be represented by

where is the clock offset, is the clock skew and is the clock drift

Clock synchronization problem Given the logical clock for a node k in the network, then

is a function of Target synchronization accuracy Amount of energy the network is willing to pay

14

2,kT

4,kT

3,kT

1,kTA

B

2 1 4 3( ) ( )

2 2

T T T T U V

1 4

2 3

, : :

, : :

T T

T T

Local time stamps at Node A Fixed portion of delays

Local time stamps at Node B Clock offset

1. Node A sends a timing message (Level of Node A and T1) to Node B at T1.2. Node B sends an ACK (Level of Node B, T1, T2, and T3) to Node A at T3.

With this, Node A calculates the clock offset.

Sources of error (time uncertainty) associated with message exchanges

o Send time: time spent to construct a message

o Access time: delays at MAC layer before actual transmission

o Propagation time: time of flight from one node to another

o Receive time: time needed for the receiver to receive the message and process it

A Sender-Receiver Protocol

15

Observations

Fixed clock offset model is not sufficient in practice Clock skew correction results in long term synchronization

and hence more energy savings Network delays being asymmetric is a more realistic scenario Even for the symmetric clock offset only model, better

estimation schemes achieving are possible Minimum Variance Unbiased Estimation (MVUE) Minimum Mean Square Error Estimation (MMSE)

Lack of analytical performance bounds and metrics Average RBS error: [Elson 02] or [Ganeriwal 03]?

16

Clock Offset

Gaussian Noise Assumption One motivation comes from experimental basis [Elson 02] In case of unknown delay distribution, we can evoke Central

Limit theorem Example: for uniform delays, the sum of even two of them

starts resembling a Gaussian curve

17

Clock Offset

The likelihood function can be written as

And the clock offset estimate and the CRLB are

2,kT

4,kT

3,kT

1,kTA

B

18

Clock Offset

Exponential Delay Assumption Random delays often modeled as exponential Several traces of delay measurements on Internet collected by

[Moon 99] fitting an exponential distribution Conformation of experimental observations with mathematical

results Experimental observations

Minimum link delay algorithm [Paxson 98] Clock Filter algorithm in NTP [Mills 91]

Mathematical results Best performance by Minimum link delay algorithm [Ghaffar 02] ML estimate based on minimum order statistics [Jeske 05]

19

Clock Offset

Likelihood function is given as

ML clock offset estimate is

CRLB is derived as

20

Clock Offset and Skew

1,kT1,NT

2,1T

4, 1,1( 1)( )NT T

Node A

Node B

4,1T 1,2T 4,2T

3,1T 2,2T 3,2T

4,kT

2,kT 3,kT2,NT 3,NT

4,NT

: clock offset

: clock skew

1,kT

4,kT1,1 0T

1,( 1) k kT X

4,( 1) k kT Y

1,1T

kX kY

2, 1,( )k k kT T X

3, 4,( )k k kT T Y

21

Clock Offset and Skew Gaussian

Likelihood function with is

Joint ML estimate for clock offset is shown to be

where

22

Clock Offset and SkewGaussian

And for the clock skew

Computationally quite complex Fixed delay must be known Open problem: Recursive implementation/update?

23

Clock Offset and SkewGaussian

Cramer-Rao Lower Bound is expressed as

where

Proportional to clock skew squared Not only dependent on number of synchronization messages

but also on the synchronization period

24

Clock Offset and Skew Exponential

The likelihood function in this case is

Four different cases need to be considered

Case I Known Known

Case II Known Unknown

Case III Unknown Known

Case IV Unknown Unknown

25

Clock Offset and SkewExponential

Case I: known, known

Constraints

ML estimator

26


27


Case II: known, unknown

Constraints

Lemma 1: lies on one of

the following curves

28


Lemma 2: lies either on point A or to the left of it (B,C,…) Lemma 3: To the left of A,

boundary of support region is

formed by a sequence of

curves with decreasing slopes Lemma 4: is unique

and is given by one of

29


30


31


Case III: unknown, known

Constraints

32


Lemma 5: Only two points satisfy the constraints ML estimator has the closed-form expression

33


Case IV: unknown, unknown

Constraints

Curves intersect on the line

Over this line, is constrained by

34


Problem can be solved by the application of four lemmas Final form of the ML estimator is

35


36


37

Simplified Schemes

Fixed delay must be known in Gaussian case Computational complexity Further simplification within the same framework is possible

suitable for WSNs in case Synchronization accuracy constraints are not stringent Energy conservation constraints are strict

One simple scheme is independent of delay distribution involved

Cost paid is slight degradation in estimation quality

38

Utilizing Data Samples 1,N

Better skew estimation for large synchronization period Utilize only 1st and last sample differences for eliminating the

clock offset Define Simplified new model

where and are either Gaussian or Laplacian distributed depending on original delay distribution

39


Gaussian delays Likelihood function for highly reduced data set is

ML-Like clock skew estimator is expressed as

CRLB-Like lower bound is Depends on timestamping “distances”

40


Exponential delays The reduced likelihood function is

ML-Like clock skew estimator can be derived as

CRLB-Like lower bound

41


Simulation results

42

Two Minimum Order Statistics

Motivation Unknown delay distribution Small synchronization period

Opening the model equations as

Choose two points as

43


Joint the two points to obtain the estimate through its slope and intercept

The form of the estimator is

Almost as simple as the clock offset only case Knowledge of is not required

44


45


Simulations results

46


Computational complexity comparison with the MLE

47

Summary

Gaussian Exponential

Offset Model MLE + CRLB CRLB

Offset + Skew Model MLE + CRLB MLE + Algorithms

Offset + Skew Model Using First and Last sample

ML-Like + LB

Using First and Last sample

ML-Like + LB

Offset + Skew Model Two minimum order statistics

Algorithm + Computational Complexity

48

Best Linear Unbiased Estimation – Order Statistics

Limited power resources in WSN implies better estimation techniques should be utilized

Results derived so far correspond to symmetric delays, although asymmetry is a more realistic scenario

Best Linear Unbiased Estimation (BLUE) is suboptimal in general due to linearity constraint

What if the linearity constraints are on the order statistics of observed data, instead of the raw observations?

49


Transforming the data as

Following relations hold for ordered data

50


The covariance matrix for can be derived as

Its inverse can be found by Gauss-Jordan elimination Let the ordered observations be represented as

51


Asymmetric Link Delays The asymmetric linear model can be written as

And the Gauss-Markov theorem implies

52


The covariance matrix for is

The final expression for is

53


Symmetric Link Delays The symmetric linear model can be written as

The Gauss-Markov theorem yields the solution

54


Covariance for is

The expression for is

BLUE-OS for same as the MLE for symmetric link delays

55

Minimum Variance Unbiased Estimation

Asymmetric Link Delays Found by the application of Rao-Blackwell-Lehmann-Scheffe

theorem Likelihood function can be expressed as

According to Neyman-Fisher factorization theorem, the sufficient statistics is

56


Notice that Find such that

Applicable only if is a complete sufficient statistic

Only function of is unbiased

57


Unbiased estimator of ? Note that BLUE-OS is unbiased and hence MVUE !

Compensation for asymmetry through

Variance of the clock offset

58


Symmetric Link Delays Again applying the Rao-Blackwell-Lehmann-Scheffe theorem,

the likelihood function is

More than one unbiased functions of complete statistic? Through Neyman-Fisher factorization theorem, the actual

sufficient statistics is

59

Minimum Variance Unbiased Estimation is proved to be complete Unbiased estimator of ? BLUE-OS is unbiased and hence the MVUE

In symmetric case, the MVUE and BLUE-OS of coincide with MLE

Its variance is

60

Summary

Clock Offset

MVUE

Symmetric Delays MSE

Remarks

Asymmetric Delays MVUE

MSE

Remarks

61

Explanatory Remarks

Does this discontinuity in clock offset estimates performance make sense?

Which estimator is better when the network delays are slightly symmetric?

MVUE is the best in unbiased class of estimators, not all. For asymmetric case,

62

Explanatory Remarks

The MLE outperforms the MVUE under the condition

Estimator could be chosen according to the number of synchronization messages if knowledge of is available

Around the point , MLE attains lesser MSE as the link asymmetry decreases, i.e.,

63

Explanatory Remarks

Simulations results

64

Explanatory Remarks

Apparently, adapting between the two estimators a good idea according to

since have been obtained too. Despite the fact that MLE is functionally invariant,

considerable amplification of errors occurs due to repeated nonlinear processing

Results are even applicable to Internet time synchronization

65

Explanatory Remarks

As a byproduct, the MVUE of the fixed and mean variable link delays are obtained

Endd-to-end delay measurements are helpful in analyzing network performance

Very useful for applications behaving adaptively based on observed network performance

Continuous media applications, such as audio and video, absorb the delay jitter perceived at receiver for smooth playout of media stream

66

Minimum Mean Square Error Estimation

In general, the MMSE estimator is not realizable due to the dependence of MSE on the unknown parameter

MSE is a sum of variance and bias squared

This dependence usually comes from the bias Setting the MSE proportional to inverse of the scale parameter

cancels the dependent factors

67


The MMSE estimator comes out to be a function of MVUE Closed-form expression for

And for mean link delays

MSE of clock offset

68


The MMSE estimator comes out to be a function of MVUE Closed-form expression for


MSE of clock offset

69

Clock Synchronization for Inactive Nodes

Packet synchronization protocols Receiver-Receiver (R-R) Sender-Receiver (S-R)

For WSNs implementing any sender-receiver protocol, the inactive nodes can exploit the timing messages received

m p

qm p

q

70


The model can be represented as

m

p

q

1ms 1

p mr mjs p m

jr p mJr

mJs

1m pr

1p ms m p

jr p mjs m p

Jr p m

Js

1m qr

1p qr m q

jr p qjr m q

Jr p q

Jr

71


Likelihood function, assuming symmetric delays, is

The maximum likelihood estimator is derived as

72


73


The pdf of is obtained as

Cramer-Rao Lower Bound is

Is the ML estimator efficient?

74


An efficient estimator does not exist due to the rule “if an efficient estimator exists, the ML procedure will produce it”.

Simulation results

75


Symmetric delay assumption was less realistic Better estimation techniques can be employed Using the transformed data,

the linear model can be written as

76


and

Hence, the covariance matrix is

Final form of estimator

77


The likelihood function in symmetric case is

MVUE for the clock offset is derived and shown to coincide with BLUE-OS and MLE

Its variance is give by

78


Similarly, for asymmetric link delays, the BLUE-OS is

which is also the MVUE Its variance is given by

79


MMSE estimator can be derived as a function of MVUE Closed-form expression for


MSE of clock offset

80

Clock Offset and Skew in a Receiver-Receiver Protocol

Main sources of errors – send time and channel access time – are removed

A receiver-receiver model can be represented as

81


The likelihood function can be expressed as

where

Objective function to be maximized is

over the constraints

82


JML estimator

83

Clock Offset and Skew in a Receiver-Receiver Protocol Why Gibbs Sampler?

Joint ML estimator is biased MVUE does not exist since the sufficient statistics depend on

unknown parameters

Posterior distribution can be found and clock parameters can be estimated by its mean for better results

Straightforward extension to additional unknown parameters, e.g., clock drift

Posterior distribution involves complex integrations, hence the Markov-Chain Monte Carlo (MCMC) methods

84


Algorithm for Gibbs Sampling is to iterate the following with initial values :

After a threshold value , the set behaves as the sample values from the joint posterior

85


In the current scenario, the Gibbs Sampler is implemented as follows

Performs better than the JML estimator Simulated with

MVUE as the lower bound with one parameter known BLUE as the upper bound due to linearity constraints

86


Simulation results

87

Conclusions

General exponential family model Accumulated error analysis for multihop protocols Effect of mobility on time synchronization

88

Future Research Directions

General exponential family model Accumulated error analysis for multihop protocols Effect of mobility on time synchronization

Documents

Qasim M. Chaudhari and Dr. Erchin Serpedin Department of Electrical and Computer Engineering