1

VarianceReduction

Techniques

2

Outline

Importance of Variance Reduction Types of Variance Reduction

Techniques Common Random Numbers Example: Common Random Numbers Implementing Common Random

Numbers in Arena Antithetic Variates Implementing Antithetic Variates in

Arena Control Variates

3

Reasons for Importance

Associated with an estimate of a performance measure (e.g., mean time in the emergency room for a patient).

The variance is a measure of the amount of uncertainty in an estimate -- the smaller the variance, the less the uncertainty.

The smaller the variance, the easier it is to distinguish between/among (i.e., rank, or select the best) alternative systems/policies.

To reduce the variance of an estimate, one could:1. Make more replications.2. Use one of several variance reduction

techniques.

(The latter is the preferred approach, especially if computational time is a constraint).

4

Types of Variance Reduction Techniques

1. Common Random Numbers (CRN)

2. Antithetic Variates (AV)

3. Control Variates (CV)

4. Others: Indirect Estimation, Conditioning

5

Common Random Numbers

Applies only when comparing two or more alternative systems

Most Popular VRT

Basic idea: compare alternative system configurations using “similar experimental conditions” (i.e., the “same” set of random numbers)

A form of “blocking”

Also called correlated sampling or matched streams

“Synchronization” of random numbers across different alternatives is important

A pilot study may be appropriate, since “backfiring” is a possibility

Formal statistical analysis can be made complicated with CRN

6

Justification for the Use of Common Random Numbers

Let Xij be the observation for the jth replication of the ith alternative (i=1, 2; j=1, 2, ..., n).

If the two sets of replications are done independently, then Cov(X1j, X2j)=0, but if positive correlation is induced with CRN, then Cov(X1j, X2j) >0.

Z X X

Z n Z n X X

j j j

j j jj

n

1 2

1 21

( ) / ) )., is an unbiased estimator of E( E(

Since the s are i.i.d:

Var [ ( )] = Var (

= Var ( Var( Cov( ,

Z

Z n Z n

X X X X

n

j

j

j j j j

'

) /

) ) )1 2 1 22

7

Common Random Numbers

“Synchronization” is an important aspect in the use of CRN. For example, you would want to dedicate particular streams of random numbers for particular purposes (e.g., for interarrival times) for each alternative.

8

Example:Common Random Numbers

Suppose we have a queuing situation in which two alternative system configurations are being considered:Alternative 1: 1 server, with a processing time that is exponentially distributed, with a mean of .9 minute.Alternative 2: 2 servers, each with a processing time that is exponentially distributed, with a mean of 1.8 minute.

Assume that the interarrival time for customers is exponentially distributed, with a mean of 27 seconds, and the queuing discipline is FIFO.

Suppose that the performance measure of interest is the average time in the queue of the first 100 customers.

(This model will be called the ATM model).

9

Example: Common Random Numbers (continued)

Running each of these models for 10 replications with no variance reduction technique gave the following results for average time in the queue for the first 100 customers.

j X1j X2j Zj = X1j - X2j_____________________________________________________________

1 5.04 1.02 4.022 6.10 3.03 3.073 6.26 1.85 :4 1.77 2.815 2.43 3.406 3.20 1.257 11.96 2.638 4.00 2.879 12.58 3.71

10 11.97 2.18

Sample Mean 6.53 2.48 4.06Sample Variance 17.2 .78 16.5695% CI Half-Width 2.97 .63 2.91

10

Example: Common RandomNumbers (continued)

Now, implementing Common Random Numbers, through theuse of the SEEDS module (from the Elements panel), weobtain the following output for average time in the queue forthe first 100 customers. (See the following pages for how thisimplementation was accomplished).

j X1j X2j Zj = X1j - X2j_____________________________________________________________

1 5.71 5.17 .542 2.70 2.29 .413 .53 .41 .124 5.78 5.64 .145 8.77 7.82 :6 2.45 2.017 4.18 3.468 .83 .659 5.17 5.00

10 1.65 1.38

Sample Mean 3.78 3.38 .395Sample Variance 6.81 6.00 .07695% CI Half-Width 1.87 1.75 .197

11

Example: Common Random Numbers

Note how the sample variance for the difference was decreased from 16.56 to 0.076 ( a decrease of 99.5%) and how the half-width for the 95% confidence interval for the difference between the two alternatives was decreased from 2.91 to 0.197 ( a decrease of 93%).

In summary, we have:95% CI without CRN: (1.15, 6.97)95% CI with CRN: (0.319, 0.592)

12

Implementing Common Random Numbers in Arena

By default, Arena uses stream 10 to generate all of its random numbers for a series of simulation replications. This is not what you want for CRN.

Use the SEEDS module from the Elements panel to implement CRN. This allows us to name and dedicate different streams for different purposes in a model

Using the Common selection for the Initialize Option spaces the seeds for each replication within each stream 100,000 random numbers apart, so that there will be no overlap of random number usage within a stream across replications.

Generally, do not use stream 10 when using this approach.

The SEEDS module allows you to dedicate the streams 1,2,… in series.

13

Each location in the model which involves generation of random variates needs to be modified. For

example, instead of EXPO(5)

for interarrival time in a Arrive

module, one would use EXPO(5, Stream

Interarrivals)where Stream Interarrivals is defined as a particular stream in the SEEDS module.

To implement CRN, the SEEDS module should be employed in all alternative model runs.

Implementing Common Random Numbers in Arena

14

Example: Implementing CRN in the Arena model

For each of the two alternative models, 1) attach the ELEMENTS panel, and place the

SEEDS module in the model window.

2) Name and dedicate streams for Interarrival Times and Processing Times, by adding the elements:i) for stream 1:

Identifier: Interarrival Times StreamSeed Value: (default)Initialize Option: Common

ii) for stream 2: Identifier: Processing Times StreamSeed Value: (default)Initialize Option: Common

3) In the Arrive module, replace the EXPO(1) in the Time Between field with EXPO(1, Interarrival Time Stream)

15

4) In the Server module, replace the Process Time in the single server model with EXPO(0.9, Processing Time Stream) and in the two server model with EXPO(1.8, Processing Time Stream).

5) Save the observations for the average value of the time in queue for each model to one file for the single server model and to another file for the two-server model, by modifying the Outputs section of the Statistics module.

Example: Implementing CRN in the Arena model

16

After running each of the models, we can use the output analyzer to compare the alternatives:1) Select Tools/Output Analyzer.2) Select File/New.3) Add Data from Files4) Select Analyze/Compare Means.5) Select

Data File AReplications: LumpedData File BReplications: Lumped

We obtain the following results for a 95% paired-t confidence interval:

Std Dev = 0.275CI: [0.198, 0.592]Reject Ho: means are not equal

at the 0.05 level.

X X1 2 0 395 .

Example: Implementing CRN in the Arena model

17

Antithetic Variates (AV)

Useful in reducing the variance of an estimate for a single alternative.

Basic idea: make pairs of runs for a single alternative so that a “small” observation for one run is offset by a large observation for the second run of the pair.

Operationally, rn’s for run 1 of a pair: r1, r2, r3, …

rn’s for run 2 of a pair: 1-r1, 1-r2, 1-r3,…

Note that both streams are i.i.d., u(0,1) rn’s, hence everything is “valid” with respect to each run.

Synchronization must be used.

18

Implementing Antithetic Variates in Arena

This is much like implementing CRN in Arena, the only difference is that you need to specify Antithetic in the Initialize Option field if the SEEDS module. This will give you Antithetic pairs in your replications.

For example, implementing the antithetic option in the single-server ATM model used for the CRN example would give us the following results for average queuing time (the AV model is contained in the file av1).

19

Replication No Variance Reduction Antithetic Variates1 5.04 6.832 6.10 3.153 6.26 7.164 1.77 2.515 2.43 3.416 3.20 2.687 11.96 0.588 4.00 7.259 12.58 2.4310 11.97 1.99

Sample Mean 6.53 3.8Sample Variance 17.2 5.795% CI Half Width: 2.97 1.71

Note how the sample variance was reduced by 67% (from 17.2 to 5.7) and how the confidence interval half-length was reduced by 42% (from 2.97 to 1.71) by using antithetic variates.

Implementing Antithetic Variates in Arena

20

Control Variates

The basic idea is to take advantage of correlation between certain random variates, in order to reduce the variance of the output. For example, suppose we have a queuing system with:-- Interarrival Time Mean (specified) = 3 min.-- Interarrival Time Mean (actual, from model

run) = 3.21 min.-- Mean Time in system(estimated, from model) = 3.4 min.

Since the mean interarrival time from the model (3.21) is greater than it should be (3.), one might expect that the sample mean time in the system (3.4 minutes) is less than it should be. We should adjust this time (3.4 minutes) upwards; the question is, by how much?

21

Important factors to consider include:

The correlation between the control variate and x should be high,

The control variate itself should have a low variance.

Choosing Control Variates