35
Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7 th International Workshop on Agent-based Mobility, Traffic and Transportation Models, Methodologies and Applications Mads Paulsen, Thomas Kjær Rasmussen & Otto Anker Nielsen Division of Transport Modelling, DTU Management Engineering, Technical University of Denmark May 10, 2018 [email protected] Random Seeds Output Variability in MATSim 1 / 33

Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Output Variability Caused by Random Seeds in aMulti-Agent Transport Simulation Model

7th International Workshop on Agent-based Mobility, Traffic andTransportation Models, Methodologies and Applications

Mads Paulsen, Thomas Kjær Rasmussen & Otto Anker Nielsen

Division of Transport Modelling,DTU Management Engineering,Technical University of Denmark

May 10, 2018

[email protected] Random Seeds Output Variability in MATSim 1 / 33

Page 2: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Random Seeds

Normally random seeds do not receive much attention.

But should they?

[email protected] Random Seeds Output Variability in MATSim 2 / 33

Page 3: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Motivation

Transport model outputs are fundamentalfor policy support.Their results need to be trustworthy.

Do random numbers hinder this?

[email protected] Random Seeds Output Variability in MATSim 3 / 33

Page 4: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Outline

1 Literature Review2 Methodology3 Case Study & Results4 Conclusions & Future Work

[email protected] Random Seeds Output Variability in MATSim 4 / 33

Page 5: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Literature Review

Outline

1 Literature Review2 Methodology3 Case Study & Results4 Conclusions & Future Work

[email protected] Random Seeds Output Variability in MATSim 5 / 33

Page 6: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Literature Review

Selected Studies

Nagel, 1997∗

Two seeds.TRANSims.Solely observesdifference.

Castiglione et al., 2003†

100 seeds.San Francisco model.Use partial means, andthus incomparable.

Cools et al., 2011‡

200 seeds.FEATHERS.Trip level:C .V .′s . 1 %.

∗Nagel, Kai (1997). “Experiences with iterated traffic microsimulations in Dallas”.In: Traffic and granular flow ’97. Ed. by Michael Schreckenberg and D. E Wolf.Heidelberg: Springer, pp. 199–214. arXiv: 9712001v1 [arXiv:adap-org]

†Castiglione, Joe, Freedman, Joel, and Bradley, Mark (2003). “SystematicInvestigation of Variability due to Random Simulation Error in an Activity-BasedMicrosimulation Forecasting Model”. In: Transportation Research Record: Journal ofthe Transportation Research Board 1831, pp. 76–88

‡Cools, Mario, Kochan, Bruno, Bellemans, Tom, Janssens, Davy, and Wets, Geert(2011). “Assessment of the effect of micro-simulation error on key travel indices:Evidence from the activity-based model Feathers”. In: 90th Annual Meeting of theTransportation Research Board. Washington D.C.

[email protected] Random Seeds Output Variability in MATSim 6 / 33

Page 7: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Literature Review

Additional Studies – Conclusions

More or less consensus in the literature that the Monte Carlo variabilityis a non-issue for link loads.

Aditionally backed up by Veldhuisen et al., 2000∗, Lawe et al.,2009†and Ziems et al., 2011‡.

‡Veldhuisen, Jan, Timmermans, Harry, and Kapoen, Loek (2000). “MicrosimulationModel of Activity-Travel Patterns and Traffic Flows: Specification, Validation Tests,and Monte Carlo Error”. In: Transportation Research Record: Journal of theTransportation Research Board 1706, pp. 126–135.

‡Lawe, Stephen, Lobb, John, Sadek, Adel, Huang, Shan, and Xie, Chi (2009).“TRANSIMS Implementation in Chittenden County, Vermont”. In: TransportationResearch Record: Journal of the Transportation Research Board 2132, pp. 113–121.

‡Ziems, Sarah Ellie, Sana, Bhargava, Plotz, Joseph, and Pendyala, Ram M. (2011).“Stochastic Variability in Microsimulation modeling Results and Convergence ogCorridor-Level Characteristics”. In: Transportation Research Board 90th AnnualMeeting. Vol. 11-3560.

[email protected] Random Seeds Output Variability in MATSim 7 / 33

Page 8: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Literature Review

A Very Similar Study...

Horni et al., 2011 ∗

Discovered after final submissiondeadline.

Thus not referenced in theproceedings paper.

Actual study on MATSim.30 random seeds.Link load average C .V . ≈ 4 %.

∗Horni, Andreas, Charypar, David, and Axhausen, Kay W (2011). “Variability inTransport Microsimulations Investigated With the Multi-Agent Transport SimulationMATSim”. In: ETH Library. Working Paper.

[email protected] Random Seeds Output Variability in MATSim 8 / 33

Page 9: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Literature Review

What’s Missing?

ContributionsCompare between-seed variation towithin-seed variation.Analyse each seed separately.Larger dataset for link load analysis(network and number of seeds).

[email protected] Random Seeds Output Variability in MATSim 9 / 33

Page 10: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Outline

1 Literature Review2 Methodology

MATSimExperiment Design

3 Case Study & Results4 Conclusions & Future Work

[email protected] Random Seeds Output Variability in MATSim 10 / 33

Page 11: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

The Multi-Agent Transport Simulation MATSim

MATSim∗

Agent-based.Activity-based.Large-scale applicable.Scenarios around the world.Open source.

∗Horni, Andreas, Nagel, Kai, and Axhausen, Kay W, eds. (2016). The Multi-AgentTransport Simulation MATSim. London: Ubiquity Press.

[email protected] Random Seeds Output Variability in MATSim 11 / 33

Page 12: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

How Are Random Numbers Used in MATSim?

Used to Determine which...... in-going links of a node to be handled first.... agents use which...

... plan mutation strategy.

... plan selection strategy.... plan an agent choose (from choice set).

For choosing between reasonable alternatives.MATSim ought note to be particularly prone tohigh output variability.

[email protected] Random Seeds Output Variability in MATSim 12 / 33

Page 13: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Experiment Design

100 runs (100 different random seeds).100 iterations in each run.A total of 100× 100× |L| link loads produced.x s,i

l is daily link load on link l in iteration i with seed s.

It. 1 It. 2 · · · It. 98 It. 99 It. 100Seed 1 x1,1

l x1,2l · · · x1,98

l x1,99l x1,100

lSeed 2 x2,1

l x2,2l · · · x2,98

l x2,99l x2,100

l...

...... . . . ...

......

Seed 99 x99,1l x99,2

l · · · x99,98l x99,99

l x99,100l

Seed 100 x100,1l x100,2

l · · · x100,98l x100,99

l x100,100l

We (primarily) use those from the last iteration.

[email protected] Random Seeds Output Variability in MATSim 13 / 33

Page 14: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Experiment Design

100 runs (100 different random seeds).100 iterations in each run.A total of 100× 100× |L| link loads produced.x s,i

l is daily link load on link l in iteration i with seed s.

It. 1 It. 2 · · · It. 98 It. 99 It. 100Seed 1 x1,1

l x1,2l · · · x1,98

l x1,99l x1,100

lSeed 2 x2,1

l x2,2l · · · x2,98

l x2,99l x2,100

l...

...... . . . ...

......

Seed 99 x99,1l x99,2

l · · · x99,98l x99,99

l x99,100l

Seed 100 x100,1l x100,2

l · · · x100,98l x100,99

l x100,100l

We (primarily) use those from the last [email protected] Random Seeds Output Variability in MATSim 13 / 33

Page 15: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Measures I – C.V.

Our initial analysis measure is the coefficient of variation,

cxlv = Standard Deviation

Mean

=

√1|S|−1

∑s∈S

(x s,100

l − xl)2

xl, l ∈ L.

With xl being the mean link load of link l ∈ L,

xl =

∑s∈S

x s,100l

|S| , l ∈ L.

[email protected] Random Seeds Output Variability in MATSim 14 / 33

Page 16: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Measures II – Proportions

We introduce additional measures:Proportion of links that deviate more than q · 100% from their mean whenusing seed s ∈ S,

r sq =

∑l∈L

[|x s,100

l −xl |xl

> q]

|L| , s ∈ S, q > 0.

Proportion of seeds in which the link load of link l ∈ L deviates more thanq · 100% from the mean,

r lq =

∑s∈S

[|x s,100

l −xl |xl

> q]

|S| , l ∈ L, q ≥ 0.

[email protected] Random Seeds Output Variability in MATSim 15 / 33

Page 17: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Measures IIIa – Between-Seed vs Within-Seed Variation

Finally, we introduce two auxiliary measures:Between-seed variation:

Bl = 1|S| − 1

∑s∈S

(x s,100

l − xl

)2, l ∈ L.

Within-seed variation:

Wl = 1|S|∑s∈S

(x s,99

l − x s,100l

)2, l ∈ L.

[email protected] Random Seeds Output Variability in MATSim 16 / 33

Page 18: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Measures IIIb – Between-Seed vs Within-Seed Variation

Bl =

It. 1 · · · It. 99 It. 100 Between-seed mean

Seed 1 x1,1l · · · x1,99

l

(x1,100

l − xl)2

...... . . . ...

......

Seed 99 x99,1l · · · x99,99

l

(x99,100

l − xl)2

Seed 100 x100,1l · · · x100,99

l

(x100,100

l − xl)2

Wl =

It. 1 · · · It. 98 It. 99 It. 100Seed 1 x1,1

l · · · x1,98l

(x1,99

l − x1,100l

)2

...... . . . ...

......

Seed 99 x99,1l · · · x99,98

l

(x99,99

l − x99,100l

)2

Seed 100 x100,1l · · · x100,98

l

(x100,99

l − x100,100l

)2

[email protected] Random Seeds Output Variability in MATSim 17 / 33

Page 19: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Methodology MATSimExperiment Design

Measures IIIc – Between-Seed vs Within-Seed Variation

These allow calculating the importance of the between-seed variation,

Rl =√

Bl + WlWl

, l ∈ L.

Rl → 1 ⇒ Not important.Rl =

√2 ⇒ Equally important.

Rl �√

2 ⇒ Strongly dominates the within seed-variation.Thus, Rl can indicate whether it is better to...

... run more iterations (low Rl ) or

... run more seeds (high Rl ).

[email protected] Random Seeds Output Variability in MATSim 18 / 33

Page 20: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Outline

1 Literature Review2 Methodology3 Case Study & Results

Case StudyResults

4 Conclusions & Future Work

[email protected] Random Seeds Output Variability in MATSim 19 / 33

Page 21: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Case Study

Santiago de Chile Open Data Scenario v2b∗

10% Population Sample: 665,201 agents.22,981 link network.Schedule based public transport.

Default “Out-of-the-Box” ConfigurationsMode choice: Walk, public transport, car.100 iterations.After 80 iterations:

Choice sets are locked.Plan level MSA initialised.

∗Kickhofer, Benjamin, Hosse, Daniel, Turner, Kai, and Tirachini, Alejandro (2016).“Creating an open MATSim scenario from open data: The case of Santiago de Chile”.In: VSP Working Paper 16-02. VSP Working Paper

[email protected] Random Seeds Output Variability in MATSim 20 / 33

Page 22: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Coefficients of Variation: All links.

Generally not high – somewhat in accordance with earlier studies.2/3 of links have a coefficient of variation ≤ 5 %.∼ 15 % of links have a coefficient of variation > 10 %.

0.00 0.05 0.10 0.15 0.20 0.25

0.0

0.2

0.4

0.6

0.8

1.0

cvxl

cdf c

vxl

Empirical DataLog−Normal Distribution

cxlv =

√1|S|−1

∑s∈S

(x s,100

l −xl)2

xl, l ∈ L.

[email protected] Random Seeds Output Variability in MATSim 21 / 33

Page 23: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Coefficients of Variation: 50 busiest links.

The 50 busiest links seem more stable.Coefficient of variation ≈ 1 %.

3000

3500

4000

4500

5000

5500

l

x lS [

veh.

]

Maximum value within 1.5 IQR of Q3

Q3

Median

Q1

Minimum value within 1.5 IQR of Q1

Outlier below(/above) 1.5 IQR of Q1(/Q3)

IQR = Q3 − Q1

cvxl :

1.19

%1.

05 %

1.02

%0.

74 %

1.07

%1.

09 %

1.09

%1.

46 %

0.93

%1.

08 %

0.86

%1.

03 %

1.55

%2.

20 %

0.97

%1.

22 %

0.70

%1.

01 %

1.01

%1.

23 %

2.89

%0.

90 %

1.53

%2.

74 %

1.55

%0.

72 %

1.18

%1.

22 %

0.98

%0.

94 %

1.32

%5.

05 %

1.72

%1.

45 %

1.04

%1.

00 %

0.77

%1.

17 %

2.21

%1.

69 %

2.67

%1.

71 %

1.81

%1.

68 %

2.11

%1.

81 %

2.24

%2.

32 %

2.21

%2.

24 %

[email protected] Random Seeds Output Variability in MATSim 22 / 33

Page 24: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Are Some Seeds Worse than Others?

All seeds perform more or less equally well.... i.e. 42 will do just fine.

●●

●●●

●●

1% 2.5% 5% 10% 15% 25% 50%

r qS

q

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8Mean:

SD:66.28 %

0.65 %42.16 %

0.70 %23.08 %

0.52 %10.19 %

0.33 %6.02 %0.21 %

3.18 %0.13 %

1.49 %0.06 %

r sq =

∑l∈L

[∣∣xs,100l −xl

∣∣xl

> q

]|L|

, s ∈ S, q > 0.

[email protected] Random Seeds Output Variability in MATSim 23 / 33

Page 25: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

How Often Does It Go Wrong? I

∼20 % of links have a relative error of ≥ 5 % for ≥ 50 % of the seeds.∼10 % of links have a relative error of ≥ 15 % for ≥ 20 % of the seeds.

rql

cdf r ql

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

q = 50%q = 25%q = 15%q = 10%q = 5%q = 2.5%q = 1%

r lq =

∑s∈S

[∣∣xs,100l −xl

∣∣xl

> q

]|S|

, l ∈ L, q ≥ 0.

[email protected] Random Seeds Output Variability in MATSim 24 / 33

Page 26: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

How Often Does It Go Wrong? II

Generally worst for the smallest links.However, many busy links are quite volatile.

0.0

0.2

0.4

0.6

0.8

1.0

xl

r ql

10 100 1,000 10,000

● ●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●● ●●

●●

●●

●●

●●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●●

●●

●●

●●

●●

● ●

●●

● ●

●●●●●

●●●

●●● ●

● ●

●●

●●●

●●

●●

● ●

●●●

● ●●●●

●●

●● ●

●●

●●●

● ●

●●

●●

●● ●

● ●

● ●

●●

●●

●●

●●

● ●●

●●

● ●

●●

● ●

●●

●●●

●●

● ●

●●

●● ●●

●●●

●●

●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●● ●

●●●

●●

●●

●●

● ●

●●

●●●

●●

●●●

●● ●

●●

●●

● ●●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●●●● ●● ●

● ●

●●

●●

●●

● ●●

● ●

●●●●

● ●●●

●●

●●●

● ●

● ●

●●

●●

●●●●

●●

●●

●●

●●

●●

●● ●●●

●●

● ● ●

●●

● ●

●●

●●

●●

● ●●

● ●

●●

●●

● ●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●● ●●

●●

●●

● ●

● ●

●●

●●

●●●

●●

●●

● ●● ●●

●●

● ●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●●

●●

●●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●●●●●

●●

● ●

● ●● ●

●●

●●

●●

●●●

● ●

●● ●●

●●

●●●

●●

●●

●●●

● ●●

●●

● ●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●● ●●

●●

● ●● ●

●●

● ●

●●

●● ●

●●

●●

●●

●●●●

●●●

●●

●●

●●

●●●

●●

●● ●

●●

●●

● ●

●●

● ●●●●●●

●●

● ●

●●

● ●

●●●

●●●●

●●●

●●● ●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

● ●

●●

●●

● ● ●

●●

● ●

●●●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

● ●

●●

●●

●● ●●●

● ●●

●●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●● ●

●●

●●

●●●

●●

●●

●●● ●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●● ●

●●

●●

●●

●●

●●●

●●●●

●● ●

●●

●●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●● ●●

● ●

●●

●●● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●●

● ●●

●●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●● ●

●●●

●●●

●●

● ●

●●

●●

●●

●●

● ●

● ●●

●●●●

●●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●●●

● ●

●●

● ● ●●

●●

● ●

● ●●

●●●

●●

●●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●●

● ●●●

● ●●

● ●

●●

●●

●●

●●

●●

●●

●● ●

● ● ●

●●●●●●

●●●

●●●

●●

●●

● ●●

●●

● ●

●●

●●

● ●

●●

●●●

● ●●

●●

● ●

● ●

●●

●●

● ●

●●

●●

●●●●

●●

●●

●●

●●●

● ●

●●

● ●●

● ●●

● ●

●●

● ●

●●

●●

● ●●

●●

●●●

●●

● ●

●●

● ●● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●●

●●

●●●

●●●

●●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

● ●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●● ●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

● ●

●●

●●● ●

●●

●●

●●●

●●

●●●

●●

●●●

●●

●●

●●●

●● ●

●●

●●

●●

● ●

● ●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●●

● ●

●●

●●

● ●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●●●

●●

●●●●●

●●●

●●

●●●

● ●●●

●●

●●●●

●●

●●●

●● ●

●●●

● ●

●●

● ●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●●

● ●

●●

●●

● ●●

●●

● ●

●●

● ●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●● ●

●●

●●●

●●

●●●

●●

●●●

●●●

●● ●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●●●●

●●

●●

●●

● ●

● ●

●●

● ●●

●●

●●

●●

●●●●

●●●

●●

●●●

● ●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●● ●

●●

● ●

●●

●●● ●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●●

●●

●●

●●

●●

●●

●●

●● ●●

●●

● ●

●●

●●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

● ●●

●● ●

●●●

●●

● ●●

●●

●●

●●

●●

●●●●● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●●

●●●

●●●●

●●

●●●

● ●

●●●●

● ●●

●●

● ● ●

●●

●●

●●●

●●●●

●●●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●●

● ●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●●

● ●

●●

● ●

●●●●

● ●

● ●

●●

●●

●●

●●● ●

●●

●●

●●

●●●

●●●

●●

● ●

●●

●●

●●

●●

●●●●

● ●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●

● ●

●●

●●

●●

●● ●●●

●●

●●

● ●●

●●

●●

●●●

●●●

●●●

●●●

●●

●●

●●●●

●●

●●●

● ● ●

●●●

●●

●●

●●

●●●

●●

● ●

● ●

● ●●●

●●●

●●

●●

● ●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

● ●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●

●●●●

●●●●

● ●

● ●

●● ●●

●●●

●● ●

●●

●●

● ●

●● ●

●●●

●● ●

●●

●●

●●

●●●

●●

●●

●● ●

●●

●●

●●●

●●

●●●

● ●

● ●●

●●

●●

●●

●●●●●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

● ●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

● ●●

●●

●●

●● ●● ●● ●

● ●

●●

●●

●● ●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●● ●

●●

●●

● ●●●

● ●●●

●●●●●

●●

● ●●●

●●

●●

●● ●●

●●

●●

●●

●●

●●

●●●

● ●

● ●

●●

●●●

●●●

●●

●● ●

●●

●●

● ●

●●●

●●

●●

●●●

● ●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●● ●

●● ●

●● ●

●●●●

● ●

● ●●

●●

●●

●●●

●●

● ●●

●●

●●

●●

●●

● ●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●

● ●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●● ●

●●

●●

●● ●

● ●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●●

●●

●●

● ●●

●●●

●●●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

● ●

● ●

●●

●●

● ●●

●●

●●

● ●

●●●

●●

●●●

●●● ●

●●

●●

●●

●●

● ●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●● ●

●●

●●●

●● ●

●●

●●

●●

● ●

●●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●●

● ●

●●

● ●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●●●

●●

●●

●●● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●● ● ●

●●

●●●

●●

●●

●●

●●

● ●●●

●●

●●

●●

●●

●●●●●

●●

●●●●●●

● ●

●●●

●●●

●●●

●●●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●● ● ●

●●

●●●

● ●

●●

●●

●●

●●● ●●

● ●

●●

●● ●

●●

●●

●●

●●

● ●

●●

●●

●● ●

● ●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●● ●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ● ●

●●●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●● ●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●●

●●

●● ●

●●

● ●

●●

●●

●●

●●

●●

●● ●●

●●

● ●

● ●

●●

●●

●●

●●

●●●

●●

● ● ●● ●●

●●

●●

●●●

●●

●●

●●●

●●●

●●

●●●●

● ● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●● ●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●● ●

●●

●●

●●

●●

●●

●● ●

● ●

●● ●●●

●●

●●●● ●

●● ●

●●

●●

● ●

●●

●●●

●●

●●

●●

● ●● ●●

●●

●●

●●

●●

● ●

● ●

●●

● ●

●●

●●

● ●

●●

●● ●●

●●

●●

● ● ●● ●

● ●

●●

● ●

● ●

●●

●●

● ●

●●●●

● ●

●●

●● ●

●●

● ●

●●●

●●

●●

●●

● ●

●●

●●

● ●

● ●

●●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

● ●

● ●

● ●

●●

●●

●●

● ●

● ●

●●

●●

●●

● ●

●●●

●●

●●

●● ●

●●

● ●●

●●

●●

●●

●●

● ●

● ●

●●●

● ●

● ●

●●

●●

●●

● ●

●●

●● ●●

●●

● ●● ●

●●

●●

● ●

● ●●

●●

●●

●●

●●

● ●●

●●

●●

●●●

●●

●●

●●

●●●

● ●

●●●●

●●

●●● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

● ●

● ●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●● ●

●●

●●●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●●

●●

●●

●● ●

●●

●●

●●●●

●●

●●●

●●

●●●●●●●

●●

●●

● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●●

●●

●●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

● ●

●●●

●●

● ●

● ● ●

● ●

●●

●●●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●● ●

● ●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●●

●●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●● ●

●●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●●

●● ●●

●●

●●●

●●

●●

●●● ●●

●●

● ●

● ●●

●●

●●

●●●

●●●

●●

●●●

●●●

● ●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

● ●●

●●●

● ●

●●

●● ●

●●

●●

● ●

● ●

● ●

●●

● ●

●●●

●●

● ●

●●

●●●

●●

●●

● ●

●●●

●●

●●

●●

●●●●

●●

●●

●●●

● ●

●●●●

●●

●●

●●●

●●

●● ●

●●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●●●

●●

●●

● ●

●●

● ●

● ●●

●●

●●

●●

● ●●

●●

●●●

●●

●●● ●

● ●

●●●

● ●

●●

●●

●●●

●●

●●●

●●

● ●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

● ●

●●

●●

● ●

●●●

●●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●●

●●

●● ●

●●

● ●

●●

●●

● ●● ●

●●●●

●● ●

●● ●●

●● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

● ●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●● ●

● ●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

● ●

●● ●

● ●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●●

●●

●●

● ●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ●●

●●

● ●●

●●

●●

●●

● ●●●

●●●

● ●

●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●●

●● ●● ●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●● ●

●●

●●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●● ●

●●

●●

● ●●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

● ●●

●●

●●

● ●

● ●

●●

●●

●●

●●●

●●

●●

●●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●● ●● ●● ●●●

●● ●●

●●

●●●

●●

●● ●

● ●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

● ●

●●

● ●●●

●● ●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●●●

●● ●

●●

● ●

●●

● ●

● ●

●●●

●●

●●

●●

●●

●●

●●

● ●● ●

●●

●●

●●

●●

● ●

●●●●

●●

●●

●●

● ●●

●● ●

●●●●

●●

●●●

●●●

● ●●

●●

● ●

●●

● ●●

●●●

● ●

●●

●●

●●

●●●

●●●

●●●

●●●

●●

●●

●●●

●●

●●

●●

●● ●

●●

●●●

●●

●●●

●●●

●●

●●

●●●

●●

● ●

●●

● ●

●●

●●

●●●

●●

●●●

●●

●●

●●

● ●●●

● ●

●●

●●

●● ●

●●

●●

●●

●●

●●

●● ●●

●●

● ● ●●

●●

●●●

● ●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●● ●●

● ●●

●●

●●

●●

●●

●●

●● ●●●●

● ●

● ●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●● ●

●●●

●●●●

●●●

● ●

●●

●●●

●●

●●●●

●●

●●●

●●

● ●●

●●

●●

●●

● ●●●

●●●

●● ●

● ● ●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

● ●

●●●●

●●

●●●

●●

● ●

●●● ●

●●

●●

●●

● ●

●●● ●

●● ●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●●

●●

●●

●● ●

●● ●

● ●

●●

●●

●● ●

● ●●

●●

●●

●● ●●

●●

●●

●●

●● ●

●●

●● ●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●●●

●●

●●●

● ●●●

●●

●●

● ●

●●

●●●

● ●

●●●

●●● ●

●●

●●

●●●

●● ●●●● ●●

●●

●●

●●

●●●

●●

●●●

● ●

● ●

●●

●●

●●

●●

● ●

●●●

● ●

●●

●●

●●

●●

●●

●●●●●●

●●

● ●

●●

●●

● ●●

●●

●●

●●●

●●

●●

●●

● ●

● ●

● ●●

●●●

● ●●●

● ●

● ●

● ●●

● ●●

●● ●

●●

●●

● ●●

●●

●●

●●

●●

●● ●

● ●

●●

●● ● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●● ●●●●●

● ●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●● ●● ●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

● ● ●

● ●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●● ●

● ●

●●

●●

●●●

●●●

●●●●●

● ●●

●●

●●

●●●●

●●●●●●●●

●●

●●

● ●

●●

● ●

●●●

●●

●●●

●●

●●

●●

●●

● ●

● ●

●●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●●

●●

●●

● ●

●● ●●

●●

●●

●● ●

●●

●●

●●

●●●

●●

●●

●●

●● ●

●●

●●

●●●

● ●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●● ●●

●●

●●

●●●

●●

● ●●●

●●●

● ●

● ●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●● ●●

●●●

●●

●●

●●

●● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●● ●

●●

●●

●●

● ●

●●●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●●

● ●

●●

●●

●●

●● ●

● ●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●● ●●

●●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●

●●●●

●●●●

●● ●●

●● ● ●

●●

●●

●●●

●●

● ●

●●●

●●

● ●

●●●●

● ●

●●●

●●

● ● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●●●

●●

●●

●●

●●

●●

●●●●

●●●●

●●

●● ●

●●

● ●

●●●

●●

● ●

●●

●● ●●●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●●●

●●

●●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●●●

●●●

●●●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●● ●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●●

●● ●●

●●●●

● ●●

●●●●

● ●

●●

● ●●

●●

●●

●●

●●

● ●

●●

●● ●●

●●

●●

●●

●●

●●●

●●●

●●

●●● ●

●●

●●

● ●

●●

●●●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●●●●●●

●●

●●●●●

●●

●●

● ●

●●

●●● ●●

● ●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●● ●

● ●

●●

● ●

● ●

●●

●●

●●

●●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ● ●

●●

● ●

●●

●●

●●

● ●

●●

● ●

●●

●●●

●●

●●●

●●

●●

● ●●

●●

●●

●●

●● ●●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●●

●●

● ●

●●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

● ●●●●●

●●

●●●●●

●●

●●

● ● ●

●●

● ●

●●

●●

●●

●● ●

● ● ●●

●●

●● ●●●

●●

●●

●● ●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●●

●●

●●

●●●●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●● ●●

●●

●●

●●

● ●●

●●●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●●●

●●

●●

● ●●

●● ●

● ●

● ●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●●

●●●

●●

●●

● ●

● ●

●●

● ●

● ●

●● ●

●●●

●●

●●

●●●

●● ●●

●●

●●

● ●

●● ●●

●● ●●

● ●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●● ●● ●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●●

●●●

●●●

● ●

●●

● ●

●●

●●

● ●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●●

● ●●

●●●

● ●●

●●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

● ●●

●● ●

●●

●●

● ●

●●●●●

●●

● ●●●

●●●

●●

●●

● ●

●●

● ●

●●

●●

● ●

●●●

● ●●●

●●

●●

● ●

●●●●

●● ●●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●● ● ●●

●● ●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●● ●

●●●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

●●

● ●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●

●●

● ●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●●●

● ●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●●

●●

●●

●●●●●

●●●

● ●● ●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

● ●

●●

● ●●

●●

●●

●●

●●

●●

●●●

●●●

●●●●

●●●

●●

●●●●

●●

●●●

●●

●●

●●● ●

●●

●●●●

●●

●●●●

●●●

●●

●●●

●●

● ● ●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

● ●

● ●

●●

●●●

●●

●●

●●

●●

● ●

●●●●

●●●

●●

●●●

●●

●●●

●●

●●

●●

●●●

●●

●●●

● ●

●●

●●

●●

● ●

●● ●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

● ●

●●

●●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

● ●

● ●

●●

● ● ●

●●● ●●●

● ●

● ●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●●●● ●

●●

●● ●

●●

●●

●●

●● ●

●●

●●

●●

●●●

●●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●●

●●

●●● ●

● ●●●●

●●

●●● ●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●● ●

●●

● ●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●●●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●●

●●

●●

●●●

●●

●●

●●●

●●●●●

●●

●●

●●

●●

●●●●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

● ●●

●●

●●

●●

●●●

●●

● ●●

●●

● ●

●●

●●● ●

● ●

●●

●●●●

●● ●●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●●

●●

●●

●●●

●●

●●

●● ●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

● ●

●● ●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●●

●●

●● ●

● ●

●●

●●

●●●

●●

●●

● ●●

●●

●●

●●

●●

●●●●●●

●●

●●

●●●

●●

● ●●

●● ●

●●

● ●

●●

●●

●●

●●●

●●●●

● ●

●●

●●

●●

●●

●●●

●●●

●●

●●

●● ●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●●

●●

●●● ●

●●

● ●● ●

●●

●●

● ●

●●

●●

● ●

●●

●●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●● ●

●●

●●

● ●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

● ●

●●

●●●

●●

●●

●●

●●

● ●

● ●

●●

● ●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●●● ●●

●●

●●

●●

●●

●●

●●

● ●●●●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●●● ●●●●

●●

●●

●●● ●

●●

●●

●●

●●●

●●●

●●● ●

● ●

●●

●●●●

●●

●●

● ●

●●

●●

●●●

●●●

●●

● ●

●●

●●●

●●●

●●

●●

●●●

●●

●●

●●●●●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●

●●

●●●

●●●

●●●

●●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●●●

●●

● ●●

●●

●●

● ●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●● ●

●●

● ●

●●

●●

●●

●●●●

●●●

●●● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●●

● ●

●●●

●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●

●●●

●●

● ●

●●

●●

●●

●●

● ●

●●●

●●●

●●

●●

●●●●●●

● ●●

●●

● ●

●●

●●

●●

●●

●●●●

●●

●●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

● ●

●●●

● ●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●● ●

●●

●●

●● ●

●●●

●●●

●●

● ●

●●

●●

●●●

●●

●●●●●●

●● ●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●●

●● ●

●● ●● ●●

●●

● ●

●●

●●

●●

●●

●●

● ●

● ●

●●●

●●

●●

●●●●

●●

●●

● ●●

●●

● ●●●● ● ●

● ●●

● ●

●●●

●●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●●

●●● ●●

●●

●●

●●

●●

●●

●●

●●● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●●

●●

●●

●● ●●●

●●

● ●

●●

●●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●●

●●

● ●● ●

●●

●●●

●●

●●●

●●

●●●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●● ●

●●

●●●

● ●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

● ● ●●

●●

●●●

●●

●●

●●

● ●

● ●●

●●

●●

●●

●●

●●

●●●

●●

● ●

● ●

●●

●●

●●

● ●

●●

●●

●●

● ●

● ●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●● ●

●●

●● ●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●●

● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●●

●●

● ●

●●

●●

● ●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●●●

●●

●●

●●● ●

●●

● ●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●● ●

●●●

●●

●●

●● ●

●●

●●

●●

●●●●●

●●

●●

●●●●● ●●●●●●● ●●

●●●●●●●

●●

●●

● ●●

●● ●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●

●●

● ●

●●

●● ●

●●

●●

●●

●●

●●●●●

●●

●●●

●●●

●●

●●

●●●

●●

●●

● ●

●●

●● ●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●● ●

● ●

●●

●●●●

●●

●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

●● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

● ●

● ●

●●

●●

●●

● ●

●●

● ●●●

●●

●●

●●

● ●

● ●

●●

●●

● ●●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●●●

●●

● ●

●●●

●●

●●●● ●

●●

●●

●●●

● ●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●● ●

●●

●●

●●●●

●●

●●

● ●

●●

●●●●

●●●

●●

●●

●●

●●

●●

●●●

● ●●

●●

●●

●● ●●●

●●

●●

● ●

●●

●●

●●

●● ●

●● ●●

●● ●

● ●

● ●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●●● ●

● ●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●● ●

●●

● ●

●●●

●●

●●

●●

●● ●

● ●

●●●

●●

●●●

●●●

●●

●●

●●

●●●●

●●

●●

●●● ●●

●●

●●●●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

● ●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●● ●

●●

●●●

●●

● ●

●●●

●●●●

●●

●●

●●

●●●●●●

● ●

●●

● ●●

●●

●●

● ●●

●● ●●

●●

●●

●● ●

●●●

●●

●●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●●

●●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

● ●

●● ●

●●

●●●●●●

●●●

●●●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●● ●●

● ●

●●

●●

● ●

●●

●●

●●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●

●●

●● ●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

● ●

●●●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●●

●●

●●

●●

●●

●● ●●

●● ●●●

● ●●

●●

●●

● ● ●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ● ●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ● ●

●●

●●

● ●

●●●●

● ●

●●

● ●●

●●●

●●

●●

●● ●

●●

● ●

●●

●●

●●●

● ●●

●●

●●

●● ●

●●

●●●

●●● ●

●●

●●

●●

● ●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●● ●●

●●

●●

●●

●●

●●

●●

●●

●●●● ●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

● ●

●●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●●

●●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●●

●●

● ●●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●●

●● ●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●● ●

●●

●●●

●●

● ●

● ● ●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●●

● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●

● ●●

●●●●●● ●●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●●

●●●

●●●

●●

●●●●

●●

●●

●●●

● ●

●●

● ●

●●●

●●

●●

● ●

●● ●

●●●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●●

●●

●●●

●●

●●

●●

● ●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●●●

●●

● ●

●●

●●

●●

●●

●●●

● ●

●●●

●●

●●●●

●●●

● ●

●●

●●

● ●

●●●

●●

●●

●●

●●

●● ●

●●●

●●●

●●

● ●

●●

●●

●●● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●

● ●

●●●

●● ●

● ●

●●

●●

●●●

●●

●●

●●

● ●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●●

●●

● ●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●●

●●

●●

●●●

●●

●●

●●●

● ●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●● ●●

●●

●● ●

● ● ●●

●●

● ●● ●

● ●

● ●

●●

●●

● ●

●●

●● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

● ●●

●●

●●

●● ●●

●●

●●

●●

●● ●●●

●●●

●●●

●●

●●

●● ●

●●

●●

●●● ●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●● ●

●●●

●●●

●●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●● ●

●●

●●

●●

●●

●●

●●

● ●●●

●●

●●

●●

●●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●●●

●●

●●

●●●●

●●

●●

●● ●

● ●

●●

●●

● ●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●●

●●

● ●●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●●

●●

●●

● ●

●● ●

●●●

●●

●●

●●

●●

●●

●●●●●● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●● ●●

●●

●●

●●●

●●

●●●

●●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●●●

●●

●●●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●●

●●●

● ●●●●●

●●

●●

●●

●● ●●

●●

●●

● ●

●●

● ●

●●●

● ●●

●●

●●

●●

● ●

●● ●

●●

●●

●●

●● ●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●●●●

●●●●●●●● ● ●

●● ●

●●

●●

●●

●●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

● ●

●●●● ●●●●

●●

●●

● ●

●● ●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

● ●

● ●

●●

●●●●

● ●● ●

●●●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●● ●

● ●

● ●●

● ●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●●

●●

● ●

●●

●●●

●●●

●●●

●●

●●

●●

● ●

●● ●

●●

●●

●● ●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●●

●●

● ●

●●

●●●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●● ●● ●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●● ●●

●●●

●●●

●●

●●

●●

●● ●●

●●

●●

●●●●

●● ●

● ●●

●●

●●

●●

●●●

●●

●●● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

● ●

● ●

●●

●●●●●

●●●● ●●

●●●●●●

●●●

●●●

●●

● ●

●●●

●●

●●

●●

● ●

●●●

● ●

●●●

●●

● ●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

● ● ●●●

●●●

●●●

●●

●●

●●

●●

●●●●

●● ●

●●●●

●●●

●●

●●●

●●

●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●●●

●●

● ●

●●

●●

● ●●

●●

●●

●●●

●● ●●

●●

●●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●●

● ●

●●

●● ●

● ●

●●

●●

●●

●●

●●

●●

●● ●●

●●

● ●●●●

●●

●●●

●●●● ●

●●●●●●●●●●

●●●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●● ●

●●

●●

● ●

●●

● ●

●●

● ●

●●

●●●

●●

●●●

● ●

●●

●●

●●●

●●

●●●

●●

●●●

●●

● ●

●●

● ●

●●

●●

●●●

●●

●●●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●●

●●

●●

●●

● ●●

●● ●

●●

●●

●●

●●

● ●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

● ●●●

●●

●●●

●●●●●

●●

●●●

● ●

●●

● ●

●●

●●●

●●

● ●

●●

●● ●

●●

●●

●●●●

●●

● ●

●●

●●

● ●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●●

●●

● ●

● ●

●●

●●

●●

●●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●●

●●

●●●●●

●●

●●

●●

●●

● ●

●● ●

●●

● ●

●●

●●

●●

●●●●●

●●

●●

● ●

●● ●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●●

●●●

● ●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●●

● ●●

●●

●●●

●●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●● ●

●●

●●

●●

●●●●

● ●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●●●●●

●●

●●

●●

●●

●● ●●

●●

●●

●●●

●●

●●

● ●

●●

●●●

●●

●●

● ●

●●

●●●

●●●

●●

●●●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

● ●

●●

●●

● ● ●

● ●

●●

●●

●●

●●

●●●

●●

●●

● ●●

●●

●●

● ●●

● ●

●●

●●

●●

●● ●

●●

● ●●

●●

● ●●

● ●●

●●

●●●

●●●●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●● ● ●●

● ●

●●

●●

● ●

●●

● ●

●●

●● ●●

●●

● ●●●

●●●

●●

●●●

● ●●

●●

●●

●●

●●●

●●

●●●●●

●●●

●●●● ●● ●

●●

●●

●●

●●●

●●●

●●●●

● ●

●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●●

●●

●●

●●

●●

●● ●●●

●●

●●●

● ●

● ●●●

●●

●●

●●●

●●●●

●●●●

●●● ●●

● ●

● ●

●●

●● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

● ●

● ●

●●

● ●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

●● ●

●●

●●

●●

●●

● ●●● ●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●●●●●●●●

●●

● ●●

●●●

●●

● ●

●●

●●

●●

●●

●● ●●

● ●

●●● ●

● ●●●

● ●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●

●●

●●

●●●●●●

●●●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●●

●●●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

●●

●● ●

● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●● ●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●● ● ●

● ●

●● ●

●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●● ●●

●●

●●●

● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●● ●

●●

●●

● ●

●● ●

●●

●●

●● ●●●● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●●

●●●●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●● ●●

●●

●●

● ●

●●

●● ●

●●●

●●

● ●

● ●

●●

●●

●●●

●●

●●●●●●●

● ●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●

●● ●●●

●● ●

●●

●● ● ●● ●●●●●● ●●

●●

●●

●●

●●●●

●●

●● ●●

●●

●●

●●

● ●●

●●●

●●

●●

●●

●●●●●●

●●

●●●●●

●●

● ●

● ●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

● ●● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●●

●●

●●● ●●

●●● ●

● ●

●●

●●

●●

●●

●●

● ●●●

●●

●● ●

●●

●●●

● ●

●●

●●

● ●●

● ●

● ●●

●●

●● ●●●

●●

● ●

●●

●●

●●

●●

● ●

●●

● ●● ●

● ●

●●●

●●●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

● ●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●● ●

●●

●●

●●●●

●●

●●

●●

●●●

●●●

●●

● ●●

●● ●

●●

● ●●

● ●

●●

●●

●●●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●●

●●● ●●

●●

● ●

●●

●●

●●●

●●●●●●

●●●

●●●

●●●

●●

●●

●●

●● ● ●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●● ●

● ●

●●

●●

●● ●●●

●●

●●

●●

● ●

●●

● ●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●●

●●●

●●

●●●

● ●

●●●●

● ●●

● ●●●

●●

● ●● ●

●●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●●

●●

●●

●● ●

●●

●●

●●●●●

●●

●●

● ●

●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

●●

●●● ●

●●

●●

●●

● ●

●●

● ●

●●●

●●

●● ●

●●●

●● ●●●●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●

●● ●

● ●

●●

●●

●●

● ●●

●●●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●●

●● ●●

●●

●●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

● ●●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●●●

●●

●●

●●

●●

●●●

●●

● ●●●●

●●

●●

●●

●●

●●●

●●●

● ●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●● ●

●●●

● ●●●●

●●

●●

●●

●●

●●

●●

●●●

●●●●●

● ●

●●

●●

●●

●● ●●

●●

●● ●

●●●

● ●

●● ●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●●

●●

●●

●●

●● ●

●●

●●

●●●●●

●●

●●● ●●

●●

●●

●●

●●●

●●

●● ●

●●

● ●

●●

● ● ●●●

● ●

●●

●●

●●

●●

● ●●

● ● ●●

●●

●●

●●

● ●

● ●

●●

● ●

● ●

●●

●●

● ●

●●

● ● ●●●

●●

●●●

●●

●●●

●●●

●●● ●

●●●

●●

●● ●

●●●● ●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●● ●●

●●

●●

●●

● ●

●●●●

●●●

●●●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●● ●

● ●

●●

●●

● ●

●●

●● ●

●●

●●●●●

●●●

●●

●●

●●

●●

●●●●●

●●

● ●

●●●

●●

●●

●●●●●

●●

● ●

●●

●●

●● ●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●● ●●

●●●

●●

●●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●●

●●

●●

●●

●●●

●●

● ●

● ●

●●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

● ●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●●

●●

● ●●●

●●

●●

●●●

● ●

● ●

●●

●●

●●

●●

●●

●●

● ●●

●●

● ●●

●●

●●

●●●

●●

●●

●●

●●●

●●●●

●●

●●

●●●

●●

●●

●●● ●●●● ●●

●●

●●

● ●

●●

●●

●● ●

●●

●●

●●

●● ● ●

●●●●

●●

●●

●●

●●●●

●●●

●●

● ●

●●●●

●●●●

●● ●●

●●

●●

● ●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●●

●●

●●● ●●

●●●●●●●

●●●

● ●●

●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●●

●●●●

● ●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●●

●●● ●●

●●

● ●

●● ●

●●

●●

●●

●●

●●●

●● ●●

●●●●

● ●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

● ●

●●

● ●

●●

● ●

● ●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●● ●

●●●

●●●●

●●●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

● ●●

●●

●●●●●

●●

●●●●

●●

●●

●●

● ●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●● ●●●

●●

●●

●● ●

●●●

● ●●●

● ●●

●● ●●●●

● ●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●● ●

●●

●●

●●

●●●

●●

●●

●● ●

●●

● ●●

●● ●

●●

● ●

●●●

● ●

●●●

● ●

●●

● ●

●●

●●●●

●●●●

●●●

●●

● ●●

●●

●● ●●●

●●

●●●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●● ●

●●●

●●

●●●

● ●●

●●

●●

●●

●●●

● ●

●●● ● ●● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●●

●●●

●●

●●

●●

●●●

●●

●●

●● ●

●●

●●

●●●

●●

●●

●●

●●●

●●

● ●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●●

●●●

●●

●●●●

●●

●●

●●●

●●

●●

●● ●

●●●●●

●●

●●

●●

● ●●

● ●

●●

●●

● ●● ●

●●

● ●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●

● ●

●●

● ●● ●

●● ●

●●

●●●●

● ●● ●

●●●

● ●

●●

●●

●●●

●●

●●●

●● ●

●●

●●

● ● ●●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●

● ●●

●●

● ●

●●

●●

●●

● ●

●●

●●●

●●●

●●

● ●

● ●

●●

●●

●●●

●●

●●

●●●●●●

●●

●●

● ●●●●

●●

● ●●

●●

● ●

● ●●●

●●

●●

●●

●●

●●●●

●●

●●

●●●●

●●●

●●

●●

●● ●

●●

●●

●●

●●

● ●●

●●

●●

●●●● ●

●●

●●●

● ●

●●

●●

●●

● ●●

●●

●●

●●

●●

● ●● ●

●●●

●●

● ●●

●●

●●●

●●●

●●

● ●

● ●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

● ●

●●●

●●●

●●

●●

● ●●

●●

●●

● ●

●●

●●●

●●

●●

●●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●●

●●

● ●

●●

● ●

● ●●

●●

● ●

●●

●●

●●

● ●●

● ●

●●

● ●

●●

●●

● ●

●●

●●

●●

● ●●

●●

● ●

●●●●

● ●

●●

●●

●●

●●

●● ●

●● ●●●

●●●

●●

●●

●●

●●

●●●

●●

● ●

●●

●●●

●●●

●●

● ●

●●

●●●●

●●

●●●

●● ●●●

●●

●●

● ●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●●

●●

●● ●

●●●●

●● ●●

●●●

●●

●●

●●●

●●●●● ●●

●●

● ●

●●●

●●

●●●

●●

●●

● ●●●●

●●

●●

●●

●●

●●

●●●●

●●

●●●● ●

●●●

●●●

●●

●●

●●

●● ●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●●

●●

●●●●

●●

●●

●● ●●●

●●●

●●●

●● ●

●● ●

●●●

● ●●●

● ●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●● ●

● ●●

●●

●●●

●●

●●

●●

● ●

●●

●●●●

● ●●

●●

●●

●●●

●● ●

●●●

● ●● ●●●

●● ●

●●

● ●●

●●●

●●

● ●●

●●

●●

● ●

●●

●●●

●●

●● ●●●● ●●●●

●●

● ●

●●

●●

● ●●● ●●

●●

●●

●●

●●

●●

●●

● ●

●●

●● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

● ●

●● ●

●● ●●

●●●

●●

●●

●●

●●● ●

●●●

●●●●

●●

● ●●

●●

●●

●●

●●

●●●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●●

●●

●●

●●

●● ●●

●●

●●

●●●

●●

●●●

●●

●●●

●●●●●

●●

●●● ●

●●

●●

●●●

●●

●● ●

● ●

● ●

●●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

● ●

●●

● ●●●●●

● ●

●●

● ●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●●●●

●●

●●

●●

● ●●

●●●

●●

●●

●●

●●●●

●●

●●● ●●

●●

●●

●● ●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●● ●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●● ●

●● ●

●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●●

●●

● ● ●●

●●

●●●●●

●●

●●

●●●

●●●

●●●●●

●●

●●

● ●

●●

●●

● ●

● ●●●

●● ●

●●●

●●

●●

●●

●●●●

● ●

●●

● ●●●

●●

●●

●●

●●

●●

●●● ●

●●●

● ●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

● ●

●●

● ●

●●

●●

● ●●

●●

● ●

●●

●●

● ●

●●

●● ●

●●

●●

● ●

● ●●

●●

●●●●●

●●

●●

●●

● ●

●●

●●

●●

● ●

● ●

●●●●

● ●

●●

●●

●●

●●

●●

●●

●●●●

● ●

● ●

●●

●●

●●

●● ●●

● ●●

● ●

●●

●● ●● ●●

●●

●●

● ●

●●

●●

●●

●●

●● ● ●

●●

●●

●●●

●●

●●●

●●

●●

● ●●●●●

●●●

●●●●

●●

●●

●●

●●

●●

●●

● ●

● ● ● ●

●●

●● ●

●● ● ●

●●

●●

●●

●●●

●●

●●●

● ●●

● ●●

● ●

●●

●●

●●● ●

●●

● ●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●●● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

●● ●

●●●●

●● ●

●●

●●

● ●

● ●

●●

●●

●●

●● ●

●●

●●

●●

● ●

●●

●●

●●

●● ●●●●

● ●●

●●

●●

●●

●●●● ●● ●

● ●

●●

●●

●●

● ● ●

●●

●●

●●

●●

●●

●●

●● ●●

●● ●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

● ●

● ●●

●● ● ●●

●●

●●

●● ●

●● ●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

● ●

●● ●

●●

●●

●● ●●● ● ●

●●●

● ●

●●● ●

●●

●●

●●

●●

●●

●●

● ●

●●● ●

● ●

●●●●

● ●

● ●●●

●●

●●●● ●

● ●

● ●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

● ●●

● ●●

●●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ● ●●●

● ●

●●

● ●

● ●●

●●

●●

●●●

●●

● ●

●●

●●

● ●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●● ●●

● ●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●

● ●

●●●

●●●●

●●

●●

●●

●●

●●

●●

●● ●●● ●

●●

●●

●●

● ●●

●●

●●

●●

●●●●●

●●

●●

●●

●● ●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●● ●

● ●

●●

●●

●●

●●● ●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●● ●

●●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●●

●●●●

●● ● ● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●●

●●●

●●

●●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●

● ●

● ●● ●● ●● ●

●●

●●

●●

●●

●●

●●● ●●●

● ●

●● ●

●●

●●

●●

●●

●●

●●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●●●

● ●

●● ●

● ● ●●

●●

●●

●●●

●●

●●

● ●

●●

●●●

●●

●●●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●●●●

●● ●

●●●●●

●●

●●●

● ● ●●●●

●●

●●●●●

●●●

●●

●●

●●

●●

●● ●

●●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●●

●●●

●●

●●

●●

●●●

● ●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

● ●

●● ●● ●

●●

●●●

● ●

●●●

● ●● ●

● ●

●● ●●

● ● ●

●●

● ●

● ●

●●

●●●

●● ●

●●

● ●

● ●

●●●●●

● ●●

●● ●

● ●● ●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●●●●●

●●

●●

●●

●● ●

●●

●●

●●

●● ●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●●●● ●

●● ●

●● ●

●●

● ●●●●

●●●

●●●● ●● ●●

●●

● ●

●● ●

●●●●

●●●●

●●●

●●●

●●

● ●●

●●

●●

●●

●●

●●

●● ●●

● ●

●● ●

●● ●

● ● ● ●

●●● ● ●

●●

●●●●

●●

● ●

● ●

●●

●●● ●

●●

● ●●

●●●

●●

●●

●●●

●●

● ●

● ●

●●

●●

●●●●

●●●

●●

●●

●●

●●

●● ●

●●

●●

● ●●

●●

●●

●●

● ●

●●

●●●

●●

●●●●

●●

● ●

●●

●●

●●

●●

●●

●●●

●●

●● ●

● ●

● ●

●●

●●

●●●

●●

●● ●● ●

● ●

●●

●●●

●●

●● ● ●

● ●●

●●

●●

● ●●

● ●

● ●

● ● ●●

●●●

●●●●●●●

●●

●●

●●

●●● ●

●●

● ●●●

●●

●●

●●

●●●●

● ●●●

●●

●●

●●

●●

● ●●

●●

●●

●●●●

●●

●●

●● ●

● ●●

●●●

● ●

●●

●●●

●●

●●

●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●●

●●●●●

●● ●●

●● ●

●●●

●●

●●

● ●

● ●

●●

●●

●●

●● ●●

●●●●

●●

●●

●●

●●

●●

●●●

●●

● ●●●●●●

●●

●●

●●●●

● ●●

● ●

●●

●●

●●●●

● ●

●●

●●

● ●

●● ●

●●

●●●

●●

●●

● ●

●●

●●

●●

● ●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●●●

● ●●

● ●

●●

●●

●●● ●

●●

●●

● ●

●● ●

●●●

●●

●● ●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●●

●●●

●●

● ●

●●

●●

●●●●

●●

●●●●

●●●

●●

●●● ● ●

● ●●

● ●

●●

●●

●●●

● ●●

●●

●●

●●

●●●

●●

● ●●

● ●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●● ●●●

●●

●●●

●●

●●●●●

●●●

●●

● ●

●●

●●

● ●

●●●

●●

● ●

●●

● ●●

●● ● ●

●●

●●

●●

●●

●●●

●●

●●

● ●

● ● ●

●●

●●

●●

●●●

● ●●

●●

●●

●●

● ●

●●●●

● ●

● ●●

●●

●●

●●

● ●●

● ●

●●

●●

●●

●●●

●●

●●● ●●

● ●

●●

● ●

●●

●●

●●●●

●●●

●●●●

●●●●

●● ●●

●●●● ●● ●

●●

●●

● ●

●●

● ●

●●

●●

●●

● ●●

●●

● ●

●●

● ●

●● ●

● ● ●●

●●

●●●

●●●

●●

●●

● ●●●

●●

●●

● ●●●●

●●●

●●

●●

●●

●●

●●

● ●

●●

● ●

●●●

●● ●● ●

●● ●

●●

●●

●● ●

●●●

●●●

●●

● ●● ●●● ●

● ●●●

●●●

●●

●●● ●●

● ●

●●

●●

●●

● ●

●● ●

●●

● ●●●●

●●

●●

●●●●●

● ●

● ●●

●● ●

●●

● ●●●

●●

●●●

●●●

●●

●●

●●

●●

● ●

● ●

●● ● ●●●

●●

●●

● ●

●●

●●

● ●●●●

●●

●●

●●

●●

●● ●

●● ●●

●●

● ● ●

● ●

●●

●●

●●●

●●●

●●

● ●

●●●●

● ●

●●

●●

● ●● ● ●

●●

●●

●●

●●●

● ●

● ●

●●

● ●

●●

●●

●● ●●

● ● ●●

● ●●

●●●

●●

●●

●●

●●

● ●

●●●

●●●

●●

● ●

●●●

● ●

●●

●●

● ●

●●●

● ●●

● ●

●● ●●

●●●

●●

●●

●●

●●

●●

●●

●● ●●●

●●

●●

●●●

● ●●

●●

● ●

● ●●● ●●

● ●●

●●

● ●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●●

●●

●●

● ●●

●●●

●● ●

●●

●●

● ●

● ●

●●

●●

●●

●●●●

●●●

●●

●●●

●●

●●●

●●●

●●

●●

●●

●●

●●

●●

●● ●●

●●●●

●●

● ●●

●●●

● ● ●●

●●

● ●

●●

●●

●● ●

●●

● ●

● ●

● ●

●●

●●●

●● ●

●● ●

●●

●●

●●

●●

●●

● ●●● ●

●●

●●

●●

● ●●

●●

●●

●●

●●

●● ● ●

●●

●●

●● ●●

●●●●●●

●●

●●●

● ●●●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●● ●

● ●

●●

● ●●

●●●●

●●

●●

●●●

●●

●●

●●●

●●

●●

●●

● ●●●●

●●●

●●

●●

●●●

●●●

●●●●●

●●

●●

● ●

●●

●●

●●

●●●

●●●●

●●●

● ●

●●●

●●

● ●●●

●●●

●●●

●● ●

●●

●●●

●●

● ●

● ●

● ●

●●

● ●

●●

●● ●

●●

●●●

●● ●

●●

●●

● ●

●●

●●

●●●

●● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●● ●

●●

●●

● ●●●●

●●

●●

●●

●●

●●●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●● ●●

●● ●

●●

●●●

●●

●●

● ●●●

●●

●●●

●● ●

●●● ●

●●

●●

● ●

● ●●●●

●●●

●● ●●● ●

●●●●

●●●●

●● ●

●● ●

● ●

●●

●● ●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

●●

●●●

● ●

● ● ●●

●●

●●

●●●

●●

●●

●● ●

●●

● ●

●●

●●

●●

●●

●●

●● ●

●●●● ●

● ●●

●●

●●

● ●●

●● ●●

●●

●●

●●●●●●●

● ●● ● ●●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

● ●●

●● ●

● ●

●●●

●●

●●

● ●●

● ●

●●● ●

● ●●●

●●

●● ●

●●

● ●● ●●

●●●

●●

●●●●●●●●●

●●

●●

●●

● ●●●●

●●

●●

●●

●●

●● ●●●

●●●

● ●●●

●● ●

●●

● ●

●●

●●

● ●

●●

●●●●

●●

● ●

● ●● ● ●●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●●

●● ●

● ●

●●

●●

●●●

●●

●●

●●●

●●

●●

●●●

●● ●

●●● ●●●

● ●

●●

●●

●●

● ●

●●

●●

● ●

●●

● ●●

● ●

●●

●● ●

● ●●●●

●●

● ●●●

●●

●●

●●

●●

● ●

● ●

●●

●●

● ●

●●

●●

●●●●●

● ●

●●

●●

● ●

●●

●● ●●

●●

●●

●●

●●●

●●●

● ●

●●

●●

●●

●●

●●

● ● ● ● ●●●

●●

●●●

● ●

●●

●●

●●

●●● ●●

●●

●●

● ●●●

●●

●●●●

● ●

●● ●

●●

●●

●●●

●●

●●●

● ●●● ●●●●

●●

●●

●●

●●●●

●●

● ●

● ●●

● ●

●●●

●●●

●●

●●

● ●●

● ●

●●

● ●

●●●

● ●●

●●

●● ●●

●●●

●●

●●

●●

●●●

●●

● ●●

● ●

●●●

●● ●

●●

●●●

● ●

● ●● ●●

●●

● ●

● ●

● ●

●●

●●

●●

●●●

●●●●

●●

●●●

●●

●●●●●

● ●●

●●● ●

●●●

● ●●

●●

●●●●●

●●

●● ●

●● ●

●●

●●

●●

● ●

● ●

●●

●●

● ●●

●●

● ●●

●●

● ●

●●●● ●

●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

● ●● ● ●●●

●●● ● ●

●●

● ●

●● ●

●●●●

●●

●●

●●

●●

●● ●

●●

●●●

●●

●●

●●

●●

●●

● ●●●

●● ●●●

●● ●●●●●●

●●

●●

● ● ●

●●●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●

● ●

●●

●●●

●● ●●

●●

● ● ●

●●

●●

● ●

●●

●●

●●

●●

● ●●

●●

●●

● ●

●●

●●●

●● ●

●●

●● ● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●

●●

●●

● ●

●●

●● ●

●●

●● ●

●●

●●

●●

●●

●●

●●●

●●

●●● ●

●●●●

● ●●●●

●●

● ●

●●●●

● ●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●● ●

●● ●

●●

● ●●

●● ●

●●

●●

●●●

● ●

●●

●●

● ●●

● ●

●●

● ● ●

● ●

●●●

●●●

●● ●● ●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●●

● ●

● ●●●

● ●

●●● ●

●●

●●●

●● ●

● ●●

●●

●●

●●

●●●

● ●

● ●

●● ●●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●● ●

●●

●●

●●

● ●

●●

●●

●●●

●●

● ●● ●●

● ●

●● ● ●●●

● ●●

●●

●●

● ●

●●

●●●

●●

●●●●●

●●●

● ● ● ●

●●

●●

●● ●

●●

● ●

● ●

●●

●●●

● ●●

●●●

●●

●●

● ●●●

●●

●●●

● ●

●●●

●●

●●

● ●●

●●

●●●

●●●

●●●

●●●

●● ●

●●●

●●

● ●

●●

●●

●●●● ●●

● ● ●

●●

●● ●

●●

●●

● ●●

●●●●

● ●

●●●

● ●

●●

●●●

●●

●●●

●●

●●●●●

● ●

●● ●●●

●●

●●

●●

●●

● ●

● ●●

●●

● ●

●●

● ● ●●

● ●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●●

●●

● ●

●●

●●●

●●

●●●

●●●●

●●

●● ●

●●

● ●

●●

● ●● ●

●● ●

●●

●●

●●●

●●

●●

● ●● ● ●

●●

●● ● ●● ●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●● ●

● ● ●

●●●

●●●●●● ●

● ●

●●

● ●●

●●

●●●●

●●

●●

● ●

●●

● ●●

●●

●●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●●●●●

●●●

●●

●●●

●●●

●●

● ●

●●

●●

●●

● ●

●●●

● ●

●●

●●●●●

● ●●

●●

● ●●●●

●●

●●

●●

●●● ●

●●

●●

●● ● ● ●●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

●●

● ●● ●●

●●

●●

●●●

●●●

●●

● ●●

●●

●●

● ●●

●● ●

● ●

●●●●●

●●

●●

● ●

●●

● ●●

●●●●

●● ●

●●

● ●●● ●● ●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●● ●●

● ●●

●●●

●●

●●

●●● ● ● ●

●●

●●●

● ●

●●

●●

●●●

●●●

●●

●●●

●●

●●

● ●

●●

●●

●●●

●●

●●●

● ●

● ●

●●●

●●

●●

●●

●●

●●● ●

●●●

● ●

● ●

● ●

●●

●●●

●●

●●

●●

●●●

●●●●

● ●●●

●●

●●

●●

●●

●●●

● ●●●

● ●●

●●

●●

●●

● ●

● ●

● ●

●●

●●●

● ●

●●

●●

● ● ●● ●

●●

●●●

● ●●●

●●

●●

●●

●● ● ●●

● ●

● ●

● ● ●

●●

●●

● ●

●●

●●●

● ●●

●● ●

● ●●

●●

●●

● ●

●●

● ●●

●● ●●

●●●

● ●● ●●

● ● ●●● ● ●

●●

●●

●●

●● ●

●●

●● ● ●

● ●●

●●

●●

●●

●●

●●

●● ●● ●

●●●

●● ●●

●●

●●

●● ●

●●

●●●

●●

● ●●

●●

●●●

● ● ●●

●● ●● ●

●●●●● ●

●●

●●

●●

●●●

●●

●●

● ●

●●

● ● ●

●●

●●

●● ●●●

●●

●●

●●

● ● ●

● ●●

●●

●●●

●●

●●● ●

●●

●●

●●

●●

●●●● ●

● ● ●● ●

●● ●●

●●

● ●

●●● ●

● ●

● ●

●●●

●●

●●

●●●●●●●●●

●●

●●

● ●

●●● ●

● ●

●●

● ●

● ●●

●●

●●

● ●

● ●

●●

●●

●● ● ●

●●

●●

●●●

● ●

●●

●●

●●●● ● ●●

●●●

●●

●●

●●

●●

●●

●●

●● ●

●●● ●

●●

● ● ●

● ●

● ●● ●●

●●

● ●

●●

●●

● ●

●●●●●● ●

●●

●●

●●●

●●

●●●

● ● ● ●

●●

● ●● ●

●● ●●

●●●

●● ●

●●

●●

●● ● ●

● ●●

●●●

●●●

●●

● ●

●●

● ●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●●

●●

●●

● ●●●

●●

● ●●

●●

●●

●●

●●●

● ●

●●

●●●

●●

●●●

●●

●●

●●●

●●●

●●●

●●

●●

● ●●

●●

● ●●●

● ●

●●

● ●

●●

●●

●●●

●●

●●

●●

●●●

● ●●●●

●●

●●●●

●●

● ●●●

●●

● ● ●

● ●●

● ● ●

●●●●

●●●

●●●

●●●●

● ●

●●

● ●●

●●

●●

●● ●●●

●●●

●●●

●●●

●●

●●● ●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●●●

● ●

● ●

●●

●●●

● ●

● ●

●●

● ●● ●●

●●

●●●

●●

●●●

●●

●●● ●● ● ●

●●● ● ●

●●

●●

● ●●

●● ●

●●

●●●

●● ●● ●

●●

●●

●●

●●●

●●●

●● ● ●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●●● ●

●●

●●

●●

●●

●●●●

●●

●●

●●●

● ●

● ● ●

●●

● ●

●●●●● ●

●●

●●

● ●

●●●● ●

●● ●●●●

●●

● ●

●●

●●●

●●

●●

● ●

●●

● ●

●●

●●

● ●

● ●● ●

●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●● ●●

●●●

●●

●●

● ● ●

●●

●●

● ● ●●● ●

● ●●●

●●

●●

●●●●

●●

● ●

●●

●●

● ●●

● ●

●●

●●

●●

●●●

● ●

●●

● ●

●●

● ●

●●●●

● ●

●●

●●

●●●●●

●●

● ●●

● ●●

●●

●●

●● ●

●●

● ●●● ●

● ●

● ●

●●●

●●●

●●

●●

●●

●●

●●

●●

● ●●

●●●●

●●●

●●

● ●

●●

●●

●●

●● ●

●● ●

●●● ●

●●

●●●

●●

●●●●●●

●●

●●●●

●●●

●●

●●

●●●●●●

●●

●●●● ●●

●●●

●●

● ●

● ●●

●●

●●

● ●

●●●

●●

●●

●●

●●●

●●

● ●

●●

● ●

●●●

●●

●●

●●● ●

●●●

● ●

●●

●●

●●

●●

●●● ●

●●

●●

●●

●●● ●

●●

●●●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

● ● ●

●●

●●

●●

●●

●●

● ●

● ●

●●●

● ●

●●

●●●

● ●

●●

●●●

●●●●

●●

●●●●● ●

● ●

●●

●●

● ●

● ●●

●●●

●●●

● ●●●

● ●

●●●

●●●

●●●

● ●●●

● ●

●●● ●

●● ●●

●● ●

●●●

● ●

●●

●●●

●●

●●

● ●●

●●

●● ●●

●●

●●●

●●

●●●

●●

●●

● ● ●●

●●

●●

●●

●●

●●

● ●

● ● ●

●●● ● ●

●●●

●●

● ●

●●●

●●

●●

●● ●● ●

●●

● ●

●● ●●●●●● ●

●●

● ●●●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

● ●

●●

●● ●

●●

● ●

●● ●

●●

●●

●●

●●

●●● ●●● ●

●●

●●●

●●●

●●

●● ● ●●

●●●

●●●●●●

●● ●● ●

●●

●● ●●

●●●

●●

●●

● ●●● ●

●●

●●

●●

●●

●●

●●

●● ● ●

● ●●

●●

●●●

●●●

● ●

●● ●

● ●●●●

● ●

●●

●●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●●

● ●

●●

●●

●●

● ●

● ●●

●●

● ●●

●●

●●●

●●

●●●

●●● ●●

●●

●●●

●●

● ●

●●

● ●● ●● ●●

●●

●●●

●●

●● ●●

●●

●●●

●●

●● ● ●●

●●

●●

● ●●

●● ●

●●

●●

●●

● ●

●●

●●

●●●

● ●

●●

●●

●●●

●●

●●

●●●●●

● ●

●●

●● ●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●● ●

● ●

● ●●

●●●

●●●●

●●

●●

● ●

●●

●●

●●

● ● ●

●●

●●●●

●●

● ●

●●

●●●

●●

● ●●

●●●

● ●

●●

● ●

●● ●●●●

● ● ● ●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●●

●●●

●●●

●●

● ●

●●

● ●

●●● ●●

● ●

●●●

●●

●●

●●

●● ●

●●

●●●

●●

●● ●●

●●

●● ●●● ●

●●●

●●

●●●

●●● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

● ● ●●

●●●

●●

●●●

●●

●●

●●

●●

● ●

●● ●●●●

●●

●● ●

●●

●●

●●●

●●

●●

●●

● ●●

●●

●● ●● ●●

●●●●

●●●● ●●● ●

● ●

●●

●●● ●

●●

● ●

●●

●●

●●

● ●●

● ●

● ● ● ●

●● ●

●●●

●●

●●●

● ●

●● ●● ●●

●●

●●

●●

●●

●● ●●●

●●

● ●

●●

●●

●●

● ●●

●● ●

● ● ●●

●●

●●

●● ●

●●●

●●

●●

● ●

●●

● ●●

●●●

● ●●● ●●

● ●●

●●

●●

● ●

● ●

● ● ●

●●

●●●

●●

●●●●●

●● ●

● ●●

●●●

●●

●●

●●● ●

●●

●●

●●●

● ●

● ●

●● ●

●●

●●●

●●●

●● ●

●●

● ●●

●●●

●●

●●

● ●● ●●●●

●●●●

●● ●

●● ●●●

●●●

● ● ●●●

● ●

●●

● ●

●● ●

● ●

●●

●●●

●●

●●

● ●

● ●

●●

● ●

●●●

●●●

● ●

●●●

●●

●●

● ●● ●

● ●

●●

● ●●

●●

●●●●

●●● ●●●●

●●

●●●

●●

● ●●

●●●

●●

●● ● ●●● ●●●●

●●

●●●

●●

● ●●●

●●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●●

●●●

●●

●●

● ●

● ●

●● ●

● ●

●●

●●

●●●●

●●

●●●

●●

●●●

●●

●●● ●

●●

● ●

●●

●●●●●

●●●● ●

●●

●●●

●●●●● ●●

●●

●●

●●

●●

● ●

●●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

● ● ● ●

●●

● ●

●●

●●

●●

●●

●●

●● ●

● ●

●●

●●

●●

● ●

●●●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●● ●●

●●

● ●

●●

●●

●●

●●●●

●●

●●

● ●

● ●

●● ●

● ●

●●

●●

●●

● ●●

●●

● ●

●●

●●●

●●

●● ●

●●

●●

●●

●● ●●

●●

●●

●●●

●●

●●

●●

●● ●● ●

● ●

● ●

●●●●

●●

●●

●● ●

●●

● ● ●●

●●

●●

●●●

●●

●●

● ● ●●

●●

●●

●●

●●

● ●●

●●

● ●●

●●●

●●●● ●

●●●

●●

● ●

●●

● ●●

● ●● ●

● ●

●●

●●●

● ●

●●●

● ●

●●

●●

● ●

● ●

● ●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●● ●

● ●●

●●

●●

●● ● ●●

●●●

●●

●● ●● ●●

●●

●●● ●

●●●

●●

●●●

●●

●●●●●

●●

● ●

●●

●●

●●

●●

● ●● ●

●●● ●●

●● ●

●● ●

●●

●●

●● ●

●●

●●

●●

●●

● ●

●● ●● ●●

● ●●●

●●

●●

●●

●●

●●

● ●●●

●●●

● ●

●●

●●

●●

● ●

●●

●●●

●●

●●●●● ●

●●● ●

●●

●●●

●●

●●

● ●●

● ●

●●

●●

● ●

●●●

● ●

● ●●

●●

●●

● ●● ●

●●

●●

● ●

● ● ●

● ●

●● ●

●●●

●●

●● ●●

●●

●●

● ●●

●●●

●● ● ●

●●

● ●●●

●●

●● ● ●

●●

●●

●●●●

●●● ●

●● ●●● ●● ● ●

●●

●●

● ●

●● ●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●●

●●

●●

● ●

●● ● ●●● ●●

●●

● ●●●●●

●●

● ●

●●

● ●

●● ●

●●●

● ●

●●

●●

●●

● ●●●

●●

●●

● ●●●● ●

●●

●●

●●● ●

●●●●

●●

●●

● ●●

● ●

●●●

●●

● ●

●● ●

●●●

●●

●●

● ●●●●●

●● ●●

● ●

●●●

●●●

● ● ●

●●

●●● ●●●

●●●

●● ●

●●●

●●●●

●●

●●

●●

● ●●

●●

●●

● ●

● ●

●●●● ●

● ●

● ● ●●

●●

● ●●

●●

● ●

● ●●

●●●●

●●

●● ●● ●●●

●●●●● ● ●●

●●● ●●

●●

●●●

●●

●● ●

●●●

●●

●●●

●●

●●●●

● ●

● ●

●●

●●

●●●

●●

●● ●

●●

●● ●● ●●

●●

●●

●● ●

●● ●

●●

●●●●

● ●

●●

●● ●●●●● ●

●●

●●

● ●●● ●

● ●●●

●●●

●●

●●●

●●●

●●

●●●

●●

●● ●

●●

●● ●●●

●●●

●●

● ●

● ● ●

● ● ●

●● ●

●●

●●●

●●●

●●●

● ●

●●

●●

●●

●●

●●

●● ●

●● ●

●●

● ●●●

● ● ●●●

●●●●

●●● ●● ●

●●

●●●

●●

●●

●● ●

●●● ●●

●●●

●●

● ●●

●●

●●

●●

●●

●●●●

●●

●●

●●

●●●●

● ● ●

●● ●

●●

●● ●

●●

●● ●

●●●

●●

●●

●●

●●

●●●

●●●

●●●

●●

● ●●

●●

● ●

●●

●●

●●

●● ●

●●

●●

● ●●

● ●● ●●

●●

●●

●● ●

●●● ●

●●

●●

●● ●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●● ●

●●

●●

● ●● ●●●●

●●

●●

●●

● ● ●●

● ●●●

● ●

● ● ●●

●●●

●●

●●●

●●

●●

●●

●● ●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●●

● ● ●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●●

●●●

●●

●●● ●

● ●●

●● ● ●●

●●

● ● ●

● ●

●●● ●●

● ●

●● ●

●●

●●

● ●●

●●

● ●●

●●●●

●● ●

●●●

●●

●●

●●

● ●●

●●●

●● ●

●●

●●● ●

●●

●●

● ●

●●

● ●●

●●● ●

● ●●

●●

●● ●●

●●●

● ●

●●

●●

●●

● ●

●●●

●●

●●

●● ●●

●●●

●●

●● ●

●●●● ●

●●

●● ●● ●●

●●

●● ●

●●●

●●●

●●

●●

●●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●●

●●

●●

●●

●●●● ●

● ●

●●

●●

● ●

●●

●●

●●

●● ●●

●● ●

●●

●●●

● ●●●

●●●●

●●

●●●

●●●

● ●●

●●●●

● ●●

●●

●● ●● ●● ●

●●●

●●

●●●

●● ●●

●●

●●

●●

●●

●●●●

● ●

● ●

●●

●● ●

●●

●●

●●

●●

●●

●●●● ●

●●●●●

● ●

● ●●

●●

●●

●● ●

●●

● ●

●● ●●

●● ●● ●

●●

●●

● ●

●● ●

●●●●

●● ●

●●

●●

●●● ●● ●

●● ●● ●● ●

●●●

●●●●

● ●

● ●● ●

●●●● ●● ●

●●

●● ●●

● ●● ●●

●●

●●

● ● ●●●

●●●●

●●

●●●●

●● ●

●●

●●●

●●●

● ● ●●

● ●●● ●●●●

● ●● ●

●●

●● ●●●

●●

●●

●●

●●

●●

● ●● ●

●●●

●●●

●●

● ●●

●●

●●

●●●

●●

●●

● ●●●

● ●

● ●●

●●

● ●

●●

●● ●

●● ●

●●

●●

●●●

●●●

● ●

●●

● ●

●●

●●● ● ● ●● ●

●●●●●

●●●● ●●●

●●●●

●● ●●●

●●

●● ●● ●

●●●● ● ●

●● ●●●

●●

● ●●

● ● ●●●

●●●

●●●

●●

●●●

●●

●●

q = 1%q = 2.5%q = 5%q = 10%q = 15%q = 25%q = 50%

r lq =

∑s∈S

[∣∣xs,100l −xl

∣∣xl

> q

]|S|

, l ∈ L, q ≥ 0.

[email protected] Random Seeds Output Variability in MATSim 25 / 33

Page 27: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Between-Seed Variation vs Within-Seed Variation I

Practically all links have Rl ≥ 4.Between seed-variation dominates within seed-variation.Despite only using 100 iterations.

2 4 6 8 10

0.0

0.2

0.4

0.6

0.8

1.0

R~

cdf R~

l

1 2 3 4 5 6 7 8 9 10

0.0

0.2

0.4

0.6

0.8

1.0

Rl =√

Bl +WlWl

, l ∈ L.

[email protected] Random Seeds Output Variability in MATSim 26 / 33

Page 28: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Case Study & Results Case StudyResults

Between-Seed Variation vs Within-Seed Variation II

Problem intensifies for busiest links.

05

1015

20

xl

R~l

●●

●●● ●● ●

●●●

●●●●

●● ●

● ●

●●

●● ●

●●

● ●●●

●●●

●●●●●

●●

●●

● ●●●●

●●●●●●●●●

●● ●●

●●

●●●

●●●● ●●●●

●● ●●●● ●●●●

●●

●●●

●●●

●●

●●

● ● ●●

● ●●●

● ●●●●

●●

●●

●●●

●● ●●

● ●

●●

●●●● ●●● ●

● ●

● ●●

●●

● ●●●

●● ●

● ●●

● ●●● ●●

●●●●

●●●

●●●●●●● ●

●●●

●● ●●● ●●

●●

●●● ●●

●●

●●

●●

●●●

●●●

●●●

●● ●●

●●●

●●

●●

● ●●

●●●

● ●●●● ●●● ●●

●● ● ●

● ●●● ●

●●●● ●● ●

●●

●●

●● ●●

●●●

●●

●●●

● ●●

●●

●● ●

●●●●

●●●

●●

●●

●● ●●

●●●

●●●●●● ●

● ●● ●●

● ●●●●

●●

●●

●●

●●●●●● ● ●

●● ●

●●

●●

● ●●●●●

●● ●● ●●

●●●●●

●●

●●

●● ●●

●●

● ●

● ●●

●●

●●

●●●

●●

●●

●●● ●

●●●●●●●●●●●●●

● ● ●

● ●●●●

●●●● ●●

●●●

●●●

●●● ●

●●

●●●●

●●●●

●●

●●

● ●

●●

●● ●

●●

●●

●●●

● ●● ●

●●

●●

● ●

●●

● ●

●●

●●

●●●

● ● ●● ●

●●●● ● ●

●● ●●●

●●●●

●●

●●●

●●

●●●●●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●●●●

●● ●

● ●●●

●●

● ●●

●●

●●

● ●●● ●● ●● ●

● ●

●●●●● ●

●●

●●

●●

●●●

● ●

●● ● ●●

●● ●

●●●●●

●●●●

●● ●●●

●●

● ●

●●

● ●●

● ●●

●●

● ●●●●

●●

●●

●●●● ● ●

● ●●●

●● ●●●

●●●

●●●

●●

●●

●●●

●● ●

●●● ●●●●

●●

●●●●

●●

●●

●●

●●●●●

●●●●●●

●● ●●●●● ● ●

●●

●●●●●

●●●● ●

●● ●●●●●●

●●●

●●

●● ●

●●●●

●●● ● ●● ●

● ●● ●

●●

● ●

●●

●●

● ●

●● ●

●● ● ● ●●● ●

●● ●●●● ●

● ● ●

●●

●● ● ●● ● ●●

●●●

●●

● ●●

● ●●●

●●

●●

● ●

●●●●

●●

●●●●● ●●●

●●

●●

●●

●●

● ● ●●

● ●

●●

● ●●

●●

●●

●●

●●

●●

●●●●

●●

●●

● ●●

●● ● ● ● ●● ●

●●●●

●● ●●

●●●

●●

●●

● ●●

● ●●

●●

●●●●

● ●●

●●

●●●●

●● ●●

●●●

●●●

●●

●●

● ●

●●

●●

●●

●●● ●●●●

●●●●●●

●●●●●●●●●● ● ●●●

●●●

●●●●

●●

●●

●● ●

● ●

●● ●●

●●●● ●●

●● ●●●● ●

●●

●●

●●●

● ●

●●

● ●

● ●●

●●

●●

●●

●●

●●●

●● ●●● ●● ●●

●●

●●●

●●● ●

●●●●

●●● ● ●

●●

● ●●

●●●

●●

● ●●●

●●

●●

●●

●●

●●

●●●●

●●

●●● ●●● ●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●● ●● ●● ●●

●●● ●

●●●

● ● ●●

●●

●●●

●●●●

●●●●●

●●

●●●

●●●●

●●●

●●● ●

●●● ●

● ●●● ●● ●●

●●●

●●●●● ●

●●

●●

●●●

●●

● ●●

●●

●●

●●●

●●

● ●

● ●●

● ●

●●

●●

●●

●●●

●● ●●● ● ●●●●

●●

●●

●●

●●● ●● ●

●●

●●

●● ●

●●

●●

●●

●●

● ●●●

●●

●● ●

●●

●●

● ●●

●● ●●

● ●

●●●

●●●●●●●●●●

● ●●

●●

● ●

●●

●●●

●●

●●●

● ●●● ●●

●●

● ●

●●

●●

●● ●

●●

● ●●●

● ●●

●● ●●●

●●●●

●●

●●

●●

●●

●●

● ●●

●●

●●

● ●●

● ●●

● ● ●

● ●

● ● ● ●●●

● ●●●

● ●

● ● ●● ●●●●●

● ● ● ●●● ●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●● ●

●●●●

●●

●●

●●

●●● ●

●●

●●●●●

● ●● ●

●●●

●●

● ●●●

● ●●

●●●

●●

●●

● ●● ●●

●●●●●●

●●●

●●

● ●●

●●

●● ●

●●

● ●●●

●●

●● ●●●

●●●

●● ●●

● ●●●

●●●

●●

●●●● ●

●●

●●

●●● ●●●●

●●●● ●●●

●●●

● ●●●

● ● ● ●●

●●●

●●

●●●

● ●● ●

●●

●●

●●

●●

●● ●●

●●●●● ●● ●

●●

●●●●●

●● ●●

● ●

● ●●● ● ●●

●● ●●●

●●●

●●

●●●

● ●

●●

●● ●

●●●

●●

●●●

●●

●●●

●● ●●

●●

● ●

●●

●● ●

●●

●●●

● ●

● ●● ● ●●●●

●●●●●

● ●● ●●

●●●

●●●

● ●●●

●●

●●

●● ● ●● ●

●●

●●

●●●

●●

●● ●●

●●

●● ●●

●●●

● ●●

●●

● ●

●● ●●

●● ●

●●●●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●● ●●

●●

●●

●●

●●

●●

●●●●

●●●

● ●

●● ●●●●

● ●●● ●

●● ●●

●●

●● ●

●●

●●

●●●

●●● ●

●●

●●

●●●

● ●

●● ●●

●●

●●

●●

●● ●● ●● ●● ●●

●●●

●●

●●●

●●●●●●

● ●● ●●

● ●

● ●

●●● ● ●●

●●

●●● ●

●●

● ●● ●

●●

●●

● ●●

●●●●●

● ● ●●

●●

●● ●

●●●●

●●●●

●●●

●●●●●

●●●

● ●●

●●

● ●●●

● ●

● ●

●●

●● ●

● ●

●●

●●● ● ●

●●

●●

●●● ●● ●

●●●

●●

●●

●●

● ●●

●●

●●

●●

●●● ●●

●●

●●

●●

●●● ●●● ●●●●●●

● ●●●

●●● ●●

● ●

● ●● ●

●●●●● ●●

● ●

●●

●●

●●●

● ● ●

● ●

●● ●

● ●

● ●

●●

●●

●●

● ●●

●●●

●●

●● ●

● ●●

●●

● ●●

●●●

● ● ●●●●●

●●

● ●●

●●●

●●

● ●●

●●

●●

●● ● ●

● ●●

●●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

●●

●●

●●●

● ●●●

●●●

●●● ●● ●●

●●●●

● ●●●●●● ●●

●●

●●●

●●

●●

●●

● ●

●●●●

●●●●

●●

●●● ●

●● ●●

● ●● ●●●

●●●

●●

● ●●

●●

●●

● ●●●

●●

●●●

●●

●●

●●●●●

●● ●

●● ●

●●

●●

●● ●

●●●

●●●● ●

●●●

●● ●● ●●●

●● ●●●●●

●●

● ●●

●●●●● ●

● ●●●

●●

● ●●●● ●

●●●

●●

●●

●● ●

●●● ●

● ●

●●●

●●

● ●

●● ●● ●●

●● ●●● ●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●●

●● ●

●●

●●

● ●●

●● ●

● ●

●●●●●●●

●●

●●

●● ●●

●●●● ●

● ●●●

●●

●●●●●

●●●●●●

● ●●●●

●●● ●●

● ●●●

●●

●●

●●

●●

●●●●

●● ●

●●

● ●●●

●●●●●●

●●

●●

●●

●● ●

●●●

●●●

● ●● ●

● ●●●

● ●●●●●●

●● ●● ● ●

●●

●●

●● ●

●● ● ●

● ●● ●

●●

●●

● ●

●●

●●●

● ●●●●

● ●●

●●● ●

●●

● ●● ● ●●

●●●●

●●●●●●

●●●

●●

●● ●●●

● ●

● ● ●●

●●

●●● ●

●●●● ●●

●●

●●

●●

● ●●●

●●●●●

●●

●●

●●

●●

● ●●●● ●●

●●

●●

●●

●●● ●

● ●

●● ●

●●●

● ●●

●●

●●●●

●●

●● ●●

●●

●●

●●● ●

●●●

●●

●●●●●

●● ●

●●●

●● ● ● ●

● ●

●●●●●●●●

●●● ●●●

●●●●

●●●

●●●●● ●

● ●●

●● ●●

●● ●

●●●

●●

●● ● ●

● ● ●

●●

●●

● ●●

● ●● ●

●●

●●

●●

●●

●●

●●●

●●

●● ●●●●● ●

●●

●●

● ●●

●●●●

●●

●● ●● ●

●● ●●

●●

●● ●●●

●● ●

●●●●●

●●

● ●●●

● ●●

●●●

●●

●●

●●

●●●●

●●

●●●●●

●●●

●●

●●

●●

●●

● ●●●

● ●● ●

●●

●●

● ●●

● ●●

●●

● ●● ●

●●● ●

●●

●● ●

●●

●●●

●● ●●●

●●

● ●● ●● ●

●●

●●

●●●

● ●●

●●

●●●

●●●

●●

●●

●● ●●

●●●

●●

●●

● ●

●●

●●●●

●●

●●

●●

● ●●● ●●

●●●

●●

●●● ●● ●●

●●

●● ●●

●●

●● ● ●● ●

●●

●●●●●●

●● ●●

● ●

●●

●●

● ●●●● ●

●● ●●●●

●●

●●●

● ● ●● ●

● ● ●● ●●●●

●●●

● ●

●●

●●

●●

●●

● ●

● ● ●● ●●

●●●●

●●●●●

●●●●●●

●● ●●

●●

●●

●● ●

●●

● ●

●●●

●●●

●●

●● ●

●●●

●●●

●●

●●●● ●●

●●

●● ●

●● ●

●●

●●

● ●●

●●

●●

●● ●●

●●

● ●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●● ●●

●● ●● ●

●●

●●●

●●

●●

●●

●●

●●●

●●

● ●●● ●

●●

●●●

●●● ●●

●●

● ●● ●● ●

● ●● ●

●●●

●●●

●●

● ●●●●●

●●●●●

●●●●

●●

●●

● ●●

●●

●●

● ●

●●●

●●

●● ●●●●

●●

●●●●●● ●

●●

● ● ●●

●●

●●

●●

● ●●

● ●

●●

●●●

●● ●● ●

●●

●●

●●

●●

●●●

●● ●●

● ● ● ●● ● ●

●●

●●●

● ●

● ●

●● ●

● ●

● ● ●

●●

●●●

●●

●●●●● ●●●●

●●●

●●● ●●●● ●● ●●

●●●●●

●●

●●

● ●●

● ●● ●

● ●

●●●

●●

●●●●●●● ●

● ●

●● ●●●● ●

●●

● ●●●●●●●

●●●

●●

●●● ●

● ●

●●●●

●● ●

●●

●●●●

●●

●●●●●●

● ●●●

●●

●●

● ●

●●●

●● ●

●●●

● ●●●

●●●

● ●●●●●

●●

● ●●

●● ●

●●

●●●●●

●●

● ● ●●●

●●

●●●

●●

●● ●●● ●●●●●● ●●●

●●●●

●● ●●●

● ●● ●

● ●

● ●●●

●●

● ●● ●●● ●●●

●● ●

●●

●●●

●● ● ●●●

● ●●

●●

●● ●●

●●●

●●● ●

●●●

●●

●●

●●●

●●

●●●

●●●●●

●●●●●

●●●

●●●●●●

●●●

●●

●●

●●

●●●

● ●

● ●● ●

●●●

●● ● ●

●● ●●●●

● ●●●●

●● ●

●●

●●●

● ●●

● ●●●

●●

●●

●●

●● ●●●● ●

●●

●●

●●

● ●●

●●●●

●● ●●●●●

●● ● ●

●●●

●●

●●●

●●

●●

●●●

●●

●●● ●

●●●

●●●

●●●

●●

●●

●●

●●●

●●●

●●●●

●●●

● ●●●

●●

●●

●●●

●●

●● ●

●● ●

●● ●

●●● ●● ●●●

●●

●● ●

● ● ●

● ●●●●

●●●●

● ●●●

●● ●

●●

●●

● ● ●●● ●●●

●● ●●●●

●●●● ● ● ● ●●

●●●

●●

●●●

●● ●●● ●

●● ●

●● ●●

●●●

● ●

● ●●

● ●

●●

● ● ●●●●

●●

● ● ●●●●

●●

●●

●●

●●

●●

●● ●

●●

● ●● ●●

●● ●

● ●● ●

●●

●●●

●● ●● ● ●

● ●

●●● ●● ●●

●●

●●

●●

●●

●●●

● ●

● ●● ●●

●●●●● ●●●

●● ●●●

●●●●●● ●

● ●●

●● ●●

●●●

●● ● ● ●●

●● ●

● ●

● ● ●●

●●● ●●● ●●●●

●●

●●

● ●

●●

●●● ●

● ●

● ●

●●

●●●

●●●

●●

● ●

●●●●●●●●

●●●●

● ●

●●

●●● ●

● ●●●●● ●

● ●●

●●●

●●

●●

●●

●●●

● ●● ●●

● ●

●● ●●

● ●

● ● ●

●● ●

●●

●●●●

●●●

●● ●●

●●

●●

●●

● ●●

● ●

●●

●● ●

●●

●●●●●

●●

●●

● ●●

●●●

● ●●●

● ●

●●

●●

●●●

●●

● ● ●●

●●●●

●●●●● ●

● ●

●●●

●●●

●●● ●●

●●

●●

●●

●●●●●

● ●

●●

●●

●● ●●

●●●

●●

●●

●●

●●●

●●●●●

● ●

●● ●

●●●

● ●●

●●

● ●●

●● ●●

● ●

●● ●

●●

●● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●●●

●●

●●

● ●

●●

●●

●●●

●●●

●● ●●● ●● ●

● ●●

●●● ●●●

●●

●●

●●●●● ●

●●

● ●

●●

●●

●●

●●●

●●

● ●●

●●

●●●●●●

●●

●● ●

●●●● ●

●●

●● ●●

●● ●●

●●● ●

●●●●●

● ●●

● ●●●

●●●●

●● ● ●●

●●

●● ●● ●

●●

●● ●●●

● ●

●●

●● ●

●●

● ● ●●

●●

● ●● ●

●●

●●

●●

●●

● ●●

●●

● ●

●●●

● ●●●

●●

●●●

●● ●●●

● ●●●●● ●●●●

●● ●● ●●●

● ● ●●●

●●

●●●

●● ●● ●

●● ●

● ●●

●●

●●●

●●●

●●● ●●

●●

●● ● ●

●●

●●

● ●● ●●

● ●

●●

●●●

● ●

●●

●●

● ●●●●

●●●●

●● ● ●●

●● ● ●

●●●

●●●

●●●●

●●

●●

●● ●

●●

●●●●●

●●● ●● ●●

●●●

●●●

●●

●●

● ●

●●●

●●

●●●●

●●●

●●

●● ●

●●

● ●

●●●●● ●

● ●●

● ●●● ● ● ● ●

●●

●●

● ●

●● ●●

● ●

●●

●●

●●

● ●●● ●●● ●●●

● ●●●

●●

● ●●

● ●●

●●

●●●●

●●

● ●●

●●

●●●

●●●●●●

●●

●●●

● ●

● ●●●●

●●●●●

●●

●●●

● ●●

●● ●● ●

● ●

●● ●●●●

●● ●

●●

● ●

●●

●●●●●

● ●●●●

●● ●

●●

●●

●●

●● ●●● ●

●●

● ●●

●●

●●

●●

●●

●●

●●●

● ●● ●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●

●●●

●●

● ●

●●

●●

●●

●● ●●

●● ● ●●

●●

●●

●●

●● ●

●●

●●

●●

●●

● ●

●●

●●●● ● ●●

●●

●●●

●●●●

●●● ●

●●

●●●●

● ●●

●●

● ●●● ●●●

● ● ●●●

●●●●●

● ●●

●●

●●● ●●

● ●● ●●●●●

●● ●

● ●●

●●

●● ●●

●●

●●●● ●

●●

●●

●●

● ●●

● ●●●●

●●● ●

●●

●●

●●●●●●

●●●●

●●● ●

● ●●●●

●●

●●

●●●●

●●● ●●●

● ● ●●●●●●●

●●

●●

●●

●●●

● ● ● ●●●

●●●

●●

●●

● ●●

●●

● ●

●●

●●

●●

●●

●● ●●

●●●●●

●●●●

●●●● ●

●●

●●●●

●●

●● ●

●●

●●●●

● ●

●●●

●●●●●

●●

●●● ●

●●

●●●●

●●●●●

●●

● ●●● ●●●

●●●●

● ●●●● ●●

●● ●

●●●●

● ●● ●●

● ●

● ●

●● ●● ●●

●● ●

●●

● ●

●●●●

●●

●● ●

●●

●●

●●

● ●

●●

● ●

● ●

● ●●●

●●●

●● ● ● ●● ● ●●

●●

●●●●●

● ●●●

●●●● ●

●●

●● ●

●●

●●●●● ●● ●

●●●

●●

●●

●● ●

●●● ●

●●

●●

●●●

●●

●●

●●● ●●

●●

●●

●● ●

●●

●●●

● ●

●● ●● ●

● ●● ●

●●

●●

●●

●●

●●●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●● ●

●●●●

● ●

●●●●

●● ●

●●

●●

● ● ●●

●●●

●●●● ●

●●

●● ●●

●●●

●●

●●

● ●

●●

● ●●●● ●●●

● ●●

●●

●●

●● ●

● ●●●

●● ●●

● ●

●●

●●● ● ●

●●●

●●

●● ●

●●●●●● ●● ●●

●●● ●

●●

●●●

● ●●●●● ●●●

● ●

●●●

●●●

●●●

●●

●●

● ●●

●●

● ●●●

●●●

●●

● ●

●● ●

●●●● ●

●●

●●● ●

●●

●●

●●

●●

●●

●●●

●●●●

●●

●●

●●

● ●

●●●

● ●●● ●

●● ●

●●● ●

●●●

● ●●● ●● ●● ●● ●

●●

●●●●●● ●

●●

●●●

● ●

●●●

●●

●●

●●

●●●

●●

●●● ●●

●● ●● ●

●●

●●

●●● ● ●●

●●

●●●

●●●

●●

●● ● ●

●●●

●●

● ●●●

●●

●●● ●●

● ●● ● ●●● ●●●● ● ●

●●● ●

●●

●●●

●● ● ●

● ●●●

● ●

●●

●●

●●●● ●

●●

●●●

●●●● ●

●●

●●

● ●

●●

●●●

●●●

●●

●●●

●● ●●

●●●● ●

● ●●●

● ●●

●●

●●

●●

●●

● ●● ●

●●

●●

●●●

●●

●●

●●

●● ●

●●●

●● ●●

● ●●

●●

●●

●●●

●●

●●

●●

●●●●●

●●

●● ●●●●

●●●

● ●●

● ●●

●● ●

●●●●

●●

●●

●●

●●●● ● ●

●●

●●

●●

●●●

●●

●●

●● ●●

●●

●●

●●

●●

● ●● ●●

●●●

●●●

●●●

●●

●●

●●

●●

●● ●●●

●●

●●

●●●

●●

●●●●

●●●

●●

●●

●●

●●

● ●

● ●

●● ●●

● ●●

●●● ●

●●

●●

● ●

●●●

●●

●●●

●●●

●●

● ●●

●●

●●

●●●

●● ●●

●●

● ●●●●

●●

●●●

● ●

● ● ● ●

●●● ● ●●

●●

●●●

●●

●●●

●●

● ●

●●

●●

● ●●

● ●●●●●

●●

●● ●●

●●

●●

●●● ●

●●

● ●

● ● ●●●

●● ●

●● ●●●

●●●

●● ●● ●●

●●●● ●●

●●● ●

●● ●● ●

● ●

●●●●●●

●●

●●

●●●

● ●●●

● ●

●●

●●●●

● ●

●●

●●

●●●

●● ●

●●

●●

●● ●●

● ●

●●

●●

●●●

●●●

●●

●●●

●●●

●●

●●●

●●●

●●

●●●

●●●

●●

●●

● ●

●●●

●●● ●

●●●

●●●

● ●●●

● ●

●●

● ●● ●

●●● ●●●

● ● ●●●

●●●

●●

●●● ●

●●

●●

●●

● ● ●

●●

●●●

●●

●●●●●

●●● ●●

● ●●

●●

●●

●●●

●●

●●

●●●

●●

●●

● ●● ●

●●●●

●● ● ● ●●

●● ●

●● ●● ●

● ●●●

●●● ●

●●● ●●

●●●●

● ●●●● ●

●● ●●

●●●● ●●● ●●

●●●

●●●

●●●●● ●

●●

● ●●● ●●

●●

● ●●

●●

●●

● ●●

● ● ●●

●●●

● ●

●●●

● ●

●●

● ●

●●

● ●●

● ●●

●●● ●●● ●●

●●

●●●

●●●

●● ●●

● ●

● ●● ●●

●● ●● ●●

● ● ●●●

●●

●● ●●● ●●●● ●●

●●

● ●●

●●● ● ●

●●

●●

●●

● ● ●●●●●

●●

●●

●●●● ●

●●●

●●

●●●● ●●

●●

● ●●

●●

●● ●●

●●●

●●●● ●●●● ●●

●●

●● ●●

●●●● ●

● ●●

●●

●● ●● ●●●●●

●● ●●●● ● ● ●

●● ●●●

●●●

●●

●● ● ●●

●●

●● ●●●●

● ●●●●●●

●●

●●

●●●● ●●

● ●●

● ●●●

●●●

●●

●●●

●●

●●

● ●●

●●

●●

●●●

●●

●●

●●●●●●

●● ●●

●●

●●

●●

●●

●●

●●

●●●●●

●●

● ●

●●

● ●●

●●

●●

●●●

●●

●●●●● ● ● ●●

● ●●●

●●●

●●

●●

●●● ●

● ●●

● ●●●●●

●●●●

●●●●●●●●●

●●●

●●●●

●●●●

●●

●●●●●●●●

●●●●

● ●●●

●●

●●

●●●

●● ●

●● ● ●● ●● ●

●●●

●●

●● ●●

● ●● ●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●●

●●

●●

●● ●●●●

●●

●●

● ●● ●

●●●

●● ●●

●●●

●●

●●●

●●

●●

●● ●●

●●

●●●

● ●

● ●

●●

●●

●● ●●

●●●

●●●●

●● ●

●●

●● ●●

● ●

●●

●●

●●

●●

● ●● ●●●●●

● ●● ●

●●●

●●●●

●● ●

●●● ●●●●●

●●

●●

●●

●●

● ●● ●

● ●●

●●●

●●●

●● ●

●● ●

● ●

●●●

●●

●●

● ●●

●● ● ●●

●●●

●●

●●●

●●

●●●

●●●

●●●

●●●

●●

●● ●

●●

●●

●●

●●●

●●

●●

●●

●●●●

●●

●●

●●●●

●●

● ●●●●●

●●

●●

● ●●

● ●

●●●● ●●

●●

●●

●●

●●

● ●● ●

●●

●● ●

● ●●

●●

●● ●

●●

●●

●● ●

●●●

●●

●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

● ●●

●●●●●

●● ●●

●●

●●●

●●●

●●

●●●

●●●●

● ●

● ●●

●●●

●●● ●● ●

●●

●●●●

●● ●●

●●

●●

● ●●

●●●

● ● ●

●●●

● ●●●

●●

●●

●●

● ●●●

●●

●●

● ●

●●

●●

●●●●

●●

●● ●

●●

●●●

●●

●●

● ●

●●

●●●

●● ●

●●

● ●

● ●

●● ●● ●●●

●●

●●

●●●

●●

● ●●

●●

●●

●●

●●

●●●

● ●

●●

●●● ● ●● ●

●●

●● ●

●●

● ●

●●

●●

● ● ●●●

●●

●●● ●

●●●● ●●

● ●●●

●●●

● ●

●●

●●

● ● ●● ●●

● ●

● ●

●●

●●

●● ● ●● ●

●●

●●●●●●

●●●● ●● ●● ● ●

● ●

●●●●●●

● ●

●●

●●

● ●

●●

●●

●● ●●

●● ●

●●

●●

● ●●●

●●

●●

●●●●

●●

●●●

●●● ●●●

●●

●●

●●

●●●

● ●

● ●● ●●

●●

●● ●●

●●●

●●

●● ●

● ●

● ●●

●●

●● ●

●●

●●●

●●● ●●

●●● ●● ●●

●●

●●●● ● ●●●

●●

●●●

●●●● ●

●●●

●●● ●

● ● ●● ●●

●●●

●●●

●●

●●●

●●

●●●

●● ●

●● ●●●

●● ●●

● ●●●●

●●●● ●

●●●●

●● ●

●●

● ●

●●●●●

●●●● ●

●●●●

●●● ●●●●●

●● ●

● ●● ●●

●●●●●● ●●

●● ●●●● ●●

● ●● ●

●●

●●

● ●●●● ●●

●●● ●

●●

●● ●● ●

● ●●●

●●●

● ●● ●● ●●

●●●

● ●

●●●

●●●

●●

●●●●●

●●●●

●●●

●●

●● ●●●

● ●●

● ●●

●●

● ●

●● ● ●

● ●

●●

●●

● ●●

●●

●●●●

●● ●●

●●●

●●●●

● ●

● ● ● ●●

●●

●●

●●

●●● ● ●●

●●

●●●● ●

●● ● ●●● ●●

●●

●● ●●●

●●

●●● ● ●

●● ●●● ●

●●

●●

●●● ●●●●

●● ●

●●●

●●●

●●

●●

●● ●

●●● ●●●● ●●

●●

●●●●●

● ●●●●

●●● ●●● ●●●

●●

●● ● ●

●●●

●● ●●

●●

● ●● ● ●●

● ●

●●

●●

●●

●●●●●

●● ●●

●●●

●●

●●

●● ● ● ●●

●●

● ●● ●

●●

●● ●● ●●

● ●● ●●

●●

● ●●● ● ●

●●●

●●

●●

●●

●●

●●

●●

● ●●

●●

● ●●

● ●●

●●

●● ●●●

●● ●

● ●● ●●● ●● ●●●

●●

●●●

●● ●●

●● ●

●●

●● ●●

●●● ●●

● ●

●●

●●

●●

●●●

●●

●●

●●● ● ●

●●● ● ●● ●●

●●●

● ●● ●

● ●

●●

●●

●●

●●

●●●

●●

● ●

●●●●●

●●

●●●

● ●

●●●● ●●●●●●●

● ●● ● ● ●●● ●●●● ●● ●●●●●●

●●

●●

●●●

● ●

●●

● ●

●● ●●

● ●

●●●●

●●

●●● ● ●● ●

●●●

●●●●

●● ●●●●●

●●

●●

●● ●●

● ●

●● ●

● ●●●

●●

● ●● ●● ●● ●

●● ●●

●●

● ●●

●● ●●

●●

●●● ●●

●●

●●

●●

●●●

●●●

●●

●●

●●

●●●

●●

●●

●● ●

●●●

●●

●●

●●

● ●● ●●

●● ●●● ●●●

● ●●

●●●

●●●●

●●

●●

● ●

● ●

●●● ●●

●●● ●●

●●● ●●●●●●

●●●

●●● ●

●●

●●

●●

●●

●●

●●● ●

●●

● ●●

●●

●●

● ●●●

●●● ●

● ● ●●

● ● ● ●● ● ● ●

●●●●

●●●

●●

●●

●●

●●

●●●

● ●

● ●●●

●●

●● ●

●● ●

●●●

●●

●●●●●●

●●●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●●

●●

●●

● ●●

●●

●●

●●

●●

●●● ●

●●

●●

●●●

●● ● ●●●●

●● ●●●●

●●●

●●●

●●●● ●●●

●●●●●

●●●

●●●●

● ●●

● ●●●

●●

●●●

●●

●●

●● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

●●● ●● ●

●● ●

● ●

●●

● ●

●●

●● ●

●●●

●●●

●●●

●●●● ●●

● ●●

●● ●

● ●●●

●●

●●

●● ●●

●●●●●

●●

●●

●●

●●●●

●●● ●

●●●●●

●● ●● ● ●

●●●●●●●●

●●●●●●

● ●●●

●●●●

● ●● ●

●●●●

●●

●●● ●

●●

●●● ●

●●

●●

●●

●●●●

●●●

●● ●

●●

●●

●●●

●●●●●●

●●

●●

● ●●

●●

●●●

●●

●●

● ●●

●●

●●

● ●●●

●●

●●

●●●●

●● ●●●

● ●● ●

●●

●●

●●

●●

●● ●●●

●● ●

●●●● ●●

●●

●●●

●●

●●

●●

●●● ●● ●

●● ●● ●

●●

● ●●●

● ●

●●

●●

●● ●●

●●●● ●

●●●●

●●●

●●●

● ●●

●●●●●

●●

●●● ●

●●

● ●●

● ● ●●●●●

●●

●●

●●

●●● ●

● ●

● ●

●●●●●●●

● ●●

●●●

●●●●●

●●

●● ●●

●●

● ●

●●

●●

●●

●●

●●●

● ●●● ●●

● ●

●● ●

●●

●●

● ●●

●●●

●● ● ●●●●

●●●

●●

●●●

●●

●●

● ●●

●●● ●

●●

●●●

●●●●●● ●●

●●●

● ●● ●

●●

●●● ●●

●● ●● ● ●●●

●●

●● ●●

●●●● ●●● ●

●●●

●●

●● ●●● ●

●●● ● ●

●● ●

●● ●●

●●

● ●●●

●●●●●●●

●● ● ●●

● ●

●●●

●●

●●●

●●

●●●

●●●●

●●

●●

●●● ● ●●

●●● ●●●

●●

●● ●●●●● ●● ●

●●

●●●●●● ●

● ● ●●●● ●●

●●●●

●●● ●

● ●

●●

●●●

●●

●●●●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●● ●

●●

●●

●●

●●●

●● ●

●●

● ●●●●

● ●

●●●

●●● ●

●●●●

●●

●● ●

●●●

●●

●●● ●

●● ●

● ● ●● ●●●●● ●

● ●●●

●●● ●●●

●●

● ●

●●● ●

●●●

● ●●

● ●

●●●●●

●●

●●●

●●● ●●

● ●● ●●●

●●

●●

●● ●

●●●●

●●●

●●

●●

●●

●●

● ●●●● ● ●

●●●●

● ●●

● ● ●●●●

●●

●● ●

● ●●

● ●●

● ●●

●●

●●●

●●●

●●●

●●

● ●●● ●

●●●●●●

●●●●

●●●

●●

●●●

●●●

●●● ●●

●●● ●● ●

●●●

● ●●●

●●

●●

●● ●

● ●●

●●●

●●

●●

●●●

● ●● ●

●●

●●

●●

●● ●● ● ●●

● ●●

●●

●●●●

●●

●●

●●

●●● ●● ●

●●

●●

● ●●●●

●●●●●●

●●●●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●● ●● ● ● ●

●● ● ●

●●●

● ●

●●

●●

●●

●●

● ●●

●●

●●

●●●

●●

●● ●● ●●

●●●

●●●●

●●

●● ●●

●● ●●● ●

●●●●●

●●

●●

●●

●●●●●●

●●

●●

● ●

● ●●

●●

●●

●●

●●

●●●

●●

● ●●

●●●●

●●●●●●

● ●

●●●

●●

●●● ●

●● ●●

●● ●●

●●

●●● ●

●●●●

●●

●●● ●●●● ● ●

●●

● ●● ●●●

●●

●●

●●

●●

● ●● ●●

●●

●●●● ●●●●

●●●● ●

●●

●●

●● ●

●● ● ●

●● ●● ●

●●

●●●● ●●

●●●●

●●●● ●●

●●

●●

●●

●●● ●●● ●●●

●●

●●●●

● ●● ● ●●●

●●

●●●

●● ●

● ●●

●● ●●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●● ● ●●● ● ●●

● ●●

●●

● ●

●● ●●●●

●●

●●●●

● ●

● ●●● ●● ●● ●●

●●● ●●●

●●

●●

●●

●● ●●

●●

●●

●●

●●

●● ● ●

●● ●●

●●

●●

●●●

●●

●● ●●

● ●●●

●●

●●●

●● ●

●●

●●

●●● ●● ●

●●● ●● ●●

●●

●●

●●● ●

● ●

●●●

● ● ●●● ●

● ● ●●

●●●●

● ●●●

●●●●●

●●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

● ●●●

●● ●

●●

● ●●

●●

●●●●● ●

●●

●●

●● ●

●●

●● ●

●●

●●

● ●●●

●●●●

●●

●●●●

●●

●● ● ●

●●

●●

●●

●●

●●

●●●●

●●

●●●

●●

●●●

● ●

● ●

●●

●●

●●

●●●

● ●

● ●●

●●●

● ●●●

●●

●●

●●

● ●

● ● ●●●

●● ●● ●

●● ●

●●●

●●●●

● ●●

●● ●

●●●

●●

●●

●●

● ●● ●

●●●

●●

● ●●●

●●●●●● ●●●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●●

● ●

●●●

●●

●●

●●●

● ● ●●● ●

●●●●●

●●

●●● ●

● ●●

●●

●●

●●

●●

●●

●●●

●●●

●● ●●

●●

●●

● ●●● ●

● ●●●●

●●

●●

●●

●●●

●●

●● ●●

●●●

●● ●●

●●

●●

●● ●●

●●

●●

●●●

●●

●● ●

●●

● ●●

●●●●

●●

●● ●

●● ●

●●●

●●

●●

● ●●

● ●● ●● ●

●● ● ●●

●●

● ●●●

●●

●●

●●●

●● ●

●●

● ●● ● ●●

●●

● ●●●

●●●

●●

●●

●●

●●

● ●●● ●●

●●

●● ●

●●

●●

● ●●●

●●●

● ●●

●●

●●

●●

● ●●

●●

●●●

●●

●● ●●

● ●

●●

●●●

●●●●

●●●

●●

●●

● ●●●

●●●●●

●●

●●

●● ●●●

●●

● ●●

●● ● ●● ●●●

● ●● ●●● ●

●●

●●

●●

● ●

●●

● ●●

●●

●●

● ●●

● ●●

●●

●●●

●●

●● ●

●●●

●●●●●

●●●

●●

●● ●

● ●

●●●

●● ● ●●

●●● ●●●●●

●● ● ●

● ●●●

●●

●●

● ● ● ● ●●

●●●●●●●

●●●●●●●●

●● ● ●●

●●

●● ●

●●

●●●

●●●●●●

●●●

●●

●●

●●

●●● ●

●●● ●

●●

●●

● ●●

● ●

● ●●

●●

●● ●●

●●● ●●

● ●

● ●

●●

●●

●●●

●●

●●●●

●●●● ●

●●

●●

●●

●●

●●

●●

● ●● ●●●

●● ●

●●

●●

● ●● ●

●●

●●

● ●●● ●

●●

● ●●●●●●●

●●

●●●●

●● ●●

●●

● ●

● ●●● ●●● ●

● ●● ●

●●

● ● ●●

●●

●●

●● ●

● ●

●● ●●

● ●●

●●●

● ● ● ●●●● ●

●●

●●

●●●●

●●●●●●●

●●

● ●●●

●●

●● ●●

●● ●

●●

●●

●●

●●

●●●●●●

●●

● ●

●●

●●●

●●

●●

●●● ● ●

●●

●●

●●●

●●●

●● ●

●●

●●

●● ●

●●

● ●

●●

●●

●●

● ●●●●●

●● ●●

●●●

●● ●●● ●● ●●●●

●●

● ●●

●●

●●● ●●

●● ●●

●●

●●●

●●

●●

●●●

●●●

● ●

●●

●●●

●● ●

● ●●

●●●

●●

●●

●●●●●●

●●● ●

● ●

●●

● ●

●●

● ●●●

● ● ●●●

● ●●●●

●●●●

●●●

● ●

●●● ●●●●

●● ●

●●●

●●

●●

●●●

●●

●●

●●●●●

●●●

●●

●●●

●●

●●

●●●

● ●●●●

●●

●●

●●

●●●●●

●● ● ●● ●● ●

●●●

●●

●●●●

● ●●●

●●

● ●●

● ●●

●●●

●●●●

●●

●●

●●

●●●●●

● ●

● ●●●●

●●●●

●●●●●● ● ●●●●● ●●●

● ●

●● ●●● ●

●●●● ●

●●

●●●● ●●

●● ●

●●

●●

● ●●

●●

●●

●●●●

●●● ●

●● ●

●●●

●●●● ●●● ●

●●● ●

● ●

●●

●●●

●●

●●

● ●

● ●

●●●

●●●

●●

●●

●● ●

●●●●

●●●

●●

●●

●●

●●

●●●

●●

●● ●

●●

●●

●●

●●

●●

●●●

● ●

●●●

●●

●●

●●●●

●● ●● ●●●

● ●

● ● ●● ●

●●

●●● ●●● ●

●●

● ●●

●●●

●●

●●● ●

●●●

●●

●●

●● ●

●●●●

●● ●●●

●●

● ●● ●●

●●

●●●

●●

●●● ● ●●●

●●● ●●

●●

●●

●●

●● ●●●

●●

●● ● ●

●●

●●●●●● ●

● ●● ●●●

●●● ●●●●

●●●

● ●● ●

● ●●●

● ●● ●●

● ●●●● ●●

●●

●●

● ●●

● ●●●

●●

● ●●

●●● ● ●

● ●●

●●

●●

●● ●● ●●

●● ●

●●● ●

●●●

●●

●● ●

●●

●●

●●

●●

● ●●●●

●●

●●●●●

●●●

●●●

● ●

●● ● ●

●●●

●●

●●● ● ●

●●●● ●●

●●●

●●● ●● ●

● ●●

●● ●

●●

●●●

● ●●

●●●

●●

●●

●●

●●●

●●●

●●●

●●

●●

● ● ●

●● ●●

●● ●

●●●●

●●

● ●●●

●●●

●●

●●●

●●

●●

● ●●●●

●●

●●●●

●●●

● ● ●●● ●●●

●●

●●●

●●●

●●● ●

●● ●●

●●

● ●●

●●●●

● ●●● ● ●●

●●● ●● ●●

●●●●

●●

●● ●●● ●●●

●●●

●●

●●

●●

●●

●●●

●●● ●● ●

●●

●● ● ●●

●●

●●

●●● ●

●●

●●

●●

●●

●●

●●●

●●● ●

●● ● ●●

●●

●●

●● ●

●●●●

● ●

● ●●●●

●● ●

●●

●●

●● ●

●●

●●

●●

● ●●●●

● ●● ●

●●●●

●● ● ●

●●

● ●

● ●

●●

●●

●●

●●

● ●● ● ●●●●

●●

●●

●●●

● ●●

● ●

●●

●●

●●

●●

●● ● ●

●●

●●

●●

●●

●●

●●

●●●

●●●

●●

●●

●●●

●●●●●●

●●

●● ●●

● ●

●●●

●● ●●

●●

● ● ●●● ●●●●

●●

●●●●●●●

●●

●● ●

● ●●●

●●

● ●●

●●●

●●

●●●

●● ●● ●

●●

● ●●

●●●● ●

● ●

●●

●●

●●

●●

●●●

● ●●

●●● ●●

●●

●●

● ●● ●●

●●

● ●

●●

● ● ●●●

●●

●●

●●●●

● ●●

●●

● ●●●

●●

●●

●● ●●

● ●●● ●●

●●

●●

●●

●● ●

●●●●● ●●●

●● ●

●●

●●

● ●

●●

●●

●●

●● ● ●

●●

●●

●●

● ●●

●● ●

● ●●●

● ●

●●

●●●

●●●●

●●

●●●

●●

● ●●

●●

●● ●●●●

●●

●●● ●

●●

●●

●●●●●●●●●●●●●

●●

●●

●●●●

●●

●●

●●

●●

● ●

●●

●●

●●● ●●

● ●●

●●●

● ●

●●

●●

●●

● ●

● ●●●●●

● ●●● ●

●●

●● ●● ●

●●●

● ●●

●●● ●

● ●

●●

●●

●● ● ●●● ●

●●

●● ●●●●

●● ●● ●

●● ●

●● ●●

●● ●●

●●

●●

●●

●●

●● ●●

●●

●●

●●●

●●

●●●

●● ●●●

● ●●

●● ●

●●

●● ●●●

●● ● ●

●● ●

●●●●

●●

●●

●● ●

●●●

●● ● ●●●●●●●●

●●●

● ●● ●●● ●● ●●

● ●●

●●

●●

●●●

● ●●●●

●●●●

● ●

●●

●●

●●●●●●

●●●

●●

●●

●●

●●

●●●

●●

●●●●

●●

●●●●

● ●

●●

●●

●●

●●

● ●●

●● ●

● ●● ●●

●● ●

●●

●●●

●●

● ● ●●

●●●●

●● ●

●●

● ● ●● ●●

●● ●

●●

●●

●●●●

●●

●●●●●●

●●

●● ●

● ●●●●●

●●

● ●●● ●

●●●

●●

●●

●●

●●

●●

● ●

●●●

● ●●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●●

●●●●●●●

●●

●●

● ●●●

● ●●● ● ●

●● ● ●

●●

● ●●●

● ●● ●

●●● ●●● ●

●●

●●●

● ●●

●●● ●

● ●

●●

● ●

●●● ●

●●●●

●●

●●

●●

● ●●●

●●

●●●●

●●●

● ●

●● ●

●●

●●

● ●● ●

●●

● ●

●●●

●●

●●

● ●

●●

● ●●

●●

●●

●●●

●●● ●● ● ● ●●●

●●●● ●

●●● ●●

●●

● ●●●●

●●●●●●●●●●●

●●● ● ● ●

●●

● ●●●●●

●●● ●●●●

●● ●●

●●

●● ●

● ●● ●● ●

●●

● ● ●●●●

●●

● ●

●●

● ●●● ●

●●

●●●

●●

●●● ●●

●●●

● ●●

●●

●● ●

●●●● ●●● ●●

●● ● ●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●● ●

●●

● ●●

●●●

●●●

●●

●●

●●

● ●●●

●●●

● ●●●●

●●

●●●

●●●

●●●

●●

●●●

●●

●●●●

●●

● ●

●●● ●● ●● ●

●●

●●●

● ●

●●

●●

●●●●●

●●

●●

●●

●●●

●●

●●

●●●

●●

● ●

●●

●●● ●

●●

●●

●●

● ●● ●

●● ● ●

●●

●●

●●

●●

● ●

● ●●●

● ●●

● ● ●●●●●●

● ●● ●●

●●

●●● ●●●

●●

●●●

●●

●●●

●●●●

●●●●●

●● ●●●

●●● ●

●●

●●

● ●

●●

●●

●●

● ●

●●

●●

● ●●

● ●●

● ●

●●

●●

●●

●●● ●●

● ●

●●

●●

● ● ●●

●●●● ●

● ●●●

●●

●●

●●

●●

●● ●● ●●

●●●

●●●● ●

●●●●

● ●

●●

● ●●

●●

● ● ●●●

●●

●●

●● ●

●●

● ●

●●

●●

●● ●●

●● ●●

● ●●●●●●● ●

●●

●●

● ●

●●● ●● ●

●●

●●

● ●●

●●

● ●●●

● ●●

●●

●●●

●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

● ●

●●●

●●●

●● ●

●●

●●

● ●●

●● ●

●●

●●●●●

●●

● ●● ●●

●●

●●

●●●●

●●

●●

●●

●●●●

●●

●●

●●

●●●

● ●●

● ●●●●●●

●●

●● ●

● ●

●●

● ● ●

● ●●

● ●●

● ●●

● ●●●

● ● ●●

●●

●●●

●●●●

●● ●●

●● ●

●●

●●

●●

●●●

●●

● ●

●●

●●

●●

●●●

●● ●●●●

● ●●●

● ● ●

●●

● ●●

●● ●

●●

●● ●●

●●

●●

●●

●●

●●

● ●●

●●

●● ●

●●

●●

●●

● ● ●● ●●

● ●●

●●

●●

●● ●●

●●● ●●●

●● ●● ●●

● ●●●●

●●●● ●●

●●

● ●●

●●

●● ●●

●●●●

●●

●●

●●

●●

●●

●●● ●

● ●● ●●

●●●

●●●

● ●●

●● ●

●●●

● ●●●

● ●●●●

●●●

● ●●● ●

●●● ●

● ● ●

●●●

●●

●● ●●

●●●●●●●●●

●●

●●●

●●

●●●

●●

●● ● ●

● ●

●●

●●

● ●

●● ●

●●●●● ●

●●

● ●

●●● ●●

●●

●●

●●●●

●●

●● ●

●●

● ●●

●●●

●●

●●

●●

●●● ●●●

●●

●●

●● ● ● ●●

●●●●●●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●●

●●●

●●

● ●●●●●

● ●

●●

●●●

●●●●● ●●●

● ●

●●

●●●

● ● ●●●

●●●

●●

●●●

●●●●● ●●

●● ●

● ●●

●● ● ●

●●●● ●●●●●

●●●

● ●●● ●

●●●

● ●●

●●

●● ●

● ●

●●● ●●

●●●

●●

● ●

●●

●●

●●

●●

●● ●

●●

● ●●

●●● ●●

●●●●

●●

●●

●●●●

●●

●●

● ●● ●● ●●

●●●

●●

●●

●●

●●

●● ●●

●●●

●●

●●●

●●

●●

●●●●

●●

● ●

●● ●

●●

● ●

●●

●● ●●

●● ●

● ●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

● ●●

● ●●● ●

● ●●●●● ● ●● ● ● ●● ●

●●●●

●●

● ●

●●

●●

●●

●● ●●

●●

●● ●

●●

● ●●

●●

●●

●●

●●● ●

●●●

●●

● ● ● ● ●● ●

●●●●●

●●

●●●●

● ●

●●

●● ●● ●●●

●●●

● ●● ●● ●●

● ●●

● ●● ●●

●●

● ●● ● ●●●●●

●●

●● ●

●●

●●● ●● ●● ●

● ●●

●● ●●

●● ●● ●●● ●

●●●

●● ●

●●●

●●

●●●

●●

●●●●

●●

●●

● ● ●●●●

●● ●●● ●

●●●●●● ●

●● ●●

●●

●●

●●

● ●

●●●

●●

●●

●●● ●

● ●●

●●

● ●●●

●●●

●●

●●

●●●

●●

● ●

●●●

● ●●

● ● ●● ●

●●

●●

●●●

● ●●

●●●

●● ● ●●

●●

● ●● ●●

●●●●

●●●

● ●● ●

●●

●●

●●●●●●●●

● ●

●●

●●●● ●●

●●

●●

●● ●●

●●● ●●●●

●●●●●●●●●●●

●●●●●

●●●

●●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●

●● ●●●●

● ●● ●●●●

●●●

●●

● ●●●

●●

●●●●

●●

●●● ●

● ●●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●●●

●●

● ● ● ●

●●●

●●●

● ●●●

●● ●●●

●●

●●

●●

●●

●●

● ●●

●●

●●●

●●

●●

●●●

● ●●●

●● ● ●●●

● ●

●●●

●●●

●●

●●

● ●

●●

● ●●● ●●

●●●● ● ●●

● ● ●● ●

●●●

● ●●

●●

●●● ●

●● ●

●●

● ●●

●●

● ●●●●● ●●●

●●

●●

●● ●●

●●●●● ●

●● ●●●●●●

● ●

●●

● ●●●●●●●● ●

●●●

●●●

●● ●

●●

● ● ●

● ●

●●●● ●

●●

●●

●●●

●● ● ●

●●

●●

●●●●

●●

●● ● ●●

●●

●●●

●●

●●●

●●

●●

●●

● ●●●

●●

●●

● ●

●● ●

● ●●●

●●

●● ● ●● ● ●

●● ●

● ●●●●

● ●

●●

●●

●●

●●

●●

●●●

●● ● ●

●●

●● ● ●● ●

●●

●●●

●● ●●●

●●

●●●

●●

●●

●●

●●

● ●●

●●

●●

●●●

●●

●● ●

●●

●●

●●●●

●●●● ●●

●●

●●

●●● ● ●

●●

●●

●●

●● ●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●● ●

●● ●●● ●●●●●

●●●●●●●

●●

●●●●●● ●

● ●●

●●● ●●●

●●

●●●

● ●●

●●

●●

●●

●● ●

●●

● ●

● ●

●● ●● ●●

●●

●●

●●

●● ●

●●

●●

●●●

●● ●●

●●

●●●

●●●●●●●

● ●

● ●●

● ●

●●●●

●●

● ●●●

● ●●

●●

●●●

●●● ●●

●●●●●

●●●

●●

● ●● ●●

●●

●●● ●

●●●●●●

●●●

● ●● ●●

●●●

● ● ●●

●●

●●

● ●●●

●● ● ●

● ●●●●●●● ●●●●

●●

●●● ●

●●

●●

●●●

●●●

●● ●

● ●●

● ●●●

●●

●●

●●●

●●

● ●●●●●

●●

●●●

●●●●

●●●

●●●

● ●●●

●●●

●●

●●●

●●

●●

●● ●●●

●●●●●●

●● ●●●

●●

●●

●●● ●● ●

●●● ● ●● ●

●●

●●

● ●●●●●

●● ●

●●

●●●

●●

● ●●

●●

●●

●●

●●

●●●

●●●

●●

●●● ●

● ●

●●

●●

●● ●

●●●

● ●

●●

●●

●●

●●●

●●●

●●

● ●● ●

● ●● ●●

● ●●●

●● ●●

●●

●●

●●

●●

●●

●●

●●

●● ●●

●●

●●●

●● ●

●● ●

●●

●●

●●

●●

●●

●● ●

●●●

●●●

●●●

● ●

●●

● ●

●●

●●

●●● ●

●●●●

●●●●●

●●

●●● ●● ●●

●● ●● ● ●

●● ●●

●●● ●● ●●

●●● ●●

●●

●●●

●●

●●

●●● ● ●

●●●

●●

●●

●●● ●

●●●●

●●●

●●

●●●

● ●

●●

●●

●●●●

●●●

●●

●● ●

●●

●●

●●

● ●●●

●● ●

●●

●●

●●

●● ●●

●●

● ●

● ●

● ●

● ●●●

● ●● ●●●

●●●

● ●●

●●

●●

●●●

●●●

●●

●●

●●●●●● ● ●

●●

●●

● ●

●●

●●●

●●

●●

●●

●●

●●

●●

●●

●● ● ●

●●

●●●

●●●

●●

●●●

● ● ●●●

●● ●

●●

●●

●●

●●●●●●

●● ●

●●

●● ● ●

●●

● ●●● ●●

●●●●

●●●

●●●

●●

● ●●

●● ●●

●●

●●●

●● ●

●● ●●●

●●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

● ●

●● ●●

●●

●● ● ●●

●●

●● ●

●●

●●●● ●●

●●● ●●

●● ●

● ●●

●●

●●

●●

●●● ●

● ● ●

●●

●●

●● ●●

●●●●●●●

●●

●●●

●●

●●●

●● ●

●●

●●●●

● ●●

●●●

● ●●●

●●●●●

●●

●●

●●●

● ●

●●●●●● ●

●●

●●

●●

●●

●●●

●●

●●

●● ●

●●●●

●●

● ●● ●● ● ●●●●●

●●

● ●●

●● ●

●●

●●●

●●

●● ●

●●

● ●

●●

●●

● ● ●●

● ●

●●

● ●

●●●

●● ●●

●● ●

●●●

●●

● ●

●●

● ●

●●

● ●

●●

●●

●●

●●● ● ●● ●

●●

●●● ●● ●● ●

●●●● ●

●●

●●● ●●●

●● ●

●●

●● ●

● ●●●

●●●

●● ●●●●

● ●● ●●

●●●

●●

●●

10 20 50 100 200 500 1,000 2,000 5,000

Rl =√

Bl +WlWl

, l ∈ L.

[email protected] Random Seeds Output Variability in MATSim 27 / 33

Page 29: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Conclusions & Future Work ConclusionsFuture Work

Outline

1 Literature Review2 Methodology3 Case Study & Results4 Conclusions & Future Work

ConclusionsFuture Work

[email protected] Random Seeds Output Variability in MATSim 28 / 33

Page 30: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Conclusions & Future Work ConclusionsFuture Work

Conclusions I

FindingsCoefficient of variation is generally low,and a decreasing function of link load.Large relative errors do happen – also forthe busiest links.No bad seeds.Between-seed variation is much largerthan within-seed variation.

[email protected] Random Seeds Output Variability in MATSim 29 / 33

Page 31: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Conclusions & Future Work ConclusionsFuture Work

Conclusions IIHow to Deal with It?

Use multiple seeds.May be better to run 10× 100 iterationsthan 1× 1, 000.

Average out results across all seeds.But point estimates may not correspondto an actual solution.

Present results as a distribution.As suggested in Chapter 48∗of theMATSim book.Contributes to deeper understanding ofuncertainties.

∗Flotterod, Gunnar (2016). “MATSim as a Monte-Carlo Engine”. In: TheMulti-Agent Transport Simulation MATSim. Ed. by Andreas Horni, Kai Nagel, andKay W. Axhausen. London: Ubiquity Press. Chap. 48, pp. 361–370.

[email protected] Random Seeds Output Variability in MATSim 30 / 33

Page 32: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Conclusions & Future Work ConclusionsFuture Work

Future Work

Spatial AnalysisCurrent study ignores spatial interdependencies.

Visual overview.Actual geostatistical analysis (e.g. Kriging).

Network SensitivitySensitivity to small changes in the network.

Does the output variability overshadow theeffects of infrastructural changes?Extremely relevant for scenario analysis.

[email protected] Random Seeds Output Variability in MATSim 31 / 33

Page 33: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Thank You for Your Attention

Output Variability Caused by Random Seeds in aMulti-Agent Transport Simulation Model

7th International Workshop on Agent-based Mobility, Traffic andTransportation Models, Methodologies and Applications

Mads Paulsen, Thomas Kjær Rasmussen & Otto Anker Nielsen

Division of Transport Modelling,DTU Management Engineering,Technical University of Denmark

[email protected] Random Seeds Output Variability in MATSim 32 / 33

Page 34: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Bibliography

References I

Castiglione, Joe, Freedman, Joel, and Bradley, Mark (2003). “Systematic Investigation of Variability due to Random SimulationError in an Activity-Based Microsimulation Forecasting Model”. In: Transportation Research Record: Journal of theTransportation Research Board 1831, pp. 76–88.

Cools, Mario, Kochan, Bruno, Bellemans, Tom, Janssens, Davy, and Wets, Geert (2011). “Assessment of the effect ofmicro-simulation error on key travel indices: Evidence from the activity-based model Feathers”. In: 90th Annual Meeting ofthe Transportation Research Board. Washington D.C.

Flotterod, Gunnar (2016). “MATSim as a Monte-Carlo Engine”. In: The Multi-Agent Transport Simulation MATSim. Ed. byAndreas Horni, Kai Nagel, and Kay W. Axhausen. London: Ubiquity Press. Chap. 48, pp. 361–370.

Horni, Andreas, Charypar, David, and Axhausen, Kay W (2011). “Variability in Transport Microsimulations Investigated Withthe Multi-Agent Transport Simulation MATSim”. In: ETH Library. Working Paper.

Horni, Andreas, Nagel, Kai, and Axhausen, Kay W, eds. (2016). The Multi-Agent Transport Simulation MATSim. London:Ubiquity Press.

Kickhofer, Benjamin, Hosse, Daniel, Turner, Kai, and Tirachini, Alejandro (2016). “Creating an open MATSim scenario fromopen data: The case of Santiago de Chile”. In: VSP Working Paper 16-02. VSP Working Paper.

Lawe, Stephen, Lobb, John, Sadek, Adel, Huang, Shan, and Xie, Chi (2009). “TRANSIMS Implementation in ChittendenCounty, Vermont”. In: Transportation Research Record: Journal of the Transportation Research Board 2132, pp. 113–121.

Nagel, Kai (1997). “Experiences with iterated traffic microsimulations in Dallas”. In: Traffic and granular flow ’97. Ed. byMichael Schreckenberg and D. E Wolf. Heidelberg: Springer, pp. 199–214. arXiv: 9712001v1 [arXiv:adap-org].

Veldhuisen, Jan, Timmermans, Harry, and Kapoen, Loek (2000). “Microsimulation Model of Activity-Travel Patterns and TrafficFlows: Specification, Validation Tests, and Monte Carlo Error”. In: Transportation Research Record: Journal of theTransportation Research Board 1706, pp. 126–135.

Ziems, Sarah Ellie, Sana, Bhargava, Plotz, Joseph, and Pendyala, Ram M. (2011). “Stochastic Variability in Microsimulationmodeling Results and Convergence og Corridor-Level Characteristics”. In: Transportation Research Board 90th AnnualMeeting. Vol. 11-3560.

[email protected] Random Seeds Output Variability in MATSim 33 / 33

Page 35: Output Variability Caused by Random Seeds in a Multi-Agent … · Output Variability Caused by Random Seeds in a Multi-Agent Transport Simulation Model 7th International Workshop

Random Number Generation in MATSim?

Linear Congruential Generator (LCG)Pseudo random numbers are constructed in MATSim using Java’s defaultLCG with minor modifications,

Xn+1 = aXn + c mod m.

Xn = latest draw.Xn+1 = next draw.a = 25, 214, 903, 917.c = 11.m = 248.

In order to get a uniformly distributed number Xn+1 is divided by m.

[email protected] Random Seeds Output Variability in MATSim 34 / 33