Niek Ba er - University of Twente Research Information · Voorwoord Ruim vier jaar geleden begon ik met het onderzoek voor dit proefschrift en er zijn momenten geweest dat ik niet

Queueing and Traffic

Niek Baer

Graduation committee:

Chairman and secretary: prof. dr. P.M.G. ApersSupervisor: prof. dr. R.J. BoucherieCo-Supervisor: dr. J.C.W. van Ommeren

Members:dr. techn. B.F. Nielsen Technical University of Denmarkprof. dr. R. Nunez Queija University of Amsterdamprof. dr. N.M. van Dijk University of Amsterdamprof. dr. ir. E.C. van Berkum University of Twentedr. ir. A. Al Hanbali University of Twentedr. G.J. Still University of Twente

CTITCTIT Ph.D.-thesis Series No. 14-339Centre for Telematics and Information TechnologyUniversity of TwenteP.O. Box 217 – 7500 AEEnschede, The Netherlands

Beta Ph.D.-thesis Series No. D190Beta Research School for Operations Managementand LogisticsEindhoven University of TechnologyP.O. Box 513 – 5600 MBEindhoven, The Netherlands

This work was financially supported by the Centre for Telematics and InformationTechnology (CTIT) of the University of Twente.

Typeset with LATEX. Printed by Ipskamp Drukkers.Cover designed by Niek Baer

ISBN: 978-90-365-3812-1ISSN: 1381-3617 (CTIT Ph.D.-thesis Series No. 14-339)DOI: 10.3990/1.9789036538121

Copyright c© 2015, Niek Baer, Deventer, the NetherlandsAll rights reserved. No part of this publication may be reproduced without theprior written permission of the author.

QUEUEING AND TRAFFIC

PROEFSCHRIFT

ter verkrijging vande graad van doctor aan de Universiteit Twente,

op gezag van de rector magnificus,prof. dr. H. Brinksma,

volgens besluit van het College voor Promoties,in het openbaar te verdedigen

op vrijdag 12 juni 2015 om 14.45 uur

door

Niek Baer

geboren op 29 juli 1986te Lelystad, Nederland

Dit proefschrift is goedgekeurd door:prof. dr. R.J. Boucherie (promotor)dr. J.C.W. van Ommeren (assistent promotor)

Voor Ellen.

Voorwoord

Ruim vier jaar geleden begon ik met het onderzoek voor dit proefschrift en er zijnmomenten geweest dat ik niet had verwacht dat er ooit nog een proefschrift zoukomen. Dat dit eindresultaat alsnog tot stand is gekomen komt door de steun vangroot aantal mensen, te beginnen bij mijn promotor.

Richard, wij leerden elkaar kennen toen ik in mijn vierde jaar mijn Bachelor afs-loot bij SOR. Onder jouw en Judiths begeleiding hebben we een subsidieringsmodelgemaakt voor het Nederlands Filmfonds. Ik raakte geınspireerd door jouw stijl vanwerken en toen ik na mijn studie besloot om te gaan promoveren, was de keuze voorde leerstoel SOR snel gemaakt. Door de jaren heen wist jij je altijd te interesserenvoor mijn werk en wist je, door het stellen van kritische vragen, ons onderzoek vaakop een hoger niveau te tillen. Richard, wij konden niet altijd samen door een deur,maar ik ben je heel erg dankbaar dat jij mijn promotor hebt willen zijn.

Jan-Kees, vele malen ben jij voor mij een luisterend oor geweest. Als er weereen artikel afgewezen was, of wanneer de voorzitter van een sessie op een conferentiehet op mij gemunt had, of gewoon zomaar; jouw deur stond altijd voor mij open,dank daarvoor. We hebben ook vele interessante discussies gehad over zowel jouwonderzoek als het mijne. Toch blijft er na al die jaren nog n vraagstuk onbeantwoord:de eigenaardige overeenkomsten tussen q = k · v en de Wet van Little. Beide zijnwe er van overtuigd dat er een verband bestaat tussen die twee, en het lijkt mij leukom hier samen eens een regenachtige zondagmiddag, of een zonnige (maar dan welop het terras), aan te besteden.

Ahmad, every now and then, you joined us in our struggle to fully understandthe threshold queues. Your constant enthusiasm is inspiring and it made me reallyenjoy the discussions we had. Thank you for the many fruitful discussions we had.

Bo, during the autumn of 2013 I have visited you for three months in Lyngby.I was reluctant to leave home for several weeks, but arriving in Denmark, I waswarmly welcomed by you and David, the two of you made me feel right at home.David, I really enjoyed our coffee breaks and lunch times during my visit. I am stillworking on losing all the weight I gained by eating all those cookies. I am reallygrateful to the both of you: Tak!

Furthermore, I would like to thank the members of my graduation committee. Iam honoured that you were all willing to participate in my graduation committeeand I want to thank you for your careful reading of my thesis.

Mijn dank gaat uit naar al mijn collega’s aan de UT voor de leuke gesprekken bijde koffiemachine (ook al was de koffie niet te drinken) en tijdens de gezellige lunches.Jullie aanwezigheid zorgde voor veel plezier op de werkvloer.

viii Voorwoord

Maartje, ik kan er natuurlijk niet onderuit komen om jou bij naam te noemenin mijn voorwoord. Lange tijd hebben wij een kantoor gedeeld in de Zilverling enhebben we de grootste lol gehad om wie de slechtste muziek kon draaien. Door jou ismijn kennis van carnavalskrakers flink opgevijzeld, iets waarvoor ik nog steeds nietweet of ik daar nou dankbaar voor moet zijn of niet. Desalniettemin heb ik heel erggenoten van onze tijd als kamergenoten.

Een speciaal dankwoord voor mijn familie en vrienden, voor jullie steun envertrouwen. In het bijzonder wil ik mijn paranimfen Hans en Chiel bedanken. Ikwaardeer het enorm dat jullie mij willen bijstaan tijdens de afronding van mijn pro-motie.

Als laatste wil ik jou bedanken, Ellen. Jij weet als geen ander hoeveel moeite ikheb gehad om mijn promotie af te ronden en zonder jouw onvoorwaardelijke steunhad ik het zeker nooit gehaald. Juist op die momenten dat ik jou het hardste nodighad, was jij er voor mij, no matter what. Daarvoor ben ik je voor altijd dankbaar.

NiekDeventer, april 2015

Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

I Traffic 5

2 Queueing models for highway traffic flows 72.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.1 Microscopic, mesoscopic and macroscopic models . . . . . . . 82.2.2 Fundamental diagram . . . . . . . . . . . . . . . . . . . . . . 10

Single-regime traffic models . . . . . . . . . . . . . . . . . . . 10Multi-regime traffic models . . . . . . . . . . . . . . . . . . . 11

2.2.3 Queueing theory in uninterrupted traffic flows . . . . . . . . . 12Heidemann’s model . . . . . . . . . . . . . . . . . . . . . . . 12Jain and Smith’s Model . . . . . . . . . . . . . . . . . . . . . 15

2.3 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Threshold queueing describes the fundamental diagram of unin-terrupted traffic 173.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2 Two-stage M/M/1 threshold queue . . . . . . . . . . . . . . . . . . . 19

3.2.1 Model validation . . . . . . . . . . . . . . . . . . . . . . . . . 213.2.2 Sensitivity analysis of the fundamental diagram for the two-

stage M/M/1 threshold queue . . . . . . . . . . . . . . . . . 223.3 Four-stage M/M/1 feedback threshold queue . . . . . . . . . . . . . 22

3.3.1 Model validation . . . . . . . . . . . . . . . . . . . . . . . . . 283.3.2 Sensitivity of the fundamental diagram for the four-stageM/M/1

feedback threshold queue. . . . . . . . . . . . . . . . . . . . . 293.4 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 31

4 A tandem network of M/M/1 threshold queues 334.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

x Contents

4.2 A three queue tandem network of two-stage M/M/1 threshold queues 344.3 A three queue tandem network of four-stage M/M/1 feedback thresh-

old queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.4 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 42

II Queueing 45

5 Matrix analytic methods 475.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.2 Phase-Type distribution . . . . . . . . . . . . . . . . . . . . . . . . . 475.3 Markovian Arrival and Service Processes . . . . . . . . . . . . . . . . 505.4 Quasi-Birth-and-Death processes . . . . . . . . . . . . . . . . . . . . 52

5.4.1 Stationary distribution for a QBD . . . . . . . . . . . . . . . 535.4.2 Stationary distribution for a finite QBD . . . . . . . . . . . . 545.4.3 Fundamental matrix of a transient QBD . . . . . . . . . . . . 54

5.5 Level Dependent QBD processes . . . . . . . . . . . . . . . . . . . . 565.5.1 Stationary distribution for a LDQBD . . . . . . . . . . . . . 575.5.2 Stationary distribution for a finite LDQBD . . . . . . . . . . 585.5.3 Fundamental matrix of a transient LDQBD . . . . . . . . . . 58

6 The PH/PH/1 multi-threshold queue 616.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636.3 Steady-state analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 656.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.4.1 Extended traffic model . . . . . . . . . . . . . . . . . . . . . . 706.4.2 Le Ny and Tuffin [61] . . . . . . . . . . . . . . . . . . . . . . 716.4.3 Choi et al. [21] . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.5 Fundamental diagram of traffic . . . . . . . . . . . . . . . . . . . . . 746.6 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 76

7 A successive censoring algorithm for a system of connected LDQBD-processes 777.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787.3 Successive censoring algorithm . . . . . . . . . . . . . . . . . . . . . 82

7.3.1 Reduction step k . . . . . . . . . . . . . . . . . . . . . . . . . 847.3.2 Intermediate step . . . . . . . . . . . . . . . . . . . . . . . . . 907.3.3 Expansion step k . . . . . . . . . . . . . . . . . . . . . . . . . 917.3.4 Inverse of −Qk

k,k . . . . . . . . . . . . . . . . . . . . . . . . . 917.4 Simplified successive censoring algorithm . . . . . . . . . . . . . . . . 927.5 Complexity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.6 Demonstration of the successive censoring algorithm . . . . . . . . . 977.7 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 99

Contents xi

8 An iterative aggregation method for a queueing network withfinite buffers 1018.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1038.3 Iterative aggregation method . . . . . . . . . . . . . . . . . . . . . . 104

8.3.1 Aggregation of a Markovian Arrival Process . . . . . . . . . . 1048.3.2 Iterative procedure . . . . . . . . . . . . . . . . . . . . . . . . 1098.3.3 Marginal distribution . . . . . . . . . . . . . . . . . . . . . . 111

8.4 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1128.4.1 A tandem of M/M/1/N queues . . . . . . . . . . . . . . . . . 1138.4.2 A tandem of two-stage M/M/1/N threshold queues . . . . . 1178.4.3 A tandem of four-stage M/M/1/N feedback threshold queues 120

8.5 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 1308.A Sets and Entrance states . . . . . . . . . . . . . . . . . . . . . . . . . 132

8.A.1 A tandem of M/M/1/N queues . . . . . . . . . . . . . . . . . 1328.A.2 A tandem of two-stage M/M/1/N threshold queues . . . . . 1338.A.3 A tandem of four-stage M/M/1/N feedback threshold queues 135

Concluding remarks 139

Bibliography 141

Summary 149

Samenvatting 151

About the Author 155

xii Contents

CHAPTER 1

Introduction

1.1 Motivation

Few things unite cultures more than the frustration of sitting in aline of stationary traffic, with no discernible reason for the blockageand no end in sight.

The Economist, November 3rd 2014

Traffic jams are everywhere, some are caused by constructions or accidents butmost occur naturally. These “natural” traffic jams may be a result of variable drivingspeeds combined with a high number of vehicles. To prevent these traffic jams, wehave to understand traffic in general, and to understand traffic we have to understandthe relations between the three key parameters of highway traffic, speed, the averagespeed of a vehicle, flow, the number of vehicles passing a reference point, and density,the number of vehicles on the road. These relations are often displayed in the so-called fundamental diagram of traffic, see Figure 2.1 for an example. A well-knownrelation between these parameters is that flow equals the product of speed anddensity. This thesis demonstrates that queueing theory can offer new insights in theremaining relations between these three parameters.

An important aspect of traffic is congestion. Congestion is a phenomenon thatarises when roads become more and more crowded, speeds will decrease and traveltimes will increase. This phenomenon is well-known in queueing theory. In basicqueueing models we observe that the average queue length and average time spent inthe queue increases when the density increases. In the past, see Boon [14], queueingtheory has shown its usefulness in the analysis of intersections where vehicles mustwait before crossing, however, in the analysis of highway traffic, queueing theory hasreceived little attention.

Another aspect of highway traffic is its hysteretic behaviour, explained by Helbingin [46]. He describes traffic by two different phases, non-congested and congested,and describes a hysteretic transition between these two phases. On a quiet highway,traffic is non-congested and average speeds are close to the speed limit. However,when density increases, vehicles will interact with one another until, at some criticalpoint, a transition occurs and traffic becomes congested. In this phase, density ishigh and average speeds are well below the speed limit, i.e., a traffic jam emerges.

2 1. Introduction

This traffic jam is resolved when the number of vehicles on the road drops below acritical number and traffic becomes non-congested again. In general, it is assumedthat the two critical numbers are unequal.

In this thesis we model the hysteretic behaviour of traffic on a single highway sec-tion by a so-called threshold queue. The service and arrival rates of these thresholdqueues are controlled by a threshold policy based on the queue length, i.e., they arechanged when the number of customers in the queue reaches certain critical values.The analysis of these threshold queues results in a relation between the density ofthe queue and average time spent in the queueing system. With the help of Heide-mann’s model in [40] we are able to translate this to a relation between density andspeed for highway traffic.

1.2 Outline of the thesis

This thesis is organised in two parts. Part I consists of the Chapters 2, 3, and 4and discusses the application of queueing models in the analysis of highway traffic.Part II consists of the Chapters 5, 6, 7, and 8 and discusses the underlying queueingmodels in more detail.

Chapter 2 gives a broad overview of the literature on traffic models and providesthe basis for Part I. In this chapter we give a historical overview of the traffic modelsused to create the fundamental diagram of traffic. Furthermore, we give an overviewof the traffic models based on queueing theory. The traffic models in this thesisare based on Heidemann’s model which makes it possible to obtain the fundamentaldiagram from a queueing model.

In Chapter 3, we introduce two queueing models to model traffic on a singlehighway section, the two-stage threshold queue and the four-stage feedback thresh-old queue. The two-stage threshold queue models the hysteretic behaviour of trafficon a single section. However, the hysteretic behaviour is not restricted to a singlesection; once a certain highway section is jammed, it will eventually affect the pre-vious section. This feedback process is modelled by the four-stage feedback thresh-old queue. We obtain the fundamental diagram for each queue using Heidemann’smodel. Next, both queueing models are fitted such that their fundamental diagramsresemble that of traffic on a Danish highway. Finally, we present a sensitivity analy-sis on the system parameters in which we investigate the effects of small changes inparameters on the shape of the fundamental diagram. Chapters 2 and 3 are based on

[9] N. Baer, R.J. Boucherie, and J.C.W. van Ommeren. Threshold queueingdescribes the fundamental diagram of uninterrupted traffic. Memorandum 2000,Department of Applied Mathematics, University of Twente, Enschede, The Nether-lands, 2012.

In Chapter 4, we model a series of highway sections. We introduce a tandemnetwork of two-stage threshold queues and a tandem network of four-stage feedbackthreshold queues. These networks are analysed numerically and the fundamental

1.2 Outline of the thesis 3

diagram of each separate queue is obtained using Heidemann’s model. We close thischapter with a sensitivity analysis on the system parameters of the network.

Chapter 5 is an introductory chapter to Part II. It presents known results fromthe field of Matrix analytic methods on Phase-Type distributions, Markovian Ar-rival Processes and Markovian Service Processes, and both Level Dependent andLevel Independent Quasi-Birth-and-Death processes. The theory in this chapter isfrequently used in Chapters 6, 7, and 8.

The introduction of the two-stage threshold queue and the four-stage feedbackthreshold queue in Chapter 3 leads to the introduction of the PH/PH/1 multi-threshold queue in Chapter 6. While the queueing models in Chapter 3 are limitedto 2 or 4 stages and restricted to exponential distributions, the PH/PH/1 multi-threshold queue is a logical extension that relaxes both restrictions. In Chapter 6 weformally present the PH/PH/1 multi-threshold queue and we discuss why knownresults from Matrix analytic methods are not applicable. Next, we discuss how thestationary distribution of this queueing model can be obtained. Finally, we showthat the fundamental diagram for the PH/PH/1 multi-threshold can be obtainedand we perform a sensitivity analysis on the chosen Phase-Type distributions. Chap-ter 6 is based on

[10] N. Baer, R.J. Boucherie, and J.C.W. van Ommeren. The PH/PH/1 multi-threshold queue, volume 8499 of Lecture Notes in Computer Science, pages 95–109.Springer International Publishing, 2014.

The tandem networks discussed in Chapter 4 are special Markov chains that can-not be characterised as (Level Dependent) Quasi-Birth-and-Death processes. Theyturn out to be systems of connected Quasi-Birth-and-Death processes, which meansthat the Markov chain can be divided into subsets, each describing a Quasi-Birth-and-Death process. In Chapter 7 we extend this class of Markov chains and discuss asystem of connected Level Dependent Quasi-Birth-and-Death processes, in which theMarkov chain can be divided into subsets, each describing a Level Dependent Quasi-Birth-and-Death process. We provide a successive censoring algorithm to obtainthe stationary distribution of such a system and investigate the possible connectionsbetween different subsets. Chapters 4 and 7 are based on

[7] N. Baer, A. Al Hanbali, R.J. Boucherie, and J.C.W. van Ommeren. A suc-cessive censoring algorithm for a system of connected qbd-processes. Memorandum2030, Department of Applied Mathematics, University of Twente, Enschede, TheNetherlands, 2013.

The analysis of the tandem networks in Chapter 4 is computationally very de-manding causing the networks to be limited to three queues. In Chapter 8 we presentan iterative aggregation method which gives an approximation of a single queue in alarger tandem network. While focusing on a single queue in the network, the aggre-gation method aggregates all upstream network behaviour into a Markovian Arrival

4 1. Introduction

Process, and all downstream network behaviour into a Markovian Service Process.This is done in an iterative fashion, aggregating one queue in each iteration, until allupstream (or downstream) queues are aggregated. The resulting queueing model isthen analysed using results from the field of Matrix analytic methods. The iterativeaggregation method will be compared to simulation results of a tandem networkof two-stage threshold queues, a tandem network of four-stage feedback thresholdqueues, and a tandem network of M/M/1/k queues. Chapter 8 is based on

[8] N. Baer, A. Al Hanbali, R.J. Boucherie, and J.C.W. van Ommeren. Aniterative aggregation method for a tandem queueing network with finite buffers.Unpublished work, 2015.

Part I

Traffic

CHAPTER 2

Queueing models for highway traffic flows

2.1 Introduction

In 1935, Greenshields [37] captured the empirical relation between speed, flow anddensity for uninterrupted traffic in the fundamental diagram, see Figure 2.1. Math-ematical models for uninterrupted traffic have been developed and the fundamentaldiagram in its basic form is now well-understood, see e.g. Newell [79, 80, 81] for aconcise exposition.

Traffic jams are a major concern for highway operation and may occur in highdensity traffic due to variability in driving speed. A wide range of traffic modelshas been developed over the past decades. These models are mainly from statisticalphysics and non-linear dynamics, see [23, 46]. Congestion due to variable arrivaland/or service processes is the main topic of queueing theory, that, however, hashardly been invoked to analyse the fundamental concepts of uninterrupted trafficflows. Notable exceptions are the models introduced by Heidemann [41] and Jainand Smith [50]. However, these models do not capture the empirical shape of thefundamental diagram for modern traffic as shown in Figure 2.2.

Section 2.2 gives a brief overview of the literature on the fundamental diagramfor uninterrupted traffic flows and on queueing models for uninterrupted traffic flows.

2.2 Literature

Congestion is a key concept in queueing theory that models both the mesoscopic andmacroscopic effects of randomness on delay and sojourn times. In interrupted trafficflows, where queues arise naturally at an intersection, queueing theory has been apopular tool since the early 1940s, see Boon [14] for a recent survey. In uninterruptedtraffic, however, queueing models have received far less attention in literature.

In this section we first focus on the three levels of detail for traffic models. Next,we provide an overview of the results on the fundamental diagram in Section 2.2.2,starting from the literature surveys [83, 103, 104]. We close this section with anoverview on queueing models for uninterrupted traffic flows.

Throughout the section, and in the remainder of this thesis, we use the followingnotation. Let k denote the traffic density, v the mean speed of a vehicle, q the flow

8 2. Queueing models for highway traffic flows

0 0.2 0.4 0.6 0.8 10

6

12

18

24

30

Density (k)

Speed(v)

0 0.2 0.4 0.6 0.8 10

2

4

6

8

Density (k)

Flow

(q)

0 2 4 6 80

6

12

18

24

30

Flow (q)

Speed(v)

Figure 2.1: Fundamental diagram from the experimental data of Greenshields [37].

rate. By definition, these three parameters of traffic are related according to

q = k · v.

Furthermore, let vf denote the desired mean speed or free flow speed, and kjam thejam or maximum density.

2.2.1 Microscopic, mesoscopic and macroscopic models

Uninterrupted or highway traffic flow models can be characterised by their level ofdetail: microscopic, mesoscopic and macroscopic.

In microscopic models a high level of detail is used in which each individualdriver is characterised by its position and behaviour over time [47, 82]. In gen-eral, microscopic models lead to systems of (ordinary) differential equations [46].

2.2 Literature 9

0 50 100 1500

50

100

150

200

250

Density (k)

Flow

(q)

0 50 100 1500

2

4

6

8

10

Density (k)

Speed(v)

0 50 100 150 200 2500

2

4

6

8

10

Flow (q)

Speed(v)

Figure 2.2: Fundamental diagram from experimental data for modern traffic Sugiyama[93]. The flow-density diagram is fitted to the experimental data.

Well-known micropscopic models are the car-following model [15, 20], the cellularautomata model [75] and the lane-changing model [1].

In mesoscopic models the individual drivers are not distinguished [47, 82]. Thebehaviour of drivers is characterised in terms of the probability density f(x, v, t) ofvehicles at position x with speed v at time t. Examples of mesoscopic models areheadway distribution models [17] and gas-kinetic continuum models [87, 88].

Macroscopic models have the lowest level of detail and consider only three vari-ables for each position x and time t: average speed v(x, t), traffic flow q(x, t) andspatial vehicle density k(x, t), that are related as q(x, t) = v(x, t)·k(x, t). These threevariables are often presented in the fundamental diagram. Two classical examples ofmacroscopic models are the Lighthill-Whitham-Richards model [67, 68, 90] and thePayne model [84].


For more elaborate surveys on traffic models for uninterrupted traffic flows, see[12, 46, 47, 82, 100].

2.2.2 Fundamental diagram

The seminal work of Greenshields [37] has initiated the research to find a simpleformula capable of capturing the fundamental diagram of traffic flows. As can beseen in Figure 2.1, the fundamental diagram proposed by Greenshields has a linearspeed-density relationship given by a single formula, see Table 2.1. Greenshieldsmodel is an example of a single-regime traffic model. In multi-regime traffic modelsthe speed-density relationship is a piecewise function, depending on the regime of thehighway section, for instance free-flow or congested, see Table 2.2. These macroscopictraffic models all follow a key equation from

Below, we first consider single-regime models. Subsequently, we consider multi-regime models.

Single-regime traffic models

For an overview of single-regime traffic models that have been developed since Green-shields model, see Table 2.1 adapted from Wang et al. [103]. In this table we denoteby vc and kc the speed and density at capacity, i.e., when flow q equals the maximumflow qmax. Furthermore, let ωv denote the jam wave speed. Wang et al. [103] fitseveral single-regime models from Table 2.1 to empirical data. Greenshields modeldescribes a linear relationship between speed and density, causing a parabolic rela-tionship between flow and density. Several models were introduced to better capturerealistic traffic flows. The Greenberg model [36] proposes a logarithmic speed-densityrelationship based on fluid-flow analogies. This logarithmic approach performs wellunder congested conditions (v = 0 when k = kjam), but it does not satisfy boundaryconditions at low densities (v →∞ as k → 0). Underwood [94] proposes an exponen-tial speed-density relationship which obeys the boundary conditions at low densitiesbut does not satisfy boundary conditions under congested conditions (v → 0 fork → ∞). This limitation is also found in the Northwestern model [29] in whicha bell-shaped speed-density relationship was proposed. The Drew model [30] andPipes-Munjal model [74] resemble the simple formulation of Greenshields with theintroduction of an extra parameter n. This parameter n allows for extra degrees offreedom in fitting the models to empirical data. The general form of the Drew andPipes-Munjal model is captured by the Kuhne and Rodiger model [56]. Recently, anextension was made to the Greenshields model by Mahmassani et al. [72], creatingthe modified Greenshields model for single-regime. In this model, which resemblesboth the Drew and Pipes-Munjal model, a parameter v0 denoting the minimal speedat jam density was introduced, such that v = v0 when k = kjam. Based on generat-ing functions, Del Castillo and Benitez [27, 28] proposed a speed-density relationshipbased on the free flow speed vf , jam density kjam, and jam wave speed ωv. The au-thors note that their model can also be obtained from Newell’s car-following modelin [78]. The Van Aerde model [95] was obtained from the Van Aerde car-following

2.2 Literature 11

Model Function

Greenshields v = vf

(1− k

kjam

),

Greenberg v = vc ln(kjamk

),

Underwood v = vfe− kkc ,

Northwestern v = vfe− 1

2 ( kkc

)2

,

Drew v = vf

(1−

(k

kjam

)n+ 12

),

Pipes-Munjal v = vf

(1−

(k

kjam

)n),

Kuhne and Rodiger v = vf

(1−

(k

kjam

)a)b,

Modified Greenshields v = v0 + (vf − v0)((

1− kkjam

)n),

Newell v = vf

(1− e−

λvf

(1k− 1

kjam

)),

Del Castillo and Benitez v = vf

(1− e

|ωv|vf

(1− kjamk

)),

Van Aerde k = 1c1+

c2vf−v

+c3v,

MacNicholas v = vf

(knjam−knknjam+ckn

),

Wang v = v0 + (vf − v0)(

1 + ek−ktθ1

)−θ2.

Table 2.1: Single-regime speed-density relationships.

model and contains three extra parameters c1, c2 and c3. These parameters can beobtained by solving the boundary conditions, see [89]. It is shown by MacNicholas[71] that a model similar to the Van Aerde model can be obtained with fewer pa-rameters. A comparison between the MacNicholas and Van Aerde model is made in[71]. Recently, Wang et al. [104] proposed a logistic speed-density model based onfive parameters. An important parameter in their model is kt, the density at whichthe transition from free-flow to congested traffic occurs. The parameters θ1 and θ2

are used to fit the traffic model. For a more concise overview and visual comparisonof a selection of the models in Table 2.1, see [83, 103].

Multi-regime traffic models

In a multi-regime traffic model it is possible to use multiple single-regime modelsat the same time, one for each regime. This way, better fits compared to single-regime traffic models could be obtained. A drawback of multi-regime models liesin finding the point at which regimes change. Some multi-regime models occurringin literature are listed in Table 2.2. Since any combination of single-regime modelscan serve as a multi-regime model, this list is by no means extensive. The simplest


Model Function

Edie v =

vfe− kkc ,

c ln(kjamk

),

k ≤ kc,k > kc.

Triangular v =

vf ,vfkc

kc−kjam

(1− kjam

k

),

k ≤ kbp,k > kbp.

Modified Greenshields v =

vf ,

v0 + (vf − v0)(

kjam−kkjam−kbp

)α,

k ≤ kbp,k > kbp.

Modified Greenberg v =

vf ,

c ln(kjamk

),

k ≤ kbp,k > kbp.

Table 2.2: Multi-regime speed-density relationships.

multi-regime traffic model is the Triangular model, named after the triangular shapeof the flow-density plot, as used by Newell [80]. The corresponding speed-densityrelationship is given by the Triangular traffic model in Table 2.2. Here, kbp denotesthe break-point density, the density at which the regime changes. Similar speed-density relationships are obtained with the modified Greenberg model [29], and themodified Greenshields model [72]. All three models describe a constant speed whenk ≤ kbp after which the speed decreases to zero (or v0). The Edie model [31] is acombination of the Underwood model and the Greenberg model of Table 2.1. TheUnderwood model is used to describe traffic flows at low density (k ≤ kc) and theGreenberg model is used to describe traffic flows during congestion (k > kc). In [29]two more multi-regime models were introduced: one consisting of two regimes whilethe other has three regimes. Both models assume a linear speed-density relationduring the regimes and both models were fitted to empirical data in [29, 73].

2.2.3 Queueing theory in uninterrupted traffic flows

Two main queueing theoretic approaches can be identified to model uninterruptedtraffic: the queue with waiting room of Heidemann [41] and the queue with blockingof Jain and Smith [50].

Heidemann’s model

Heidemann [41] introduces anM/G/1 queueing system to model highway traffic. Theserver in the queueing system corresponds to a highway segment of length 1/kjam,which is the minimal part of the highway each vehicle requires. The mean servicetime in the queue is the average time it takes a vehicle in free flow traffic to crossthe segment: E[B] = 1/(kjam · vf ). The traffic density outside the chosen segment isk, so that the mean time between two arrivals is E[A] = 1/(k · vf ). The probability

2.2 Literature 13

0 50 100 150 200 2500

2500

5000

7500

10000

Density (k)

Flow

(q)

cs = 0.25cs = 0.50cs = 1.00cs = 2.00cs = 4.00

0 50 100 150 200 2500

20

40

60

80

100

120

140

Density (k)

Speed(v)

cs = 0.25cs = 0.50cs = 1.00cs = 2.00cs = 4.00

0 2500 5000 7500 100000

20

40

60

80

100

120

140

Flow (q)

Speed(v)

cs = 0.25cs = 0.50cs = 1.00cs = 2.00cs = 4.00

Figure 2.3: Fundamental diagram obtained with Heidemann’s M/G/1 queue,kjam = 200, vf = 120, and varying coefficient of variation cs.

of the server being busy is defined by

1− π0 =E[B]

E[A]=

k

kjam, (2.1)

where π0 denote the probability of the queue being empty.In the M/G/1 queue, an arriving vehicle may find the server busy upon arrival

and must wait for service. The total time required to cross the segment is the sojourntime, E[S], which is the sum of the waiting time and the service time. For the M/G/1queue the Pollaczek-Khintchine formula [105] gives

E[S] = E[B]

[1 +

ρ

1− ρ ·(1 + c2s)

2

],


0 250 500 750 10000

750

1500

2250

3000

Density (k)

Flow

(q)

ExponentialLinear

0 250 500 750 10000

15

30

45

60

Density (k)

Speed(v)

ExponentialLinear

0 750 1500 2250 30000

15

30

45

60

Flow (q)

Speed(v)

ExponentialLinear

Figure 2.4: Fundamental diagram obtained with Jain and Smith’s M/G/c/c queue fora linear and exponential decreasing speed, kjam = 200, vf = 55 mph.

where cs is the coefficient of variation of the service time. The speed, v, of a vehiclepassing the segment then is

v =1/kjamE[S]

. (2.2)

Figure 2.3 gives the fundamental diagram for the M/G/1 queue for various choicesof cs.

Generalisations of Heidemann’s model include the transient analysis of theM/G/1queue [42, 43, 44]. Vandaele et al. [102] and Van Woensel [98] consider the G/G/squeue. Validation of their queueing model [99, 101] shows that the M/G/1 queuemodels non-congested traffic and that the G/G/s is a more suitable model for con-gested traffic. Accidents were incorporated by Baykal-Gursoy et al. [11] in anM/MSP/c queue with service rates represented by a Markovian Service Process.

2.3 Concluding Remarks 15

Jain and Smith’s Model

An alternative approach to a queueing model for highway traffic is the M/G/c/cmodel of Jain and Smith [50], where an arrival that finds all servers occupied isblocked and cleared (lost). Their model is based on pedestrian flows in emer-gency evacuation planning [107]. The servers correspond to a road segment. Asthe M/G/c/c model does not incorporate waiting, the speed of a vehicle is obtainedby the service time that equals the sojourn time in the queue. The capacity C of aroad segment equals the number of vehicles that fit in this segment, i.e., the productof the jam density, kjam, the length of the road segment, L, and the number of lanes,N , C = kjam · L · N . The mean speed of a vehicle, Vn, depends on the number ofvehicles n on the road segment and is now a function that is input for the model.Two functions for Vn are considered [50, 107]

Vn =vfC

(C + 1− n) ,

that linearly decreases in the number of vehicles on the segment, and for suitableconstants γ and β, see [107],

Vn = vf · exp

[−(n− 1

β

)γ],

that exponentially decreases with the number of vehicles. In Figure 2.4 we presentthe fundamental diagram obtained with the M/G/c/c queue for both speed functionsVn.

A network of M/G/c/c queues was considered by Cruz et al. [25] and Cruz andSmith [24]. In this network a blocked customer will occupy its server until it isno longer blocked. Simulation techniques and approximations were used to deriveblocking probabilities, throughput, mean queue length and mean waiting times.

2.3 Concluding Remarks

The queueing models in literature result in a fundamental diagram similar to thefundamental diagram by Greenshields. However, these models do not capture thehysteresis effect as seen in modern-day traffic flows. In Chapter 3 we introducethe two-stage M/M/1 threshold queue and a four-stage M/m/1 feedback thresholdqueue which mimic the hysteresis effect and we show that they capture the resultingcapacity drop.


CHAPTER 3

Threshold queueing describes the

fundamental diagram of uninterrupted

traffic

3.1 Introduction

Chapter 2 gives an overview of queueing models in literature to analyse highwaytraffic flows. These queueing models do not capture the capacity drop, an importantaspect of modern traffic flows characterised by the sharp descent in the fundamentaldiagram, see Figure 3.1. Helbing [46] explains the capacity drop as the transitionfrom non-congested traffic to congested traffic. When the density of vehicles reachesa certain critical value, ρ2, traffic will become congested and the average speed issignificantly lower than in non-congested traffic. When density decreases again andreaches another critical value, ρ1 ≤ ρ2, a transition from congested traffic to non-congested traffic occurs and traffic flows recover. In the density interval [ρ1, ρ2]both congested and non-congested traffic flows exist, which indicates the existenceof hysteresis. As is shown in our numerical results, it is precisely this hysteresiseffect captured by the threshold queue introduced in this chapter that results in thecapacity drop in the fundamental diagram of Figure 3.1 observed in empirical datafor speed, flow and density.

In this chapter we discuss two special cases of the PH/PH/1 multi-thresholdqueue which is introduced in Chapter 6, namely the two-stage M/M/1 thresholdqueue and the four-stage M/M/1 feedback threshold queue. We model the hystereticbehaviour of traffic, which follows from Helbing [46], by adjusting the service ratesin the two-stage M/M/1 threshold queue. Since this hysteretic behaviour is notrestricted to a single section alone, we also introduce the four-stage M/M/1 feedbackthreshold queue. In this queueing model we adjust both the service rates and thearrival rates and model both the hysteretic behaviour of traffic on a single highwaysection, as well as the hysteretic interaction between a highway section and thepreceding section.

Both queueing systems have exponential service times and Poisson arrivals, andthe arrival and service rates are controlled by a threshold policy. Once the queuelength reaches certain upper thresholds, the arrival and/or service rates are adjusted.

18 3. Threshold queueing model for uninterrupted traffic

0 50 100 1500

50

100

150

200

250

Density (k)

Flow

(q)

0 50 100 1500

2

4

6

8

10

Density (k)

Speed(v)

0 50 100 150 200 2500

2

4

6

8

10

Flow (q)

Speed(v)

Figure 3.1: Fundamental diagram from experimental data for modern traffic Sugiyama[93]. The flow-density diagram is fitted to the experimental data.

They are changed to their original values once the queue length drops below thecorresponding lower threshold. We will modify Heidemann’s model [41], explainedin detail in Section 2.2.3, to obtain the fundamental diagram for both queueingmodels and give the best-fit, using the method of least squares on both the flow andon the speed, to empirical data for Danish highways [86].

The remainder of this chapter is divided into two sections, Section 3.2 dis-cusses the two-stage M/M/1 threshold queue, and the four-stage M/M/1 feedbackthreshold queue is discussed in Section 3.3.

3.2 Two-stage M/M/1 threshold queue 19

0 1 · · · L − 1 L · · · U

L · · · U U + 1 · · · N − 1 N

λ λ λ λ λ λ

λ

λ λ λ λ λ λ

µH µH µH µH µH µH

µL

µL µL µL µL µL µL

Figure 3.2: Transition Diagram for the two-stage M/M/1 threshold queue.

3.2 Two-stage M/M/1 threshold queue

Consider a single server queue with finite buffer N in which service rates are con-trolled by a threshold policy. Customers arrive according to a Poisson process withrate λ and require, upon service, an exponentially distributed service time, depend-ing on the stage of the queue. In stage 1, the non-congested stage, the service rate isµH , and in stage 2, the congested stage, the service rate is µL, with µL ≤ µH . Thetransition from the non-congested stage to the congested stage, and back, are con-trolled by a threshold. Once an arrival occurs while the queue length is U , the stagechanges from non-congested to congested. The stage changes back from congestedto non-congested when the queue length is L and a departure occurs. The statespace and transition rates of this threshold queue are depicted in Figure 3.2. Thestationary queue length probabilities π for the two-stage M/M/1 threshold queuecan readily be obtained from standard Markov chain analysis, see also Le Ny andTuffin [61]. Let ρ = λ

µHand δ = λ

µLand π(i, j) the probability of having i customers

in the queue in stage j. Then

π(i, 1) = π(0, 1) ρi, i = 1, . . . , L− 1,

π(i, 1) = π(0, 1)ρi − ρU+1

1− ρU−L+2, i = L, . . . , U, (3.1)

π(i, 2) = π(0, 1)δ − δi−L+2

1− δρU − ρU+1

1− ρU−L+2, i = L, . . . , U,

π(i, 2) = π(0, 1)δi−U − δi−L+2

1− δρU − ρU+1

1− ρU−L+2, i = U + 1, . . . , N,

with π(0, 1) such that [U∑i=0

π(i, 1) +

N∑i=L

π(i, 2)

]= 1. (3.2)

The mean sojourn time is then given by

E[S] =1

Λ

[U∑i=0

iπ(i, 1) +

N∑i=L

iπ(i, 2)

], (3.3)


where Λ is the effective arrival rate to the queue

Λ = λ

[U∑i=0

π(i, 1) +

N−1∑i=L

π(i, 2)

].

Remark 3.1 (Infinite buffer). Note that if we would consider a two-stage M/M/1threshold queue with an infinite buffer, i.e., N = ∞, the stationary distribution in(3.1) still holds but with π(0, 1) such that[

U∑i=0

π(i, 1) +

∞∑i=L

π(i, 2)

]= 1.

The mean sojourn time is

E[S] =1

Λ

[U∑i=0

iπ(i, 1) +

∞∑i=L

iπ(i, 2)

],

and effective arrival rate isΛ = λ.

In order to use the threshold queue with a finite buffer as well as with an infinitebuffer we slighty alter Heidemann’s model. Recall from (2.1) and (2.2)

k = (1− π(0, 1)) kjam, v =1/kjamE[S]

, q = k · v =1− π(0, 1)

E[S]. (3.4)

Note that these three parameters are completely determined by the system variablesλ, µL, µH , L, U , and N . They can therefore be interpreted as functions of thesesystem parameters. In the two-stage M/M/1 threshold queue with an infinite buffer,k → kjam when the queue approaches instability, i.e., λ → µL. To obtain the sameresult for the two-stage M/M/1 threshold queue with a finite buffer, we adjustHeidemann’s model and define

k = (1− π(0, 1))C, v =1/C

E[S], q = k · v =

1− π(0, 1)

E[S]. (3.5)

Here, we use 1/C instead of 1/kjam as in Heidemann’s model. The parameter C isobtained by fitting the queueing model to empirical data. In this adjusted model wedefine

kjam = limλ→µL

k.

We may now obtain the capacity, qmax, the critical density, k∗, and the jam wavespeed, ωv, as follows. Let us denote by λ∗ the value of λ, with 0 ≤ λ ≤ µL, for whichthe maximum flow, qmax, is obtained, i.e., the value λ = λ∗ for which

∂

∂λq =

1

E[S]

∂

∂λ(1− π(0, 1)) + (1− π(0, 1))

∂

∂λ

1

E[S]

= 0.

3.2 Two-stage M/M/1 threshold queue 21

From (3.5), we now obtain

qmax =1− π(0, 1)

E[S]

∣∣∣∣λ=λ∗

,

k∗ = C (1− π(0, 1))|λ=λ∗ , (3.6)

where |λ=λ∗ indicates that we insert λ = λ∗ in these expressions.Finally, we determine the jam wave speed, ωv,

ωv = limk→kjam

∂

∂kq = lim

λ→µL

∂∂λq∂∂λk

= limλ→µL

1

E[S]∂∂λ (1− π(0, 1)) + (1− π(0, 1)) ∂

∂λ1

E[S]

C ∂∂λ (1− π(0, 1))

.

(3.7)

The above expressions can readily be evaluated numerically for the M/M/1 thresholdqueue. In Section 3.2.1 we fit the two-stage M/M/1 threshold queue to experimen-tal data obtained for Danish highways [86] and we determine the capacity, criticaldensity and jam wave speed.

Remark 3.2 (Multi-lane traffic). Our model also captures multi-lane traffic. On amulti-lane highway, vehicles switch from high density lanes to low density lanes toimprove their driving speed. To maintain safe driving distance other vehicles decel-erate and the average speed decreases. Using Helbing’s description of the hysteretictransition from non-congested to congested traffic we distinguish three density re-gions: light, medium and heavy traffic. In light traffic (k ≤ ρ1) vehicles will switchlanes but they will not greatly affect other vehicles. In heavy traffic (k ≥ ρ2) vehi-cles will not be able to switch lanes due to high densities on all lanes. In mediumtraffic (ρ1 < k < ρ2) vehicles will switch lanes causing average speed to decreaseand density to increase. Once the density reaches ρ2 the system becomes congestedand the average speed is considerably lower than in non-congested traffic. Whenwe aggregate all lanes the hysteresis in this multi-lane system resembles that in oursingle server threshold queue.

3.2.1 Model validation

In Figure 3.3, the two-stage M/M/1 threshold queue is fitted to empirical datapoints obtained from measurements made in September and October 2013 on threedifferent locations on the Helsingørmotorvejen (two-lane) in Denmark made availableby DTU Transport, Denmark [86]. DTU Transport provided three data sets withmeasurements on different time intervals: hourly measurements near Hørsholm, 15-minute measurements near Sandbjerg, and 5-minute measurements near Kokkedal.These data sets were uniformised to represent hourly measurements on all threelocations. The data points in Figure 3.3(a) refer to the measurements near Hørsholm,the data points in Figure 3.3(b) refer to the measurements near Sandbjerg, and thedata points Figure 3.3(c) refer to the measurements near Kokkedal. The curves ineach subfigure show the best-fit using the two-stage M/M/1 threshold queue for


0 40 80 120 160 2000

1,000

2,000

3,000

4,000

Density (k)

Flo

w(q

)

(a)

HørsholmN = 10N = 20N = ∞

0 50 100 150 200 250 3000

1,000

2,000

3,000

4,000

Density (k)F

low

(q)

(b)

SandbjergN = 10N = 20N = ∞

0 50 100 150 200 250 3000

1,000

2,000

3,000

4,000

Density (k)

Flo

w(q

)

(c)

KokkedalN = 10N = 20N = ∞

Figure 3.3: Flow-Density diagram obtained with a fitted two-stage M/M/1 thresholdqueue. The flow-density curves were fitted to empirical flow-density points of a Danishmotorway.

three different buffer sizes, N = 10, N = 20, and N =∞. The parameters for thesecurves, as well as the jam density, kjam, the capacity, qmax, critical density, k∗ andjam wave speed, ωv, are given in Table 3.1. We show that the qualitative behaviourof the two-stage M/M/1 threshold queue captures the behaviour of the fundamentaldiagram of highway traffic.

Figure 3.3 and Table 3.1 show that both the jam density, kjam, and jam wavespeed, ωv, can be regulated by the buffer size N . Decreasing the buffer size resultsin a decreasing jam density and decreasing jam wave speed.

3.2.2 Sensitivity analysis of the fundamental diagram for thetwo-stage M/M/1 threshold queue

Figure 3.4 characterises the fundamental diagram of the two-stage M/M/1 thresholdqueue for four different scenarios. Each scenario is based on the basic scenario withC = 1, µH = 25, µL = 15, L = 15 and U = 10 but one of the four parameters ismodified: (a) the lower threshold, L, (b) the upper threshold, U , (c) the high servicerate, µH and (d) the low service rate, µL. C is kept unchanged since it is merely ascaling constant.

The effects of modifying L are minimal, as can be seen in Figure 3.4(a). Fig-ure 3.4(b) shows that the steepness of the capacity drop increases by increasingU . Figures 3.4(c) shows that qmax increases and k∗ decreases when µH increases.Increasing µL results in increasing both qmax and k∗ as is shown in Figure 3.4(d).

3.3 Four-stage M/M/1 feedback threshold queue

Consider a single server queue with finite buffer N , exponential service times andPoisson arrivals. The arrival rates and service rates are stage-dependent and con-trolled by a threshold policy. This threshold policy determines the stage of the queue

3.3 Four-stage M/M/1 feedback threshold queue 23

(a)

Hør

sholm

(b)

San

db

jerg

(c)

Kokked

al

N10

20∞

1020

∞10

20

∞

L1

11

22

21

11

U3

33

22

22

22

µH

2619

0.13

213

01.4

220

984.

6210

3728

.36

55584.0

051689.5

186078.5

550447.6

547969.2

9

µL

5896

.21

6899

.43

697

0.39

6160

.00

7605.4

97775.9

35719.2

36701.1

06782.5

2

C23

4.80

187

.81

184.

7511

46.8

5598.7

8554.4

6938.0

1538.2

3510.4

6

kjam

86.3

4128

.25

184.

75122

.24

259.0

7554.4

6144.1

6271.8

2510.4

6

q max

2635

.07

2652

.65

265

3.70

2882

.80

2922.1

32923.2

82503.8

92504.7

72500.6

0

k∗

35.8

337

.20

37.

3648

.08

51.6

152.1

942.8

345.5

945.8

9

ωv

-14.

18-9

.17

-3.3

7-8

.38

-2.3

0-0

.53

-4.4

9-1

.95

-0.6

5

Table

3.1

:P

ara

met

ers

for

the

two-s

tageM/M

/1

thre

shold

queu

esin

Fig

ure

3.3

.


0 0.2 0.4 0.6 0.8 10

2

4

6

8

Density (k)

Flow

(q)

(a)

L = 0L = 3L = 6L = 9

0 0.2 0.4 0.6 0.8 10

2

4

6

8

Density (k)

Flow

(q)

(b)

U = 6U = 10U = 15U = 20

0 0.2 0.4 0.6 0.8 10

2

4

6

8

Density (k)

Flow

(q)

(d)

µL = 12µL = 16µL = 20µL = 24

0 0.2 0.4 0.6 0.8 10

2

4

6

8

Density (k)

Flow

(q)

(c)

µH = 16µH = 20µH = 24µH = 28

Figure 3.4: Flow-Density diagram for the two-stage M/M/1 threshold queue withvarying (a) L, (b) U , (c) µH and (d) µL.

based on its queue length. Let (n, s) denote the state of this Markov chain in whichn, with n = 0, . . . , N , and s, with s = 1, . . . , 4, denote the queue length and stage,respectively, of the queue. If the queue is in stage s and a departure or an arrivalcauses the queue length to drop below Ls or to exceed Us, the stage of the queuechanges. These changes are depicted in the state diagram in Figure 3.5. In thissection we assume

L1 = 0, U1 = U3 = Uµ,

L2 = L4 = Lµ, U2 = Uλ,

L3 = Lλ, U4 = N,


and

0 < Lλ < Lµ ≤ Uµ < Uλ < N.

The stage-dependent service and arrival rates are given in Table 3.2. Finally, wedefine κ to be a constant such that

λLµL

= κλHµH

. (3.8)

The stationary queue length probabilities π for the four-stage M/M/1 feedback

Stage Arrival Rate Service Rate

s = 1 λH µH

s = 2 λH µL

s = 3 λL µH

s = 4 λL µL

Table 3.2: The stage dependent service and arrival rates for the four-stage M/M/1feedback threshold queue.

threshold queue can readily be obtained from standard Markov chain analysis. Let

α =λHµH

, β =λHµL

, γ =λLµH

, δ =λLµL

,

and let π(i, j) be the probability of having i customers in the queue in stage j, then:

π(i, 1) =

π(0, 1) αi,

π(Lλ − 1, 1)αi+1(1+Z)−αLµZαLλ (1+Z)−αLµZ ,

π(Lµ − 1, 1)αi+1−αUµ+2

αLµ−αUµ+2 ,

π(i, 2) =

π(Uµ + 1, 2) βLµ−βi+1

βLµ−βUµ+2 ,

π(Uµ, 1)(βLµ−Uµ−β2)(βi−βUλ+1)

(1−β)(βLµ−βUλ+2),

(3.9)

π(i, 3) =

π(Uλ, 2)α(1−γi+1−Lλ)

1−γ ,

π(Lµ − 1, 3)γi+1−γUµ+2

γLµ−γUµ+2 ,


01

···Lλ

−1

Lλ

···Lµ

−1

Lµ

···Uµ

Lµ

···Uµ

Uµ

+1

···Uλ

Lλ

···Lµ

−1

Lµ

···Uλ

Lµ

···Uµ

Uµ

+1

···Uλ

Uλ

+1

···N

−1

N

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λH

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

λL

µH

µH

µH

µH

µH

µH

µH

µH

µH

µL

µL

µL

µL

µL

µL

µH

µH

µH

µH

µH

µH

µL

µL

µL

µL

µL

µL

µL

µL

µL

µL

Fig

ure

3.5

:T

ransitio

ndia

gra

mfo

rth

efo

ur-sta

geM/M

/1

feedback

thresh

old

queu

e.


π(i, 4) =

π(Uµ + 1, 4) δLµ−δi+1

δLµ−δUµ+2 ,

π(Uλ, 2)β

[(γUµ+2−Lµ−γUµ+2−Lλ)(δi−Uµ−1−δi+1−Lµ)

(1−γUµ+2−Lµ)(1−δ)

+(1−γUµ+2−Lµ)(1−δi+1−Lµ)

(1−γUµ+2−Lµ)(1−δ)

],

π(Uλ, 2)βδi−1−Uλ[

(γUµ+2−Lµ−γUµ+2−Lλ)(δUλ−Uµ−δUλ+2−Lµ)(1−γUµ+2−Lµ)(1−δ)

+(1−γUµ+2−Lµ)(1−δUλ+2−Lµ)

(1−γUµ+2−Lµ)(1−δ)

],

with

Z =αUµ+2−LµβUλ−Uµ

(1− βUµ+2−Lµ)

(1− αUµ+2−Lµ) (1− βUλ+2−Lµ),

and π(0, 1) such that Uµ∑i=0

π(i, 1) +

Uλ∑i=Lµ

π(i, 2) +

Uµ∑i=Lλ

π(i, 3) +

N∑i=Lµ

π(i, 4)

= 1.

The mean sojourn time is given by

E[S] =1

Λ

Uµ∑i=0

iπ(i, 1) +

Uλ∑i=Lµ

iπ(i, 2) +

Uµ∑i=Lλ

iπ(i, 3) +

N∑i=Lµ

iπ(i, 4)

,where Λ is the effective arrival rate to the queue

Λ = λH

Uµ∑i=0

π(i, 1) +

Uλ∑i=Lµ

π(i, 2)

+ λL

Uµ∑i=Lλ

π(i, 3) +

N−1∑i=Lµ

π(i, 4)

.Remark 3.3 (Infinite buffer). Note that if we would consider a four-stage M/M/1feedback threshold queue with an infinite buffer, i.e., N =∞, the stationary distri-bution in (3.9) would still hold but with π(0, 1) such that Uµ∑

i=0

π(i, 1) +

Uλ∑i=Lµ

π(i, 2) +

Uµ∑i=Lλ

π(i, 3) +

∞∑i=Lµ

π(i, 4)

= 1.

The mean sojourn time is

E[S] =1

Λ

Uµ∑i=0

iπ(i, 1) +

Uλ∑i=Lµ

iπ(i, 2) +

Uµ∑i=Lλ

iπ(i, 3) +

∞∑i=Lµ

iπ(i, 4)

,and effective arrival rate is

Λ = λH

Uµ∑i=0

π(i, 1) +

Uλ∑i=Lµ

π(i, 2)

+ λL

Uµ∑i=Lλ

π(i, 3) +

∞∑i=Lµ

π(i, 4)

.


0 150 300 450 600 7500

1,000

2,000

3,000

4,000

Density (k)

Flo

w(q

)

(a)

HørsholmN = ∞

0 150 300 450 600 750 9000

1,000

2,000

3,000

4,000

Density (k)F

low

(q)

(b)

SandbjergN = ∞

0 50 100 150 200 250 3000

1,000

2,000

3,000

4,000

Density (k)

Flo

w(q

)

(c)

KokkedalN = ∞

Figure 3.6: Flow-Density diagram obtained with a fitted four-stage M/M/1 feedbackthreshold queue. The flow-density curve was fitted to empirical flow-density points ofa Danish motorway.

We obtain the speed-flow-density relations, and thus the fundamental diagram,using the result from the altered Heidemann’s model in (3.5). To this end, we vary0 ≤ λH < µH

κ such that λLµL

< 1, see (3.8), and such that the four-stage M/M/1feedback threshold queue with infinite buffer is stable. We obtain the capacity,qmax, critical density, k∗, and jam wave speed, ωv, following equations (3.6) and(3.7). These expressions can readily be evaluated numerically for the four-stageM/M/1 feedback threshold queue. In Section 3.3.1 we fit the four-stage M/M/1feedback threshold queue to experimental data obtained for Danish highways andwe determine the capacity, critical density and jam wave speed.

3.3.1 Model validation

In Figure 3.6, the four-stage M/M/1 threshold queue is fitted to empirical datapoints obtained from measurements made in September and October 2013 on threedifferent locations on the Helsingørmotorvej (two-lane) in Denmark made availableby DTU Transport, Denmark [86]. DTU Transport provided three data sets withmeasurements on different time intervals: hourly measurements near Hørsholm, 15-minute measurements near Sandbjerg, and 5-minute measurements near Kokkedal.These data sets were uniformised to represent hourly measurements on all three loca-tions. The data points in Figure 3.6(a) refer to hourly measurements near Hørsholm,the data points in Figure 3.6(b) refer to 15-minute measurements near Sandbjerg,and the data points Figure 3.6(c) refer to 5-minute measurements near Kokkedal.The curves in each subfigure show the best-fit using the four-stage M/M/1 feedbackthreshold queue. The parameters for these curves, as well as the capacity, qmax,critical density, k∗ and jam wave speed, ωv, are given in Table 3.3. In the sensitivityanalysis in Section 3.3.2 we show that the effects of the buffer size N and the con-stant κ, which links λL to λH , see (3.8), are minimal. Therefore, we assume herethat N =∞ and κ = 1.


(a) Hørsholm (b) Sandbjerg (c) Kokkedal

Lλ 1 1 1

Lµ 3 5 2

Uµ 5 5 5

Uλ 6 6 6

µH 74951.98 78651.14 25834.87

µL 1.00 1.00 86.88

C 702.97 891.57 278.83

kjam 702.97 891.57 278.83

qmax 2598.17 2849.88 2500.57

k∗ 34.39 44.86 41.18

ωv -88.29 -43.96 -92.03

Table 3.3: Parameters for the three four-stage M/M/1 feedback threshold queues inFigure 3.6.

3.3.2 Sensitivity of the fundamental diagram for the four-stage M/M/1 feedback threshold queue.

Figures 3.7, 3.8, and 3.9, characterise the fundamental diagram of the four-stageM/M/1 feedback threshold queue for nine different scenarios. Each scenario is basedon the basic scenario with

Lλ = 3, Uµ = 9, µH = 25000,

Lµ = 6, Uλ = 12, µL = 10,

C = 400, N =∞, κ = 1,

which is based on the results in Section 3.3.1.In Figure 3.7 we modify: (a) the lower threshold Lλ, (b) the lower threshold

Lµ, (c) the upper threshold Uµ, and (d) the upper threshold Uλ while keeping otherparameters fixed. The effects of modifying Lλ or Lµ are minimal, as can be seenin Figure 3.7(a) and Figure 3.7(b). Figure 3.7(c) shows that modifying Uµ greatlyaffects the fundamental diagram. Increasing Uµ results in a greater qmax and k∗.The same effect is achieved by increasing Uλ but to a lesser extent as is shown inFigure 3.7(d).

In Figure 3.8 we modify: (a) the high service rate µH , (b) the low service rate


0 100 200 300 4000

1,000

2,000

3,000

4,000

5,000

Density (k)

Flow

(q)

(a)

Lλ = 1Lλ = 3Lλ = 5

0 100 200 300 4000

1,000

2,000

3,000

4,000

5,000

Density (k)

Flow

(q)

(b)

Lµ = 4Lµ = 6Lµ = 8

0 100 200 300 4000

1,000

2,000

3,000

4,000

5,000

Density (k)

Flow

(q)

(c)

Uµ = 7Uµ = 9Uµ = 11

0 100 200 300 4000

1,000

2,000

3,000

4,000

5,000

Density (k)

Flow

(q)

(d)

Uλ = 10Uλ = 12Uλ = 14

Figure 3.7: Flow-Density diagram for the four-stage M/M/1 feedback threshold queuewith varying (a) Lλ, (b) Lµ, (c) Uµ and (d) Uλ.

µL, (c) the constant C, and (d) the buffer size N while keeping other parametersfixed. The latter has little effect as is shown in Figure 3.8(d) in which N is mod-ified. Figure 3.8(c) shows that C serves as a scaling parameter for the density. InFigure 3.8(a) we modify µH and it shows that by increasing µH , we increase qmaxwhile keeping k∗ unchanged. Finally, Figure 3.8(b) shows that both qmax and k∗ areincreased by increasing µL.

In Figure 3.9 we modify the parameter κ. Figure 3.9 shows that both qmax andk∗ are slightly increased for increasing κ.

3.4 Summary and Conclusion 31

0 100 200 300 4000

1,500

3,000

4,500

6,000

Density (k)

Flow

(q)

(a)

µH = 15000µH = 25000µH = 35000

0 100 200 300 4000

1,500

3,000

4,500

6,000

Density (k)

Flow

(q)

(b)

µL = 1µL = 10µL = 100µL = 10000

0 200 400 6000

1,000

2,000

3,000

4,000

Density (k)

Flow

(q)

(c)

C = 200C = 400C = 600

0 100 200 300 4000

1,000

2,000

3,000

4,000

Density (k)

Flow

(q)

(d)

N = 15N = 20N = ∞

Figure 3.8: Flow-Density diagram for the four-stage M/M/1 feedback threshold queuewith varying (a) µH , (b) µL, (c) C and (d) N .

3.4 Summary and Conclusion

In this chapter we have introduced the two-stage M/M/1 threshold queue and thefour-stage M/M/1 feedback threshold queue to study the parameters of traffic thatinfluence the shape of the fundamental diagram including the capacity drop in thisdiagram observed in empirical data for modern traffic flows.

The two stage M/M/1 threshold queue has two service regimes: high and lowservice rates, and switches from high rates to low rates when the queue lengthexceeds the upper threshold, and returns to high rates when the queue length fallsbelow the lower threshold, where the lower threshold is smaller than, or equal to,


0 100 200 300 4000

1,000

2,000

3,000

4,000

Density (k)

Flow

(q)

κ = 0.5κ = 1.0κ = 1.5

Figure 3.9: Flow-Density diagram for the four-stage M/M/1 feedback threshold queuewith varying κ.

the upper threshold. The two-stage M/M/1 threshold queue was successfully fittedto experimental data and the capacity, critical density and jam wave speed werecomputed for three different data sets.

The four-stage M/M/1 feedback threshold queue has four regimes with eithera high or low arrival rate, and either a high or low service rate. The service rateswitches from high rates to low rates when the queue length exceeds the upperthreshold Uµ, and returns to high rates when the queue length falls below the lowerthreshold Lµ. Similarly, the arrival rate switches from high rates to low rates whenthe queue length exceeds the upper threshold Uλ, and returns to high rates when thequeue length falls below the lower threshold Lλ, where Lλ < Lµ ≤ Uµ < Uλ. Thefour-stage M/M/1 feedback threshold queue was successfully fitted to experimentaldata and the capacity, critical density and jam wave speed were computed for threedifferent data sets.

Comparing the three different fits of each queueing model we can conclude thatthe four-stage M/M/1 feedback threshold queue gives a better fit to experimentaldata. Sensitivity analysis of the two-stage M/M/1 threshold queue reveals thatsteepness of the capacity drop, the capacity and the critical density are determinedby the value of the higher threshold, U , and the mean service times, µL and µH .Sensitivity analysis of the four-stage feedback threshold queue reveals that the shapeof the fundamental diagram is determined by the upper threshold Uµ and the highand low service rates.

CHAPTER 4

A tandem network of M/M/1 threshold

queues

4.1 Introduction

In Chapter 3 we have introduced and analysed the two-stage M/M/1 thresholdqueue and the four-stage M/M/1 feedback threshold queue. These queueing systemswere controlled by a threshold policy based on the queue length of the queue. Inthe two-stage M/M/1 threshold queue only the service rates changed when thequeue length reached certain thresholds while in the four-stage M/M/1 feedbackthreshold queue, both the arrival rates and service rates changed. Both queueingmodels modelled a single highway section. The two-stage M/M/1 threshold queuemodelled the hysteric behaviour of traffic on the highway section while the four-stageM/M/1 feedback threshold queue, in addition, also models the hysteretic behaviourof arriving traffic, i.e., traffic on the preceding highway section. Finally, an adjustedversion of Heidemann’s method was used to create the fundamental diagram ofuninterrupted traffic for both queueing systems.

In this chapter we extend the results from Chapter 3 and model a sequence ofhighway sections by a tandem queueing network. In a tandem queueing network, acustomer is served at multiple queueing systems in a line, see Figure 4.1 for a tandemqueueing network of 5 queues. Throughout this chapter we will assume Poissonarrivals to the first queue and exponentially distributed service rates. Furthermore,the buffer at each queue is finite and we assume a Blocking Before Service (BBS)protocol, i.e., a server becomes blocked, and is unable to serve customers, if itsdeparting customer fills the downstream queue. Due to the exponential growth ofthe state space, we are limited to tandem networks of three queues. In Chapter 8we introduce an iterative aggregation method enabling the approximative analysisof larger tandem networks

In Section 4.2 we introduce a three queue tandem network of the two-stageM/M/1 threshold queues in which the service rates are controlled by a thresholdpolicy based on the queue length at each station. Each queue in the network modelsthe hysteretic behaviour of traffic on a highway section. We extend this tandemnetwork of two-stage M/M/1 threshold queues to a 3-tandem network of four-stageM/M/1 feedback threshold queues in Section 4.3. This network models the hysteric

34 4. A tandem network of M/M/1 threshold queues

N1 µ1 N2 µ2 N3 µ3 N4 µ4 N5 µ5λ

Queue 1 Queue 2 Queue 3 Queue 4 Queue 5

Figure 4.1: A tandem network with 5 queues.

0 1 · · · L − 1 L · · · U

L · · · U U + 1 · · · N − 1 N


µL


Figure 4.2: Transition Diagram for a single two-stage M/M/1 threshold queue.

behaviour of each highway section, as well as the hysteric interaction between twoconsecutive sections, i.e., once a highway section becomes incredibly crowded, thevehicles in the preceding section will drive slower. Both tandem networks are anal-ysed numerically and the fundamental diagram is obtained for each seperate queueusing the adjusted Heidemann’s method in equation (3.5). Finally, we perform asensitivity analysis on the system parameters for both tandem networks. Section 4.4gives concluding remarks.

4.2 A three queue tandem network of two-stageM/M/1 threshold queues

We consider a tandem queueing network of three identical two-stage M/M/1 thresh-old queues with finite buffer N , with exponential service times at each queue, andwith Poisson arrivals with rate λ to the first queue. Each queue is controlled by athreshold policy which determines the stage of the queue based on the queue length.Let (n1, s1, n2, s2, n3, s3) denote the states of this Markov chain in which ni, withni = 0, . . . , N , and si, with si = 1, 2, denote the queue length and stage, respectively,of queue i. The stage of queue i becomes congested, denoted by si = 2, once thequeue length grows beyond the upper threshold, U , and it becomes non-congestedagain, denoted by si = 1, if the queue length drops below the lower threshold, L, asdepicted in Figure 4.2. In this section we assume

0 < L ≤ U < N.

The arrival rate to the first queue is λ but the service rate in each queue depend onthe stage and is given in Table 4.1. Let Q be the generator of this Markov chain andlet π be its stationary distribution such that πQ = 0 for πe = 1. By modelling this

4.2 Two-stage M/M/1 tandem threshold queue 35

Stage Service Rate

si = 1 µH

si = 2 µL

Table 4.1: The stage dependent service rates for queue i, for i = 1, 2, 3.

Markov chain as a finite Level Dependent Quasi-Birth-and-Death process (LDQBD)we can obtain the stationary distribution using the Successive Censoring Algorithmfrom Chapter 7.

Let the element π(n1, s1, n2, s2, n3, s3) of π be the (stationary) probability thatthe Markov chain is in state (n1, s1, n2, s2, n3, s3). Furthermore, let πi be themarginal queue length distribution of queue i, i.e.,

π1(n1) =

2∑s1=1

2∑s2=1

2∑s3=1

N∑n2=0

N∑n3=0

π(n1, s1, n2, s2, n3, s3),

π2(n2) =

2∑s1=1

2∑s2=1

2∑s3=1

N∑n1=0

N∑n3=0

π(n1, s1, n2, s2, n3, s3), (4.1)

π3(n3) =

2∑s1=1

2∑s2=1

2∑s3=1

N∑n1=0

N∑n2=0

π(n1, s1, n2, s2, n3, s3).

The mean queue length of queue i, E[Li], is now given by

E[Li] =

N∑ni=0

niπi(ni), (4.2)

and the mean sojourn time in queue i, E[Si], is obtained using Little’s Law [69]

E[Si] =E[Li]

Λ, (4.3)

where Λ is the effective arrival rate to the tandem network

Λ = (1− π1(0))λ. (4.4)

Finally, we apply the adjusted Heidemann’s method from equation (3.5) to obtainthe fundamental diagram for each highway section. Recall from (3.5) the definitionof the traffic parameters for density, k, speed, v, and flow, q, in terms of the marginalqueue length distribution and mean sojourn time for each queue i, i = 1, 2, 3,

ki = (1− πi(0))Ci, vi =1/CiE[Si]

, qi = ki · vi =1− πi(0)

E[Si]. (4.5)


(a)

(b)

(c)

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

L = 2L = 3L = 4

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

L = 2L = 3L = 4

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 3

L = 2L = 3L = 4

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

U = 5U = 6U = 7

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

U = 5U = 6U = 7

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)Flow

(q)

Queue 3

U = 5U = 6U = 7

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

N = 8N = 10N = 15

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

N = 8N = 10N = 15

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 3

N = 8N = 10N = 15

Figure 4.3: Flow-Density diagram for a tandem network of three two-stage M/M/1threshold queue with varying (a) L, (b) U , and (c) N .

In addition, we denote the capacity of the highway section, i.e., the maximum flowthat can be achieved, by qmax. The density at which this capacity is achieved iscalled the critical density and is denoted by k∗. Finally, we denote by kjam the jamdensity.

Figures 4.3 and 4.4 characterise the fundamental diagram for each queue in atandem network of three two-stage M/M/1 threshold queues for different scenarios.The results for each of the three scenarios are displayed in three horizontal plots,characterising the fundamental diagram for the first queue (left), the second queue

4.2 Two-stage M/M/1 tandem threshold queue 37

(a)

(b)

(c)

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

µL = 15µL = 20µL = 25

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

µL = 15µL = 20µL = 25

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 3

µL = 15µL = 20µL = 25

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

µH = 25µH = 30µH = 35

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

µH = 25µH = 30µH = 35

0 0.25 0.5 0.75 10

2

4

6

8

10

Density (k)Flow

(q)

Queue 3

µH = 25µH = 30µH = 35

0 1 2 30

2

4

6

8

10

Density (k)

Flow

(q)

Queue 1

C = 1C = 2C = 3

0 1 2 30

2

4

6

8

10

Density (k)

Flow

(q)

Queue 2

C = 1C = 2C = 3

0 1 2 30

2

4

6

8

10

Density (k)

Flow

(q)

Queue 3

C = 1C = 2C = 3

Figure 4.4: Flow-Density diagram for a tandem network of three two-stage M/M/1threshold queue with varying (a) µL, (b) µH , and (c) C.

(middle) and the third queue (right) in the tandem network. Each scenario is basedon the basic scenario with

L = 3, U = 5, N = 8, µL = 20, µH = 30, C = 1.

In each of the six scenarios in Figures 4.3 and 4.4 we change one of the aboveparameters, while keeping the others unchanged.

In Figure 4.3(a) it is seen that the effects of L are minimal. Figure 4.3(b) showsthat increasing U , also increases qmax and k∗ for all three queues. Finally, Fig-


ure 4.3(c) shows that the flow at jam density is decreased when N increases.Figure 4.4(a) shows that increasing µL, also increased qmax and k∗ for all three

queues. However, the effect is dampened in higher numbered queues. Increasing µHresults in increasing qmax and decreasing k∗, as is shown in Figure 4.4(b). Finally,in Figure 4.4(c) it is shown that C serves as a scaling parameter for the density.

4.3 A three queue tandem network of four-stageM/M/1 feedback threshold queues

We consider a tandem queueing network of three identical four-stage M/M/1 feed-back threshold queues with finite buffer N , with exponential service times at eachqueue, and with Poisson arrivals to the first queue. The arrival rate to the firstqueue and service rates are stage-dependent and controlled by a threshold policy.This threshold policy determines the stage of a queue based on its queue length.Let (n1, s1, n2, s2, n3, s3) denote the states of this Markov chain in which ni, withni = 0, . . . , N , and si, with si = 1, . . . , 4, denote the queue length and stage, respec-tively, of queue i. If queue i is in stage si and a departure or an arrival causes thequeue length to drop below Lsi or to exceed Usi , the stage changes. These changesare depicted in the state diagram in Figure 4.2. For example, suppose ni = L3 andsi = 3, then, a departure changes the stage of queue i to si = 1. In this section weassume

L1 = 0, L2 = L4 = Lµ, L3 = Lλ, U1 = U3 = Uµ, U2 = Uλ, U4 = N,

and

0 < Lλ < Lµ ≤ Uµ < Uλ < N.

The arrival rate to the first queue depends on the stage of queue 1 and is givenin Table 4.2. The service rates of queue 1 and queue 2 depend on their own stage,

Stage of queue 1 Arrival Rate

s1 = 1 λH

s1 = 2 λH

s1 = 3 λL

s1 = 4 λL

Table 4.2: The stage dependent arrival rates to queue 1.

as well as the stage of the downsteam queue. In Table 4.3 we give the service ratesof queue i, i = 1, 2. Finally, the service rates of the last queue only on the stage of

4.3 Four-stage M/M/1 tandem feedback threshold queue 39

s=

10

1··

·Lλ

−1

Lλ

···

Lµ

−1

Lµ

···

Uµ

s=

2Lµ

···

Uµ

Uµ

+1

···

Uλ

s=

3Lλ

···

Lµ

−1

Lµ

···

Uµ

s=

4Lµ

···

Uµ

Uµ

+1

···

Uλ

Uλ

+1

···

N−

1N

Fig

ure

4.5

:T

ransi

tion

Dia

gra

mfo

rth

esi

ngle

four-

stageM/M

/1

feed

back

thre

shold

queu

e.


Stage of queue i Stage of queue i + 1 Service rate

si = 1 or si = 3 si+1 = 1 or si+1 = 2 µHH

si = 1 or si = 3 si+1 = 3 or si+1 = 4 µHL

si = 2 or si = 4 si+1 = 1 or si+1 = 2 µLH

si = 2 or si = 4 si+1 = 3 or si+1 = 4 µLL

Table 4.3: The stage dependent service rates of queue i.

Stage of queue 3 Service rate

s3 = 1 µH

s3 = 2 µL

s3 = 3 µH

s3 = 4 µL

Table 4.4: The stage dependent service rates of queue 3.

queue 3 and are given in Table 4.4.

Let Q be the generator of this Markov chain and let π be its stationary distribu-tion such that πQ = 0 for πe = 1. By modelling this Markov chain as a finite LevelDependent Quasi-Birth-and-Death process (LDQBD) we can obtain the stationarydistribution using the Successive Censoring Algorithm from Chapter 7.

Let the element π(n1, s1, n2, s2, n3, s3) of π be the (stationary) probability thatthe Markov chain is in state (n1, s1, n2, s2, n3, s3). Furthermore, let πi be themarginal queue length distribution of queue i. These marginal distributions areobtained by adjusting (4.1) to include four stages. The mean queue length of queuei, E[Li], the mean sojourn time of queue i, E[Si], and the effective arrival rate to thefirst queue Λ are obtained following equations (4.2), (4.3), and (4.4),

We now apply the adjusted Heidemann’s method from equation (3.5) to obtainthe fundamental diagram for each highway section, see also Equation (4.5).

Figures 4.6, 4.7, and 4.8 characterise the fundamental diagram for each queue ina tandem network of three four-stage M/M/1 feedback threshold queues for differentscenarios. The results for each of the three scenarios are displayed in three horizontalplots, characterising the fundamental diagram for the first queue (left), the secondqueue (middle) and the third queue (right) in the tandem network.

4.3 Four-stage M/M/1 tandem feedback threshold queue 41

(a)

(b)

(c)

(d)

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 1

Lλ = 2Lλ = 3Lλ = 4

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 2

Lλ = 2Lλ = 3Lλ = 4

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 3

Lλ = 2Lλ = 3Lλ = 4

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 1

Lµ = 4Lµ = 5Lµ = 6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 2

Lµ = 4Lµ = 5Lµ = 6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 3

Lµ = 4Lµ = 5Lµ = 6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 1

Uµ = 5Uµ = 6Uµ = 7

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 2

Uµ = 5Uµ = 6Uµ = 7

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 3

Uµ = 5Uµ = 6Uµ = 7

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 1

Uλ = 7Uλ = 8Uλ = 9

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 2

Uλ = 7Uλ = 8Uλ = 9

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow(q)

Queue 3

Uλ = 7Uλ = 8Uλ = 9

Figure 4.6: Flow-Density diagram for a tandem network of three four-stage M/M/1feedback threshold queue with varying (a) Lλ, (b) Lµ, (c) Uµ, and (d) Uλ.


Each scenario is based on the basic scenario with

Lλ = 3, µHH = 15, N = 8,

Lµ = 5, µHL = 10, C = 1,

Uµ = 6, µLH = 6, κ = 0.4,

Uλ = 8, µLL = 4,

In each of the, in total, 11 scenarios in Figures 4.6, 4.7, and 4.8 we alter oneof the above parameters, while keeping the others unchanged. Changing Lλ or Uλhas minimal effect on the shape of the fundamental diagram, as can be seen in Fig-ures 4.6(a), and 4.6(d). Figure 4.6(b) shows that increasing Lµ slightly increasesqmax, k∗, and kjam. Increasing Uµ increase both qmax and k∗ as is shown in Fig-ure 4.6(c), furthermore, it decreases kjam for queue 1 and increases kjam for queue3.

Figures 4.7(b) and 4.7(d) show that the shape of the fundamental diagram ofqueue 1 is not affected when µHL and µLL are increased, while the fundamentaldiagrams for queues 2 and 3 are slightly affected. Figure 4.7(a) it is shown thatqmax increases, and k∗ and kjam decrease when µHH is increased. Increasing µLHincreases qmax, k∗, and kjam as is shown in Figure 4.7(c).

Figure 4.8(a) shows that increasing N increases the flow at jam density. Increas-ing κ has a similar result on the fundamental diagram for queue 1. The effects on thefundamental diagrams for queue 2 and 3 are negligible, as is shown in Figure 4.8(c).Figure 4.8(b) shows that C is a scaling parameter for the density.


In this chapter we have introduced a tandem network of three identical queues tomodel highway traffic. We considered two different networks, one consisting of two-stage M/M/1 threshold queues and one consisting of four-stage M/M/1 feedbackthreshold queues. Both tandem networks were analysed numerically to obtain thestationary queue length distribution and the mean sojourn time in each queue. Next,the fundamental diagram was obtained for each queue in the network using theadjusted Heidemann’s method.

The 3-tandem network of two-stage M/M/1 threshold queues models three con-secutive highway sections. The service rates in each queue are controlled by a thresh-old policy, based on the queue length, to model the hysteretic behaviour of traffic onthe highway section. Numerical results shows that the shape of each fundamentaldiagram is greatly affected by the service rates of each queue, µH and µL. Otherparameters that influence the shape of each fundamental diagram are the buffer sizeN and the upper threshold U .

The 3-tandem network of four-stage M/M/1 feedback threshold queues modelthe hysteretic behaviour of traffic on three consecutive highway sections, as well asthe hysteretic interaction between consecutive section, i.e., once a highway sectionbecomes incredibly crowded, the vehicles in the preceding section will drive slower.


(a)

(b)

(c)

(d)

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

µHH = 10µHH = 15µHH = 20

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2

µHH = 10µHH = 15µHH = 20

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3

µHH = 10µHH = 15µHH = 20

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

µHL = 6µHL = 10µHL = 14

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2

µHL = 6µHL = 10µHL = 14

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3

µHL = 6µHL = 10µHL = 14

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

µLH = 3µLH = 6µLH = 9

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2


0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3


0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

µLL = 2µLL = 4µLL = 6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2


0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3


Figure 4.7: Flow-Density diagram for a tandem network of three four-stage M/M/1feedback threshold queue with varying (a) µHH , (b) µHL, (c) µLH , and (d) µLL.

Numerical results show that the shape of each fundamental diagram is greatly af-fected by the service rates µHH and µLH . Other parameters that influence the shape


(a)

(b)

(c)

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

N = 10N = 15N = 20

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2

N = 10N = 15N = 20

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3

N = 10N = 15N = 20

0 1 2 30

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

C = 1C = 2C = 3

0 1 2 30

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2

C = 1C = 2C = 3

0 1 2 30

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3

C = 1C = 2C = 3

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 1

κ = 0.2κ = 0.4κ = 0.6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 2

κ = 0.2κ = 0.4κ = 0.6

0 0.25 0.5 0.75 10

1

2

3

4

5

Density (k)

Flow

(q)

Queue 3

κ = 0.2κ = 0.4κ = 0.6

Figure 4.8: Flow-Density diagram for a tandem network of three four-stage M/M/1feedback threshold queue with varying (a) N , (b) C, and (c) κ.

of each fundamental diagram are the buffer size N and the upper threshold Uµ.Both tandem networks suffer from state space explosion making a closed-form

expression for the stationary distribution intractable. Instead, the stationary dis-tribution was obtained numerically using the successive censoring algorithm fromChapter 7. Chapter 8 introduces the iterative aggregation method which approxi-mates the marginal queue length distribution of a single queue in a tandem network.The iterative aggregation method makes it possible to consider even larger tandemnetworks.

Part II

Queueing

CHAPTER 5

Matrix analytic methods

5.1 Introduction

This chapter is a preliminary chapter to Chapters 6, 7, and 8. It presents a briefoverview of known results from the area of Matrix analytic methods that we use inthese chapters. First we discuss the Phase-Type (PH) distribution, which gener-alises the Exponential, Erlang and Hyperexponential distributions. In Chapter 6 weuse the PH distribution to introduce the PH/PH/1 multi threshold queue. Second,we discuss the Markovian Arrival Process (MAP ) and Markovian Service Process(MSP ), an extension to the PH distribution which makes it possible to incorporatecorrelated interarrival times and correlated service times. The iterative aggregationmethod in Chapter 8 gives an aggregation method for both a MAP and MSP toapproximate a tandem queueing network with finite buffers. Third, we give an intro-duction to the Quasi-Birth-and-Death process (QBD) and show how the stationarydistribution is obtained. We also give the fundamental matrix for a transient QBD,i.e., the average time spent in a state in the QBD before absorption into an absorb-ing state. Finally, we discuss the Level Dependent Quasi-Birth-and-Death process(LDQBD), an extension to the QBD in which level dependent transition rates areallowed. We give results on the stationary distribution of an LDQBD and present thefundamental matrix for a transient LDQBD. The fundamental matrices of a transientQBD and a transient LDQBD form the basis for the successive censoring algorithmpresented in Chapter 7. The results on QBD’s and LDQBD’s are extensively usedin Chapters 6, 7, and 8.

We refer to the books by Neuts [76] and Latouche and Ramaswami [58] for athorough overview on the topics discussed in this chapter.

5.2 Phase-Type distribution

In Chapter 6 we extend the two-stage M/M/1 threshold queue and four-stageM/M/1 feedback threshold queue to a queueing model with multiple thresholdsand in which both the arrival and service process are described by a Phase-Typedistribution. A continuous Phase-Type (PH) distribution is the distribution of thetime until absorption in an absorbing continuous time Markov chain described by

48 5. Matrix analytic methods

1 01 λ

Figure 5.1: Phase diagram of an Exponential distribution.

Neuts in [76]. It is a generalisation of, among others, the Exponential, Erlang, andHyper-Exponential distributions and allows us to approximate any distribution withnon-negative support arbitrarily close, see [48]. A PH distribution is described byan absorbing Markov chain consisting of n transient states and 1 absorbing stateand by the generator

G =

[0 0

T 0 T

],

where T is an n× n matrix describing transition rates between transient states andT 0 an n × 1 vector describing the transition rates into the absorbing state. Let edenote an n × 1 vector of all ones, then T 0 = −Te. The initial state probabilityvector of G is g = [t, tn+1] with te+ tn+1 = 1. The PH distribution with generatorG and probability vector g is completely determined by the matrix T and vector tand we denote this distribution by PH(T , t).

A (positive) random variable X which is PH(T , t) distributed, has distributionfunction

F (x) = 1− teTxe,

and density function

f(x) = teTxT 0,

where the matrix exponential is defined by

eT =

∞∑n=0

T n

n!.

The moments of X are given by

E[Xn] = (−1)nn!tT−ne, n = 1, 2, . . . .

Example 5.1. (Exponential Distribution) The exponential distribution, depicted inFigure 5.1, is a special PH distribution consisting of a single transient state. Withprobability 1, state 1 is the initial state and a transition from state 1 to the absorbingstate 0 occurs with rate λ. The exponential distribution is a PH distribution with

T = −λ, t = 1.

5.2 Phase-Type distribution 49

1 2 · · · k 01 λ λ λ λ

Figure 5.2: Phase diagram of an Er(k, λ) distribution.

Example 5.2. (Erlang distribution) An Erlang distribution as depicted in Fig-ure 5.2 consists of k consecutive exponential phases, each with rate λ. This Er(k, λ)distribution is represented by a PH distribution with k transient phases and

T =

−λ λ 0 · · · 0

0 −λ . . .. . .

......

. . .. . . λ 0

.... . . −λ λ

0 · · · · · · 0 −λ

, t =

[1 0 · · · 0

].

Example 5.3. (Hyper-exponential distribution) A Hyper-exponential distribution,as depicted in Figure 5.3, is defined by two vectors p = [ p1 p2 · · · pk ] andλ = [ λ1 λ2 · · · λk ] and denoted by Hk(p,λ). A Hk(p,λ) is represented by thefollowing PH distribution

T =

−λ1 0 · · · · · · 0

0 −λ2. . .

......

. . .. . .

. . ....

.... . . −λk−1 0

0 · · · · · · 0 −λk

, t =

[p1 p2 · · · pk−1 pk

],

with

pe = 1.

Example 5.4. (Coxian distribution) The Coxian distribution with k phases is de-picted by the phase diagram in Figure 5.4. The initial state is state 1 with probability1, and each state j, j = 1, . . . , k, corresponds to an exponential state with rate λj .After leaving state j, the process moves to state j + 1 with probability pj and itterminates with probability 1 − pj . After leaving state k, the process terminates,i.e., pk = 0. The Ck(p,λ) is defined by the two vectors p = [ p1 p2 · · · pk ] and


1

...

k

0

0

p1

pk

λ1

λk

Figure 5.3: Phase diagram of a H(p,λ) distribution.

1 2 · · · k 01 p1λ1

(1− p1)λ1

p2λ2

(1− p2)λ2

pk−1λk−1

(1− pk−1)λk−1

λk

Figure 5.4: Phase diagram of a Ck(p,λ) distribution.

λ = [ λ1 λ2 · · · λk ] and it is a PH distribution with

T =

−λ1 p1λ1 · · · · · · 0

0 −λ2 p2λ2

......

. . .. . .

. . ....

.... . . −λk−1 pk−1λk−1

0 · · · · · · 0 −λk

, t =

[1 0 · · · 0

].

5.3 Markovian Arrival and Service Processes

The PH distributions from Section 5.2 allow us to approximate any distributionwith non-negative support arbitrarily close, see [48]. However, they are not capableof modelling a correlated arrival or service process. To this end, the MAP andMSP were introduced by Neuts [76]. An m-state MAP is a counting process (n, j)in which n records the number of arrivals and j, j = 1, . . . ,m, describes the state ofan underlying Markov chain. This counting process is defined by two matrices A0

and A1. The matrix A0 describes transitions within the underlying Markov chainwithout arrivals occurring, i.e., a transition generated by A0 only changes the state

5.3 Markovian Arrival and Service Processes 51

of the underlying Markov chain, j, and keeps n unchanged. The matrix A1 describesa transition in the underlying Markov chain while an arrival occurs, i.e., a transitiongenerated by A1 causes n to be increased by 1 and may change the state j.

More specifically, an m-state MAP is defined by two m × m matrices A0 andA1, with [A0]i,i < 0, [A0]i,j ≥ 0, i 6= j, and [A1]i,k ≥ 0, for i, j, k = 1, . . . ,m. Theelement [A1]i,j of A1, is the rate at which arrivals occur and the state of the MAPchanges from i to j. The element [A0]i,j , i 6= j , of A0 describes the rate at whichthe MAP changes from state i to j, without the occurrence of an arrival to thequeue. The negative diagonal elements of A0 are such that A0 +A1 has row sumsequal to zero.

Similarly, anm-stateMSP is defined by two matrices S0 and S1, with [S0]i,i < 0,[S0]i,j ≥ 0, i 6= j, and [S1]i,j ≥ 0, for i, j, k = 1, . . . ,m. The matrix S1 describes therate at which the state of the MSP changes and a departure occurs. The matrix S0

describes a state change of the MSP without departures from the queue. Here, thediagonal elements of S0 are such that S0 + S1 has row sums equal to zero.

In Chapter 8 we consider a tandem queueing network with finite buffers, witharrivals to the first queue according to a MAP , and with departures generatedaccording to a MSP . We present an iterative aggregation method to approximatea single queue in the tandem network by a MAP/MSP/1/N queue and determineits mean sojourn time.

Example 5.5. (Markov Modulated Poisson Process) The Markov Modulated Pois-son Process (MMPP ) is a Poisson process in which its intensity parameter is gov-erned by a random environment, i.e., an underlying m-state Markov chain, allowingfor correlation between inter-arrival times. Typical for a MMPP is that the state ofthe underlying Markov chain cannot change during arrivals, making A1 a diagonalmatrix. Let R be the rate matrix at which the underlying Markov chain changes,then

A0 = R−A1, A1 =

λ1 0 · · · 0

0 λ2. . .

......

. . .. . . 0

0 · · · 0 λm

.

Special cases of the MMPP are the Switched Poisson Process (SPP ) with m = 2,and the Interrupted Poisson Process (IPP ) with m = 2 and λ1 = 0.

Example 5.6. (PH distribution) The PH(T , t) distribution is a trivial example ofa MAP with

A0 = T , A1 = T 0t,

where T describes the transitions between the transient phases of the the PH(T , t)distribution. With rate T 0 an arrival occurs and an initial phase is chosen accordingto t. The product gives the rate matrix A1 at which arrivals occur.


5.4 Quasi-Birth-and-Death processes

A Quasi-Birth-and-Death process (QBD) is a two-dimensional Markov chain withstate space S = (n, j), n ≥ 0, j = 1, . . . ,m in which j denotes the phase ofthe QBD, and n denotes the level of the QBD, each with m < ∞ phases. In aQBD, transitions from a state are restricted to states within the same level, orto states in one of the adjacent levels. More precisely, starting in state (n, j) itis possible to reach (n′, j′) in a single step only if n′ = n + 1, n′ = n, or n′ =n − 1 (taking into account that n′ ≥ 0). Furthermore, in a QBD the transitionsare assumed to be level independent. Ordering the states lexicographically, i.e.,(0, 0), (0, 1), . . . , (0,m), (1, 0), . . . , (1,m), (2, 0), . . . , (2,m), . . ., the generator Q has atri-diagonal structure and is given by:

Q =

L0 F 0 · · ·B L F

. . .

0 B L. . .

.... . .

. . .. . .

, (5.1)

where F denotes the forward transition (arrivals) from level i to i+1, for i = 0, 1, . . .,where B denotes the backward transitions (departures) from level i to i − 1, fori = 1, 2, . . ., and where L denotes the local transitions within level i, for i = 1, 2, . . ..The matrix L0 describes the boundary behaviour in level 0.

The results on QBDs in this section are used in the Chapters 6, 7, and 8.

Example 5.7. (M/M/1 queue) The M/M/1 queue with arrival rate λ and servicerate µ is a typical example of a QBD with

L = −λ− µ, F = λ, B = µ, L0 = −λ.

Example 5.8. (MAP/MSP/1 queue) Let us consider a MAP/MSP/1 queue witha MAP defined by A0 and A1 and a MSP defined by S0 and S1. Furthermore,let us denote by IA and IS two identity matrices of the same size as A0 and S0,respectively. Finally, let ⊗ be the Kronecker product, i.e., if C is an c1 × c2 matrixand D is a d1 × d2 matrix, then C ⊗D is the c1d1 × c2d2 matrix:

C ⊗D =

[C]1,1D · · · [C]1,c2 D

......

[C]c1,1D · · · [C]c1,c2 D

,and let ⊕ be the Kronecker sum, i.e., S⊕A = S⊗IA +A⊗IS . The MAP/MSP/1queue is a QBD with

L = S0 ⊕A0, F = IS ⊗A1, B = S1 ⊗ IA, L0 = IS ⊗A0.

5.4 Quasi-Birth-and-Death processes 53

In the remainder of this section we will discuss how the stationary distributionπ for a QBD is obtained such that πQ = 0 and πe = 1. We first focus on a QBDwith an infinite number of levels as depicted by the generator in (5.1). Next, wediscuss the stationary distribution of a QBD with a finite number of levels. Finally,we determine the fundamental matrix of a transient QBD, which records the timeuntil absorption in a transient QBD.

5.4.1 Stationary distribution for a QBD

Let us consider a QBD with generator Q as in (5.1) and let π = [ π0 π1 · · · ] beits stationary distribution, i.e., πQ = 0 such that πe = 1. The QBD with generatorQ is positive recurrent (stable) if and only if the mean drift condition is satisfied

pFe < pBe,

see [76], where e is a column vector of ones and p = [ p1 p2 · · · pm ] is thestationary probability vector of F +L+B, i.e.,

p [F +L+B] = 0, pe = 1.

If the QBD is stable, its stationary distribution π is given by, see [76],

πi = π0Ri, i ≥ 0,

where R is the minimal non-negative solution to the quadratic matrix equation

F +RL+R2B = 0,

and π0 is given by the boundary equation

π0L0 + π1B = π0 [L0 +RB] = 0,

and∞∑i=0

πie = π0

∞∑i=0

Rie = π0 [I −R]−1e = 1.

The matrix R can be numerically obtained by the fixed point iteration given byNeuts in [76], with R0 = 0 and

Rk+1 = −[F +R2

kB]L−1,

or the Logarithmic Reduction Algorithm by Latouche and Ramaswami in [57].


5.4.2 Stationary distribution for a finite QBD

Let us now consider a QBD with N + 1 levels and generator Q

Q =

L0 F 0 · · · 0

B L. . .

. . ....

0. . .

. . .. . . 0

.... . .

. . . L F

0 · · · 0 B LN

. (5.2)

The stationary distribution π = [ π1 π2 · · · πN ] is given by, see Hajek [38],

πi = x0Ri1 + xNR

N−i2 , 0 ≤ i ≤ N,

where R1 and R2 are the minimal non-negative solutions to

F +R1L+R21B = 0, R2

2F +R2L+B = 0.

and [ x0 xN ] is the solution of the system

[ x0 xN ]

[L0 +R1B RN−1

1 [F +R1LN ]

RN−12 [R2L0 +B] R2F +LN

]= [ 0 0 ] ,

and

x0

N∑i=0

Ri1e+ xN

N∑i=0

R2e = 1.

The matrices R1 and R2 can be obtained using the fixed point iteration by Neuts,presented in the previous section, and the Logarithmic Reduction Algorithm byLatouche and Ramaswami in [57].

5.4.3 Fundamental matrix of a transient QBD

Let us first consider an absorbing Markov chain with generator

G =

[0 0

T 0 T

].

Suppose that the Markov chain is entered in state i and we want to know the averagetime spent in state j, before the Markov chain is absorbed in state 0. This meansojourn time is given by the fundamental matrix of T , −T−1, i.e.,[

−T−1]i,j

= E[Time spent in j before absorption, starting in i].

5.4 Quasi-Birth-and-Death processes 55

Now suppose T has the following transient QBD structure with M levels:

T =

X F

B L. . .

. . .. . .

. . .

. . . L F

B Y

.

The fundamental matrix of T is obtained by Choi et al. in [21]. Let κ denote thestationary distribution of F + L +B, i.e., κ(B + L + F ) = 0 and κe = 1, and letρ = (κFe)/(κBe). Here we assume that ρ 6= 1 and M < ∞. We refer to [21] forthe case where M =∞ and the case where ρ = 1 and (B +L+ F )e = 0.

We define R1 and R2 as the minimal non-negative solutions to

F +R1L+R21B = 0, R2

2F +R2L+B = 0. (5.3)

Finally, we define the matrix Z as follows

[−T ]−1

= Z =

Z(1, 1) Z(1, 2) · · · Z(1,M)

Z(2, 1) Z(2, 2) · · · Z(2,M)...

.... . .

...

Z(M, 1) Z(M, 2) · · · Z(M,M)

.The rows of Z follow from Theorem 6 in [21]:

(i) The first and last rows are given by:

Z(1, k) = V (1, 1)Rk−11 + V (1, 2)RM−k

2 , 1 ≤ k ≤M,

Z(M,k) = V (2, 1)Rk−11 + V (2, 2)RM−k

2 , 1 ≤ k ≤M,(5.4)

where[V (1, 1) V (1, 2)

V (2, 1) V (2, 2)

]= −

[X +R1B RM−2

1 [F +R1Y ]

RM−22 [R2X +B] R2F + Y

]−1

. (5.5)

(ii) For 2 ≤ i ≤M − 2, the i-th row is given by:

Z(i, k) =

V (i, 1)Rk−11 + V (i, 2)Ri−k

2 , 1 ≤ k ≤ i,V (i, 3)Rk−i−1

1 + V (i, 4)RM−k2 , i+ 1 ≤ k ≤M,

(5.6)

and for 3 ≤ i ≤M − 1, the i-th row is given by:

Z(i, k) =

W (i, 1)Rk−11 +W (i, 2)Ri−k−1

2 , 1 ≤ k ≤ i− 1,

W (i, 3)Rk−i1 +W (i, 4)RM−k

2 , i ≤ k ≤M,(5.7)


where for 2 ≤ i ≤M − 2,[V (i, 1) V (i, 2) V (i, 3) V (i, 4)

]=[

0 −I 0 0

] (B(i)[R1,R2]

)−1

,

and for 3 ≤ i ≤M − 1,

[W (i, 1) W (i, 2) W (i, 3) W (i, 4)

]=[

0 0 −I 0

] (B(i−1)[R1,R2]

)−1

.

The matrix B(i)[R1,R2] is defined as,

B(i)[R1,R2] =X +R1B −Ri

1B Ri−11 F 0

Ri−22 [R2X +B] R2F +L F 0

0 B L+R1B RM−i−21 [F +R1Y ]

0 RM−i−12 B −RM−i

2 F R2F + Y

. (5.8)

In Chapter 7 we will use the fundamental matrix in our Successive CensoringAlgorithm for a system of connected QBD-processes.

5.5 Level Dependent QBD processes

A Level Dependent Quasi-Birth-and-Death process (LDQBD) is a generalisation ofa QBD in which the transitions may be level dependent. It is a two-dimensionalMarkov chain with state space S = (n, j), n ≥ 0, j(n) = 1, . . . ,mn in which j isthe phase of the LDQBD, and in which n denotes the level of the LDQBD wherelevel i consists of mi phases. In a LDQBD, transitions from a state are restrictedto states within the same level, or to states in one of the adjacent levels. Moreprecisely, starting in state (n, j) it is possible to reach (n′, j′) in a single step only ifn′ = n + 1, n′ = n, or n′ = n − 1 (taking into account that n′ ≥ 0). Ordering thestates lexicographically, i.e., (0, 0), (0, 1), . . ., (0,m0), (1, 0), . . ., (1,m1), (2, 0), . . .,(2,m2), . . ., the generator Q has a tri-diagonal structure and is given by

Q =

L(0) F (0) 0 · · ·B(1) L(1) F (1)

. . .

0 B(2) L(2). . .

.... . .

. . .. . . F (i−1)

B(i) L(i). . .

. . .. . .

, (5.9)

5.5 Level Dependent QBD processes 57

where B(i) denotes the backward transitions (departures) from level i to level i− 1,L(i) the local transitions within level i and F (i) the forward transitions (arrivals)from level i to level i+ 1.

In the remainder of this section we will discuss how the stationary distributionπ for a LDQBD is obtained such that πQ = 0 and πe = 1. We first focus ona LDQBD with an infinite number of levels as depicted by the generator in (5.9).Next, we discuss the stationary distribution of a LDQBD with a finite number oflevels. Finally, we determine the fundamental matrix of a transient LDQBD, whichrecords the time until absorption in a transient LDQBD.

The results on LDQBDs in this section are used throughout the Chapters 6, 7,and 8.

5.5.1 Stationary distribution for a LDQBD

Let us consider a LDQBD with generator Q as in (5.9) and let π = [ π1 π2 · · · ] beits stationary distribution, i.e., πQ = 0 such that πe = 1. The LDQBD is positiverecurrent (stable) if and only if the equation, see Bright and Taylor [16],

π0

[L(0) +R(0)B(1)

]= 0, (5.10)

has a positive solution π0 such that

π0

∞∑k=0

[k−1∏m=0

R(m)

]e <∞,

with R(k) the minimal non-negative solution to

F (k) +R(k)L(k+1) +R(k)R(k+1)B(k+2) = 0, k ≥ 0.

If the LDQBD is stable, the stationary distribution π = [ π1 π2 · · · ] is given by,see Bright and Taylor [16]

πk = π0

k−1∏m=0

R(m), k ≥ 0,

where π0 satisfies (5.10) and

π0

∞∑k=0

[k−1∏m=0

R(m)

]e = 1.

The matrices R(k), k ≥ 0, can be numerically obtained by the algorithm proposedby Bright and Taylor in [16].


5.5.2 Stationary distribution for a finite LDQBD

Let us now consider a LDQBD with N + 1 levels and generator Q

Q =

L(0) F (0) 0 · · · · · · 0

B(1) L(1) F (1). . .

...

0 B(2). . .

. . .. . .

......

. . .. . .

. . .. . . 0

.... . .

. . . L(N−1) F (N−1)

0 · · · · · · 0 B(N) L(N)

. (5.11)

The stationary distribution π = [ π1 π2 · · · πN ] is given as follows, see Gaver,Jacobs and Latouche [34],

C0 = L(0),

Ci = L(i) +B(i) [−Ci−1]−1F (i−1), 1 ≤ i ≤ N,

and

πNCN = 0,

πi = πi+1B(i+1) [−Ci]−1

, 0 ≤ i ≤ N − 1,

such that

N∑i=0

πie = 1.

5.5.3 Fundamental matrix of a transient LDQBD

The fundamental matrix of a transient LDQBD is obtained by Shin in [92]. Let Tbe the generator of a transient LDQBD of M levels:

T =

L(1) F (1) 0 · · · 0

B(2) L(2). . .

. . ....

0. . .

. . .. . . 0

.... . .

. . . L(M−1) F (M−1)

0 · · · 0 B(M) L(M)

.

5.5 Level Dependent QBD processes 59

We define the non-negative matrices R(k) and G(k), 1 ≤ k ≤M , given as follows

R(M) = −F (M−1)[L(M)

]−1,

R(k) = −F (k−1)[L(k) +R(k+1)B(k+1)

]−1, k = M − 1,M − 2, . . . , 2,

R(1) = −[L(1) +R(2)B(2)

]−1,

and

G(M) = −[L(M)

]−1B(M),

G(k) = −[L(k) + F (k)G(n+1)

]−1B(k), k = M − 1,M − 2, . . . , 2,

G(1) = −[L(1) + F (1)G(2)

]−1,

Let Z denote the fundamental matrix of T

[−T ]−1

= Z =

Z(1, 1) Z(1, 2) · · · Z(1,M)

Z(2, 1) Z(2, 2) · · · Z(2,M)...

.... . .

...

Z(M, 1) Z(M, 2) · · · Z(M,M)

.

Theorem 2.1 in [92] states that Z is given as follows

(i) The first row and column blocks, for k = 1, 2, . . . ,M

Z(1, k) = R(1)R(2) · · ·R(k),

Z(k, 1) = G(k)G(k−1) · · ·G(1).

(ii) The n-th, 2 ≤ n ≤M−1, row and column blocks, for k = n+1, n+2, . . . ,M ,

Z(n, n) = −[I +Z(n, n− 1)F (n−1)

] [L(n) +R(n+1)B(n+1)

]−1,

Z(n, k) = Z(n, n)R(n+1)R(n+2) · · ·R(k),

Z(k, n) = G(k)G(k−1) · · ·G(n+1)Z(n, n).

(iii) The (M,M)-block component

Z(M,M) = −[I +Z(M,M − 1)F (M−1)

] [L(M)

]−1.


CHAPTER 6

The PH/PH/1 multi-threshold queue

6.1 Introduction

We consider a PH/PH/1 queue in which a threshold policy determines the stage ofthe system. The arrival and service processes are both described by a Phase-Type(PH) distribution depending on the stage of the system. Each stage has both alower and an upper threshold at which the stage of the system changes. At thesethresholds a new stage is chosen according to a prescribed distribution.

The queueing system in this chapter is motivated by the hysteretic relation be-tween density and speed of traffic flows observed on a highway, see Helbing [46]. In[46] it is stated that this hysteretic behaviour is controlled by two critical densities,denoted by ρ1 and ρ2. When the density of cars on the highway increases vehiclesare more and more affected by each other and the driving speeds decrease. Oncethe density reaches ρ2 the highway becomes congested and driving speeds decreasedrastically. The density must reduce to ρ1 for the highway to become non-congested.In Baer, Boucherie and van Ommeren [9], an M/M/1 queue threshold queue wasused to model a single highway section, see also Chapter 3. In [9], the arrival rateswere kept constant, whereas the service rates where altered according to a two-stagethreshold policy. When the queue length surpasses an upper threshold the servicerates were decreased. The service rates were increased again when the queue lengthdropped below a lower threshold. In [9], the mean sojourn time is determined. Sincea queue represents a single highway section, this directly gives the average time tocross the highway section and the mean speed of a vehicle. The motivating examplein Figure 6.1 is an extension to the model in [9], where not only the service ratesare controlled by a threshold policy, but also the arrival rates. This models the hys-teretic relation within a highway section, but also between two consecutive highwaysections. We will get back to this example in Section 6.4.1.

In the literature, threshold policies are often used to activate or deactivate serverswhen the queue length reaches certain thresholds. The M/M/2 queue in which thesecond server is activated when the queue length reaches an upper threshold and de-activated when it reaches a lower threshold is studied by Le Ny in [59], where a closedform expression is obtained for the steady-state probabilities. In Le Ny and Tuffin[61], see also Section 6.4.2, closed form expressions are obtained for the steady-statedistributions for the M/M/c queue with c heterogeneous servers. Using Green’s

62 6. The PH/PH/1 multi-threshold queue

function, Ibe and Keilson [49] studied the M/M/c queue with homogeneous serversand the M/M/2 queue with heterogeneous servers. The M/M/c queue with hetero-geneous servers is also studied by Lui and Golubchik in [70]. A MAP/M/c queuewith homogeneous servers is analysed by Chakravarthy in [19] and the PH/M/2queue with heterogeneous servers is studied by Neuts [77]. Choi et al. [21], studieda very general setting in which the generator of the queueing system forms a nestedQBD, see also Section 6.4.3. In this queueing model a threshold policy controlsthe stage of the system which, in turn, determines the arrival process and the ser-vice process. An upper threshold increases the stage by one whereas the the lowerthreshold decreases the stage by one, creating a staircase threshold policy. Le Ny[60] studied an M/M/2 queue with two heterogeneous servers in which the secondserver is exponentially delayed before activation.

Threshold policies are also used to send servers to a certain queue, as is shown byFeng, Adachi and Kowada in [33]. In [33], a system is studied containing two queuesand two servers where both interarrival times and service times are exponentiallydistributed. After each service completion, the server chooses a queue to serveaccording to a threshold policy. A generalisation of this model is analysed by Chou,Golubchik and Lui in [22] where customers from multiple classes arrive accordingto a Poisson process and require an exponential amount of service. The queueingsystem contains a fixed number of servers which are allocated to a customer classaccording to a threshold policy. Each server experiences an exponential delay onceit is assigned to a different customer class. Zhu, Yang and Basu [108] obtained thejoint queue length distribution for an M/G/1 queue with multiple customer classesin which customers from higher class are blocked when thresholds are reached.

This chapter generalises the model of Choi [21] to an arbitrary threshold policyand introduces a novel dedicated solution method based on the LDQBD of [16]. Inparticular, a class of PH/PH/1 multi-threshold queueing systems is described forwhich the solution method in [16] can be decomposed to find the stationary queuelength vector for each stage separately. The stationary distribution of the PH/PH/1multi-threshold queue can be obtained using the results in [16] but for a large numberof stages, this may result in computational demanding calculations. In this chapterwe use the structure of the PH/PH/1 multi-threshold queue to form, based on theresults in [16], smaller and easier equations to obtain the stationary distribution.

Section 6.2 introduces the PH/PH/1 multi-threshold queue and presents thequeueing system as a LDQBD. In Section 6.3 we analyse the multi-threshold queueusing Matrix Analytic methods and obtain the stationary queue length probabilities.Furthermore, we present a decomposition theorem for a class of multi-thresholdqueues providing an explicit description of the stationary queue length probabilityvectors. In Section 6.4 we illustrate our results via three multi-threshold queuesobtained from literature. We determine the fundamental diagram of traffic using thePH/PH/1 multi-threshold queue in Section 6.5 and perform a sensitivity analysison the distributions of the arrival process and the service processes. Section 6.6 givesconcluding remarks.

6.2 Model description 63

0 1 2 3 4 5 6 7 8 9 10 11 12 13 · · ·

s = 1

s = 2

s = 3

s = 4 · · ·

λ1

λ2

λ3

µ2

µ3

µ4

Figure 6.1: State Diagram

6.2 Model description

Consider a PH/PH/1 queue, controlled by a threshold policy. The system can be indifferent stages s = 1 . . . , S, where every stage s is associated with a set of feasiblequeue lengths Ls, . . . , Us. The quantities Ls and Us are the lower, respectivelyupper thresholds for stage s. In case Us = ∞, we say that stage s has no upperthreshold. We furthermore define

Umax = 1 + max Us : s = 1, . . . , S, Us <∞. (6.1)

For each queue length n = 0, 1, . . ., a stage s is a potential stage when Ls ≤ n ≤ Us.If the system is in stage s and a departure or arrival causes the queue length todrop below Ls or to exceed Us, the stage of the system changes (the thresholdpolicy). If the queue length increases to Us + 1 the stage changes from s to t withprobability ps,t. Note that ps,t > 0 implies that t is a potential stage for queue lengthUs + 1. If the queue length decreases to Ls − 1 the stage changes from s to t withprobability qs,t. Note that this implies that the PH/PH/1 is level dependent whenn < Umax and level independent when n ≥ Umax. See Figure 6.1 for an illustrationwith exponential service times and Poisson arrivals, and with Umax = 13. Here, thestages are denoted on the left of the figure, while the number of customers are givenat the top of the figure. For example, the number of customers in the queue canrange from 6 to (and including) 11, when the stage is 2, i.e., when s = 2.

The arrival process in stage s follows a PH(Λs,λs) distribution of vs + 1 phases(vs transient phases and 1 absorbing phase). We define Λ0

s = −Λsevs , with evsa vs × 1 vector of ones. Furthermore we assume that the absorbing state is neverchosen as initial state, i.e. λsevs = 1. Similarly, the service process in stage sis PH(M s,µs) distributed with ws + 1 phases. We define M0

s = −M sews andassume µsews = 1. The mean interarrival times and mean service time are given by−λjΛ−1

j evj and −µjM−1j ewj , see Neuts [76].

When an arrival or departure changes the stage of the system both the arrival


process and service process are reset by choosing a new initial phase for both pro-cesses according to the distributions of the new stage.

This PH/PH/1 multi-threshold queue can be modelled as a four-dimensionalMarkov chain (n, s, x, y) where n and s represent the queue length and stage of thesystem, x = 1, . . . , vs the phase of the arrival process and y = 1, . . . , ws the phaseof the service process. Recall that the queueing system is level independent for n ≥Umax. Therefore, this queueing system is a Quasi-Birth-and-Death process (QBD),see Latouche [57] and Section 5.4, in which the levels are represented by the queuelength n, with n ≥ Umax, i.e., level i contains all the states with n = i+Umax. Theboundary level of this QBD (level 0) now contains the entire threshold policy, i.e., thestates with n < Umax. By ordering the states lexicographically a tri-diagonal blockstructure emerges in the boundary level. This structure is utilised by modelling thequeueing system as a Level Dependent Quasi-Birth-and-Death process (LDQBD),see Bright and Taylor [16] and Section 5.5, in which the levels of the LDQBD arerepresented by the queue length n (level i contains all states with n = i). We stressthat, from here on, we refer to the queue length as the level of the LDQBD. Theother three variables represent the phase within a level. The states are orderedlexicographically in (n, s, x, y).

The generator Q for this LDQBD is:

Q =

L(0) F (0) 0 · · ·B(1) L(1) F (1)

. . .

0 B(2) L(2). . .

.... . .

. . .. . . F (i−1)

B(i) L(i). . .

. . .. . .

, (6.2)

where B(i) denotes the backward transitions (departures) from level i to level i− 1,L(i) the local transitions within level i and F (i) the forward transitions (arrivals)from level i to level i+ 1.

If the number of potential stages for level i − 1, i and i + 1, are `, m and nrespectively, B(i) is a m× ` matrix of submatrices B(i)

(j,k), L(i) is a m×m matrix of

submatrices L(i)

(j,k) and F (i) is a m × n matrix of submatrices F (i)

(j,k), describing thebackward, local and forward transition rates from stage j to stage k. Let It denotethe t× t identity matrix and let ⊗ denote the Kronecker product. For s = 1, . . . , S,the forward, local and backward submatrices are given by

F(i)(s,j) =

Λ0s ⊗ λs ⊗ Iws , if j = s and Ls ≤ i < Us,

ps,jΛ0s ⊗ ews ⊗ λj ⊗ µj , if i = Us,

0, otherwise.

(6.3)

6.3 Steady-state analysis 65

L(i)(s,j) =

Λs ⊗ Iws + Ivs ⊗M s, if j = s, i > 0 and Ls ≤ i ≤ Us,Λs ⊗ Iws , if j = s, i = 0 and Ls = 0,

0, otherwise.

(6.4)

B(i)(s,j) =

Ivs ⊗M0

s ⊗ µs, if j = s and Ls < i ≤ Us,qs,jevs ⊗M0

s ⊗ λj ⊗ µj , if i = Ls,

0, otherwise.

(6.5)

These formulas can be obtained by closely observing the queueing system. Con-sider, for instance, the forward transition matrices F (i)

(s,j). When Ls ≤ i < Us the

stage cannot change upon an arrival, so j = s. Now, with rate Λ0s an arrival occurs

at which an initial state is chosen with probability λs, independent of the phase ofthe service process. The stage will change when an arrival occurs when i = Us. Now,with rate Λ0

s, independent of the phase of the service process, an arrival occurs andthe stage changes from s to j with probability ps,j . During this event an initial phaseis chosen for both the arrival process and the service process respectively probabilityλj and µj . Similar reasoning gives the relations for L(i)

(s,j) and B(i)

(s,j).

6.3 Steady-state analysis

In the previous section we modelled the PH/PH/1 multi-threshold queue as aLDQBD. In this section we obtain the steady-state probabilities of the Markov chainusing Matrix analytic methods following the analysis by Bright and Taylor in [16].The special structure of our generator allows us to obtain an efficient algorithm forthe R-matrices.

We assume the queueing system is stable, i.e., the mean service time is less thanthe mean interarrival time, see Neuts [76], in stages without upper threshold:

−µjM−1j ewj < −λjΛ−1

j evj , for j such that Uj =∞.

The equilibrium distribution π = [π0,π1,π2, . . .] is then given, see Bright and Taylor[16], by

πn = π0

n−1∏i=0

R(i),

where R(i) is the minimal non-negative solution to

F (i) +R(i)L(i+1) +R(i)R(i+1)B(i+2) = 0, (6.6)

with 0 the zero matrix, see [16]. The element [R(i)](r,t) describes the mean sojourntime in state (i+1, t) per unit sojourn time in the state (i, r) before returning to leveli, given that the process started in state (i, r) see p. 499 in [16]. The R(i)-matricescan be obtained using the algorithm for LDQBD’s by Bright and Taylor [16]. For


later convenience, by analogy of F (i)

(j,k), L(i)

(j,k) and B(i)

(j,k), we define the submatrixR(i)

(j,k) of R(i) in which the element [R(i)

(j,k)](r,t) describes the mean sojourn time instate (i + 1, t) and stage k per unit sojourn time in state (i, r) and stage j beforereturning returning to level i, given that the process started in state (i, r) and stagej.

We obtain π0 by solving the boundary condition:

π0L(0) + π1B

(1) = π0

(L(0) +R(0)B(1)

)= 0,

and the normalising equation:

1 =

∞∑n=0

πne = π0

(I +

∞∑n=1

n−1∏i=0

R(i)

)e.

Above level Umax only stages without upper threshold are active and we may defineF = F (i), L = L(i) and B = B(i), i ≥ Umax, i.e., the LDQBD is level independentfrom level Umax upwards. We have R(i) = R, i ≥ Umax, where R is the minimalnonnegative solution of

F +RL+R2B = 0. (6.7)

The LDQBD is level independent from level Umax upwards. Therefore, the matricesF , L, B and R are diagonal block matrices. As a consequence, (6.7) reduces to thematrix equation for the submatrices R(s,s) of R

F (s,s) +R(s,s)L(s,s) +R2(s,s)B(s,s) = 0, for s such that Us =∞. (6.8)

For i < Umax, the matrices R(i) are obtained from (6.6) by iteration

R(i) = −F (i)[L(i+1) +R(i+1)B(i+2)

]−1, i = 0, 1, . . . , Umax − 1. (6.9)

According to Bright and Taylor [16] the inverse exists and has only non-positiveelements so that R(i), given by (6.9), is the unique non-negative solution to (6.6).

Notice that, unlike [16], we do not need to truncate the iteration for large i, asthe structure of our multi-threshold queue guarantees the existence of Umax < ∞,or for Umax =∞ reduces to a single stage.

For a special class of multi-threshold queue the submatrices R(i)

(j,k) of R(i) canbe obtained efficiently by considering the block elements of the l.h.s. of (6.6). Thisresult is presented in Theorem 6.1.

Theorem 6.1. For a multi-threshold queue consisting of S stages such that

(i) F (i)

(j,k) = 0, for k < j and i = 0, 1, . . ., and

(ii) if B(i)

(j,k) 6= 0, for k < j, then L(i−1)

(x,x) = 0, for k < x ≤ j,

6.3 Steady-state analysis 67

the submatrices R(i)

(j,k) of R(i) are given by

R(i)

(j,j) = −F (i)

(j,j)

L(i+1)

(j,j) +

S∑b=j

R(i+1)

(j,b) B(i+2)

(b,j)

−1

, (6.10)

R(i)

(j,k) =

0, if k < j,

−

F (i)

(j,k) +

k−1∑a=j

S∑b=a

R(i)

(j,a)R(i+1)

(a,b)B(i+2)

(b,k)

·[L(i+1)

(k,k) +

S∑b=k

R(i+1)

(k,b)B(i+2)

(b,k)

]−1

, if k > j.

(6.11)

and

R(i)

(x,y) = 0 if B(i+1)

(j,k) 6= 0 for k < x ≤ y ≤ j. (6.12)

Proof. Assuming R(i+1) is an upper triangular block matrix one can verify that theunique solution to the block elements of the l.h.s. of (6.6), i.e.

0 = F (i)

(j,k) +

S∑a=1

R(i)

(j,a)L(i+1)

(a,k) +

S∑a=1

S∑b=1

R(i)

(j,a)R(i+1)

(a,b)B(i+2)

(b,k)

= F (i)

(j,k) +R(i)

(j,k)L(i+1)

(k,k) +

S∑a=1

S∑b=a

R(i)

(j,a)R(i+1)

(a,b)B(i+2)

(b,k) .

is given by (6.10), (6.11) and (6.12). Since R is a diagonal block matrix this provesby induction that R(i), i = 0, 1, . . ., is an upper triangular block matrix and that itssubmatrices are uniquely determined by (6.10), (6.11) and (6.12).

The conditions of Theorem 6.1 can be interpreted as (i) at upper thresholds thestage of the system can only change to higher stages, and (ii) at lower thresholdsthe stage of the system can change to higher stages and to at most one lower stage.If at level i the stage of the system changes from s to t, with t < s, then all stages,r = t+ 1, . . . , s− 1 must not be potential stage for level i− 1.

Remark 6.1 (Upper triangularity of R(i)). Note that under the conditions of The-orem 6.1, R(i) must be an upper triangular block matrix for all i. This implies thatonly stage 1 has no lower threshold.

To prove this, we extend the interpretation of R(i) to the product R(i)R(i+1).Observe that the element

[R(i)R(i+1)

](r,t)

describes the mean sojourn time in state

(i+2, t) per unit sojourn time in state (i, r) before returning to level i, given that theprocess started in state (i, r). If the element

[R(i)R(i+1)

](r,t)

= 0 then state (i+ 2, t)

cannot be reached from state (i, r) without visiting level i. The same interpretation


holds for the submatrices of the product

R(n) =

n−1∏i=0

R(i).

If the submatrix R(n)(j,k) of R(n) is 0, then stage k at level n can never be reachedfrom stage j at level 0. Under the conditions of Theorem 6.1, R(i) is an uppertriangular block matrix for i ≥ 0, therefore, R(n) is also an upper triangular blockmatrix for n ≥ 0. Suppose now that stage j 6= 1 has no lower threshold, then stagesk < j can never be reached from stage j since R(n)(j,k) = 0 for k < j and n ≥ 0.This implies that stages k < j can be removed from the threshold policy. Since theMarkov chain is irreducible, j = 1.

In Corollary 6.1, we provide an efficient algorithm to compute the stationaryqueue length vectors πi, i = 0, 1, . . ., using the submatrices of R(i) defined in Theo-rem 6.1 and equation (6.8).

Corollary 6.1. Define the vector pi =[p1i p2

i · · · pSi]

for i = 0, 1, . . . such that

pji =

j∑

a=1

pai−1R(i−1)

(a,j) , i = 1, . . . , Umax,

pjUmax [R(j,j)]i−Umax , i = Umax + 1, Umax + 2, . . . ,

(6.13)

with p10 the solution to

p10

[L(0)

(1,1) +

S∑a=1

R(0)

(i,a)B(1)

(a,i)

]= 0, (6.14)

such thatp1

0e = 1, (6.15)

and pj0 = 0 for j = 2, . . . , S. Under the conditions of Theorem 6.1, the stationaryprobability vector, πi =

[π1i π2

i · · · πSi], is given by

πji =pji∑Sk=1 βk

, (6.16)

with

βk =

Uk∑i=Lk

pki e, if Uk <∞,

Umax−1∑i=Lk

pki e+ pkUmax [I −R(k,k)]−1e, if Uk =∞,

where e is a vector of ones and I the identity matrix of appropriate size.

6.4 Examples 69

Proof. From (6.13) is follows that

pi = pi−1R(i−1),

and from (6.16)

πi = πi−1R(i−1).

At level 0, only stage 1 is active (see Remark 1), it then follows from (6.14) that

p0

[L(0) +R(0)B(1)

]= 0,

and that

π0

[L(0) +R(0)B(1)

]= 0.

Stability of the multi-threshold queue guarantees that

S∑j=1

∞∑i=0

pjie =∑

j : Uj<∞

Uj∑i=Lj

pjie+∑

j : Uj=∞

Umax−1∑i=Lj

pjie+

∞∑i=Umax

pjie

=

∑j : Uj<∞

βj +∑

j : Uj=∞

Umax−1∑i=Lj

pjie+ pjUmax

∞∑i=0

[R(j,j)]ie

=

∑j : Uj<∞

βj +∑

j : Uj=∞

Umax−1∑i=Lj

pjie+ pjUmax [I −R(j,j)]−1e

=

S∑j=1

βj <∞,

and that π is the stationary queue length distribution.

Remark 6.2 (Permutations of stages). Consider a multi-threshold queue with Sstages. If there exists a permutation of the S stages such that the conditions ofTheorem 6.1 hold, its stationary queue length vector can efficiently be obtainedusing this permutation and the results from Theorem 6.1 and Corollary 6.1.

6.4 Examples

In this section expressions for R(i)

(j,k) and the stationary queue length distribution πjiare obtained for three multi-threshold queueing systems. These expressions followusing Theorem 6.1 and Corollary 6.1 and are obtained by straightforward but tediousderivations. The three multi-threshold queueing systems we will consider are themulti-threshold queue from Figure 6.1, the staircase multi-threshold with exponentialservice and arrival rates from [61] and the staircase multi-threshold queue in a generalsetting from [21].


6.4.1 Extended traffic model

Consider the multi-threshold queue in Figure 6.1. Observe that the threshold pol-icy in Figure 6.1 satisfies both conditions of Theorem 6.1. In this multi-thresholdqueueing system, inspired by the traffic model in [9], we assume that

0 = L1 < L3 < L2 = L4 < U1 = U3 < U2 < U4 =∞and we define ρi = λi

µi. Note that by assuming exponential arrival and service rates,

each submatrixR(i)

(j,k) reduces to a single element. Therefore, the solution to equation(6.8) is ρ4 and each submatrix R(i)

(j,k) is given by:

ρ1, i = 0, . . . , L3 − 2,

R(i)

(1,1) =

ρ1

(1−ρU1−i

1

)(ρU2−U12 −ρU2−L2+2

2

)+(

1−ρU1−L2+21

)(1−ρU2−U1

2

)(

1−ρU1+1−i1

)(ρU2−U12 −ρU2−L2+2

2

)+(

1−ρU1−L2+21

)(1−ρU2−U1

2

) ,i = L3 − 1, . . . , L2 − 2,

ρ1−ρU1+1−i1

1−ρU1+1−i1

, i = L2 − 1, . . . , U1 − 1,

R(i)

(1,2) = λ1

µ2

(ρU1−i1 −ρU1−i+1

1

)(1−ρU2−U1

2

)(

1−ρU1+1−i1

)(1−ρU2+1−i

2

) , i = L2 − 1, . . . , U1,

R(i)

(1,3) = λ1

µ3

(ρU1−i1 −ρU1+1−i

1

)(ρU2−U12 −ρU2−L2+2

2

)(

1−ρU1+1−i1

)(ρU2−U12 −ρU2−L2+2

2

)+(

1−ρU1−L2+21

)(1−ρU2−U1

2

) ,i = L3 − 1, . . . , L4 − 2,

R(i)

(1,4) = λ1

µ4

(ρU1−i1 −ρU1+1−i

1

)(ρU2−U12 −ρU2+1−i

2

)(

1−ρU1+1−i1

)(1−ρU2+1−i

2

) , i = L4 − 1, . . . , U1,

R(i)

(2,2) =ρ2−ρU2+1−i

2

1−ρU2+1−i2

, i = L2, . . . , U2 − 1,

R(i)

(2,3) = 0, ∀i,

R(i)

(2,4) = λ2

µ4

ρU2−i2 −ρU2+1−i

2

1−ρU2+1−i2

, i = L2, . . . , U2,

ρ3, i = L3, . . . , L4 − 2,

R(i)

(3,3) =

ρ3−ρU3+1−i3

1−ρU3+1−i3

, i = L4 − 1, . . . , U3 − 1,

R(i)

(3,4) = λ3

µ4

ρU3−i3 −ρU3+1−i

3

1−ρU3+1−i3

, i = L4 − 1, . . . , U3,

R(i)

(4,4) = ρ4, i = L4, L4 + 1, . . . .

The stationary queue length probability of i customers in stage j, πji , follows from

Corollary 6.1 by normalising pji . For i = 0:

1, j = 1,pj0 =

0, j 6= 1,

6.4 Examples 71

and for i > 0:

p1i = p1

i−1R(i−1)

(1,1) , 0 < i ≤ U1,

p1i−1R

(i−1)

(1,2) , i = L2,

p2i =

p1i−1R

(i−1)

(1,2) + p2i−1R

(i−1)

(2,2) , L2 < i ≤ U1 + 1,

p2i−1R

(i−1)

(2,2) , U1 + 1 < i ≤ U2,

p1i−1R

(i−1)

(1,3) , i = L3,

p3i =

p1i−1R

(i−1)

(1,3) + p3i−1R

(i−1)

(3,3) , L3 < i ≤ L4 − 1,

p3i−1R

(i−1)

(3,3) , L4 − 1 < i ≤ U3,

p4i =

p1i−1R

(i−1)

(1,4) + p3i−1R

(i−1)

(3,4) , i = L4,

p1i−1R

(i−1)

(1,4) + p2i−1R

(i−1)

(2,4) + p3i−1R

(i−1)

(3,4) + p4i−1R

(i−1)

(4,4) , L4 < i ≤ U1 + 1,

p2i−1R

(i−1)

(2,4) + p4i−1R

(i−1)

(4,4) , U1 + 1 < i ≤ U2 + 1,

p4U2+1

[R(U2+1)

(4,4)

]i−U2−1, U2 + 1 < i.

6.4.2 Le Ny and Tuffin [61]

Consider a multi-threshold queue of S stages as analysed by Le Ny and Tuffin in[61]. In each stage i arrivals are Poisson distributed with rate λi, service times areexponentially distributed with rate µi and we define ρi = λi

µi. An arrival changes the

stage from j to j + 1 at Uj and a departure changes the stage from j to j − 1 at Lj .We assume

0 = L1 < L2 < · · · < LS ≤ U1 < · · · < US−1 < US =∞.

The state diagram created by this threshold policy forms a staircase as schematicallyshown in Figure 6.2.

Figure 6.2: Schematic representation of the state diagram of a staircase thresholdpolicy with 4 stages.

As in Section 6.4.1 each submatrixR(i)

(j,k) consists of a single element and equation(6.7), and in particular (6.8), gives

R(Umax)

(S,S) = ρS .

Both conditions of Theorem 6.1 are satisfied by the threshold policy and R(i)

(j,k) is


given by:

ρj , Lj ≤ i ≤ Lj+1 − 2,

ρj−ρUj+1−ij

1−ρUj+1−ij

, Lj+1 − 1 ≤ i ≤ Uj ,R(i)

(j,j) =

R(i)

(S,S) = ρS , LS ≤ i,λjµk

ρUj−ij −ρUj+1−i

j

1−ρUj+1−ij

·∏k−1a=j+1

ρUa−Ua−1a −ρUa+1−i

a

1−ρUa+1−ia

, Lk − 1 ≤ i ≤ Lk+1 − 2,

R(i)

(j,k) =

λjµk

(ρUj−ij −ρUj+1−i

j

)(1−ρUk−Uk−1

k

)(

1−ρUj+1−ij

)(1−ρUk+1−i

k

)·∏k−1

a=j+1ρUa−Ua−1a −ρUa+1−i

1−ρUa+1−ia

, Lk+1 − 1 ≤ i ≤ Uj ,

R(i)

(j,S) =λjµS

ρUj−ij −ρUj+1−i

j

1−ρUj+1−ij

∏S−1a=j+1

ρUa−Ua−1a −ρUa+1−i

a

1−ρUa+1−ia

, LS − 1 ≤ i.

The stationary queue length distribution πji follows from Corollary 6.1 by normalis-

ing pji . For i = 0:

pj0 =

1,

0,

j = 1,

j 6= 1,

for i > 0 and j = 1 or j = 2:

p1i = p1

i−1R(i−1)

(1,1) , 0 < i ≤ U1, (6.17)

p2i =

p1i−1R

(i−1)

(1,2) ,

p1i−1R

(i−1)

(1,2) + p2i−1R

(i−1)

(2,2) ,

p2i−1R

(i−1)

(2,2) ,

i = L2,

L2 < i ≤ U + 1,

U + 1 < i ≤ U2,

(6.18)

for i > 0 and j = 3, . . . , S − 1:

pji =

∑j−1a=1 p

ai−1R

(i−1)

(a,j) ,∑ja=1 p

ai−1R

(i−1)

(a,j) ,∑ja=k p

ai−1R

(i−1)

(a,j) ,

pji−1R(i−1)

(j,j) ,

i = Lj ,

Lj < i ≤ U1 + 1,

Uk−1 + 1 < i ≤ Uk + 1, k = 2, . . . , j − 1,

Uj−1 + 1 < i ≤ Uj ,(6.19)

6.4 Examples 73

and for i > 0 and j = S

pSi =

∑S−1a=1 p

ai−1R

(i−1)

(a,S) ,∑Sa=1 p

ai−1R

(i−1)

(a,S) ,∑Sa=k p

ai−1R

(i−1)

(a,S) ,

pSi−1

[R(i−1)

(S,S)

]i−Umax,

i = LS ,

LS < i ≤ U1 + 1,

Uk−1 + 1 < i ≤ Uk + 1, k = 2, . . . , S − 1,

Umax < i.

(6.20)

6.4.3 Choi et al. [21]

Consider the multi-threshold queue of S stages as analysed by Choi et al. [21]. Thismodel generalises the staircase model of [61] to PH(Λs, λs) arrivals and PH(Ms, µs)services in stage s. The forward, local and backward transition matrices are given by(6.3), (6.4) and (6.5) respectively. In this case, the submatrices R(i)

(j,k) are not singleelements and the matrix equation (6.8) must be solved numerically. The submatricesR(i)

(j,k), i = 0, . . . , Umax − 1, are iteratively given, following Theorem 6.1, by

R(i)

(j,j) =

− F (i)

(j,j)

[L(i+1)

(j,j) +R(i+1)

(j,j) B(i+2)

(j,j)

]−1, Lj ≤ i < Uj − 1, i 6= Lj+1 − 2,

− F (i)

(j,j)

[L(i+1)

(j,j) +∑j+1

b=jR(i+1)

(j,b) B(i+2)

(b,j)

]−1,

i = Lj+1 − 2,

− F (i)

(j,j)

[L(i+1)

(j,j)

]−1, i = Uj − 1,

0, otherwise,

−[∑k−1

a=jR(i)

(j,a)R(i+1)

(a,k)B(i+2)

(k,k)

]·[L(i+1)

(k,k) +R(i+1)

(k,k)B(i+2)

(k,k)

]−1, Lk − 1 ≤ i < Uj , i 6= Lk+1 − 2,

−[∑k+1

b=k

∑k−1

a=jR(i)

(j,a)R(i+1)

(a,b)B(i+2)

(b,k)

]R(i)

(j,k) =

·[L(i+1)

(k,k) +∑k+1

b=kR(i+1)

(k,b)B(i+2)

(b,k)

]−1, i = Lk+1 − 2,

−[F (i)

(j,k)1k=j+1 +∑k−1

a=j+1R(i)

(j,a)R(i+1)

(a,k)B(i+2)

(k,k)

]·[L(i+1)

(k,k) +R(i+1)

(k,k)B(i+2)

(k,k)

]−1, i = Uj ,

0, otherwise,

for j = 1, . . . , S − 1, and

R(S,S), LS ≤ i,R(i)

(S,S) =

0, otherwise.

The stationary queue length distribution πji follows from Corollary 6.1 by normal-

ising pji . The vectors pji , i > 0, are given by equations (6.17), (6.18), (6.19) and

(6.20). Finally, p10 is obtained from (6.14) and (6.15) and pj0 = 0, j > 1.


6.5 Fundamental diagram of traffic

In Chapter 3 we introduced two special cases of the PH/PH/1 multi-thresholdqueue, namely the two-stage M/M/1 threshold queue and the four-stage M/M/1feedback threshold queue, to determine the fundamental diagram of traffic. In thissection we will obtain the fundamental diagram using a two-stage PH/PH/1 thresh-old queue, and we perform a sensitivity analysis on the distributions of the arrivalprocess and the service processes.

Traffic flows are characterised by three parameters describing the speed, v, flow,q, and density, k, see Section 2.2.2. The first relation between these parameters isgiven by

q = k · v.The second relation is introduced by Heidemann in [41] and was adjusted to copewith finite queueig models in Section 3.2. Recall from (3.5) that

k = [1− π0]C, v =1/C

E[S], q = k · v =

1− π0

E[S]. (6.21)

For traffic modelling, the assumption of Poisson arrivals and exponential servicesare far from realistic. Therefore, we consider the PH/PH/1 threshold queue in whichboth the interarrival times and service times are of phase-type. The phase-typedistribution allows us to approximate any distribution with non-negative supportarbitrarily close, see [48]. Combining the results from this chapter with the equationsin (6.21) we obtain the fundamental diagram for a two-stage PH/PH/1 thresholdqueue in Figure 6.3.

Figure 6.3 characterises the fundamental diagram of the two-stage PH/PH/1threshold queue for four different scenarios. We select phase-type distributions suchthat the and mean service times, E [SH ] = 1

25 in the non-congested stage (stage 1)and E [SL] = 1

15 in the congested stage (stage 2), are equal in each scenario. We studythree different distributions, the Hyper-Exponential distribution with four phases,H4, the Exponential distribution, M , and the Erlang distribution with 4 phases, E4,with respectively c2H4

= 1.5744, c2M = 1, and c2E4= 0.25. Furthermore, we set L = 5

and U = 15.In Figure 6.3(a) we vary PH(ML,µL), the distribution of the congested service

process. We set PH(Λ,λ) = H4 and PH(MH ,µH) = H4. It can be seen thatthe fundamental diagrams for the three different distributions of PH(ML,µL) aresimilar. This implies that the coefficient of variation of PH(ML,µL) has minorinfluence on the fundamental diagram.

In Figure 6.3(b) we vary PH(Λ,λ), the distribution of the arrival process. Weset PH(MH ,µH) = H4 and PH(ML,µL) = H4. We observe that increasingvariability in the arrival process reduces speed and flow for density k < 0.6. Fork > 0.6, flows and speeds increase when the variability decreases.

In Figure 6.3(c) we vary PH(MH ,µH), the distribution of the non-congestedservice process. We set PH(MH ,µH) = H4 and PH(Λ,λ) = E4. Here the effects

6.5 Fundamental diagram of traffic 75

0 0.5 10

2

4

6

Density (k)

Flow

(q)

(a)

E4

MH4

0 0.5 10

3

6

9

Density (k)

Flow

(q)

(b)

E4

MH4

0 0.5 10

3

6

9

12

15

Density (k)

Flow

(q)

(c)

E4

MH4

0 0.5 10

3

6

9

Density (k)

Flow

(q)

(d)

E4

MH4

Figure 6.3: Flow-Density diagram for the two-stage PH/PH/1 threshold queue byselecting various distributions for the (a) congested service process, (b) arrival process,(c) and (d) non-congested service process.

of the Erlang distribution are clearly visible in the fundamental diagram. In the casewhere both the arrival process and the non-congested service process are Erlang withfour phases, the fundamental diagram reaches a maximum density of 0.6. This isa result of the low probability of reaching the congested stage caused by the lowvariability in the E4 distribution. Note that for deterministic arrival and serviceprocesses the congested stage is never reached. The maximum density of 0.6 isobtained for a mean interarrival time close to 1/15, the mean service time in thecongested stage. For this value the queue is still stable and the density is 15/25 = 0.6.

In Figure 6.3(d) we vary PH(MH ,µH), the distribution of the non-congestedservice process. We set PH(MH ,µH) = H4 and PH(Λ,λ) = H4. The results are


similar to those for Figure 6.3(b). Increasing variability of the non-congested serviceprocess decreases flow and speeds for k < 0.6. For k > 0.6, flows and speeds increasewhen the variability decreases.


We introduced the PH/PH/1 multi-threshold queue where the arrival process andservice process are controlled by a threshold policy. The threshold policy determines,based on the queue length, the stage of system, and the stage determines the arrivaland service processes. We modelled this queue as a LDQBD and obtained thestationary queue length probabilities using Matrix analytic methods.

A special class of multi-threshold queues is presented and explicit description ofthe R-matrices has been obtained in terms of its submatrices. This decompositiontheorem allows for computation of each R-submatrix as well as the stationary queuelength probability vectors.

The service distribution in the congested stage has minor influence on the fun-damental diagram. In contrast, increasing variability in the arrival process or non-congested service process is shown to reduce both speed and flow for a density lessthan the capacity drop density.

CHAPTER 7

A successive censoring algorithm for a

system of connected LDQBD-processes

7.1 Introduction

We consider a class of Markov chains in which the state space can be partitionedinto sets. Transitions within each set constitute a finite Level Dependent Quasi-Birth-and-Death process (LDQBD), and the transitions between sets follow a specialstructure. This way, we create a system of connected LDQBDs on the whole statespace. Such a system of connected LDQBDs often occurs in queueing systems withhysteresis in both traffic, see Baer, Boucherie and van Ommeren [9] or Chapter 3,and telecommunication systems, see Le Ny and Tuffin [61]. To obtain the stationarydistribution of such a Markov chain, we present a successive censoring algorithmbased on the censoring algorithm by Kemeny and Snell [54] for discrete time Markovchains. This successive censoring algorithm allows for easy computation of the sta-tionary distribution of a network of multi-threshold queues as introduced by Baer,Boucherie and van Ommeren in [9].

The concept of the successive censoring algorithm is not new, Gaver, Jacobsand Latouche [34], use the same approach to determine the stationary distributionof a LDQBD with a finite number of levels. Our work extends the work of Gaver,Jacobs and Latouche [34] to more general transitions, see Remark 7.1. The censoringalgorithm [54] also forms the base for the folding algorithm in Ye and Li [106] and Liand Sheng [66], where the stationary distribution of a finite QBD was obtained bysequentially splitting (and renumbering) the state space in odd and even numberedsets, followed by application of the censoring algorithm to the two resulting subsets.

In the literature, the censoring algorithm is also called exact aggregation/dis-aggregation algorithm in which the state space is aggregated to obtain a smaller(and easier to solve) Markov chain. The stationary distribution for this aggregatedMarkov chain is then disaggregated to obtain the stationary distribution of the fullMarkov chain. Most recent is the work of Katehakis and Smit [52] and Katehakis,Smit and Spieksma [53]. In [52], a Markov chain is studied in which the state spaceis partitioned in sets, without any restrictions on the transitions within a set. Intheir successive lumping procedure it is crucial that a set contains a single entrancestate, i.e., a single state through which the set can be reached from other sets. Our

78 7. A successive censoring algorithm

work extends this aggregation method by allowing multiple entrance states, underthe restriction that the transitions within a set form a QBD. The work in [52] isapplied to Quasi-Skip Free Processes to the left in [53] where it is assumed thatlower levels are entered via a single entrance state only. The single entrance statesin [52, 53] are called mandatory states in Kim and Smith [55] and input states inFeinberg and Chui [32] in which a parallel lumping procedure was introduced.

For a thorough overview and comparison of several aggregation/disaggregationalgorithms see Cao and Stewart [18], Haviv [39], Kafeety, Meyer and Stewart [51]and Rogers and Plante [91].

Section 7.2 introduces the system of connected LDQBDs as a three-dimensionalMarkov chain and specifies the exact restrictions on the transitions between theLDQBDs. In Section 7.3 we present the successive censoring algorithm to determinethe stationary distribution of the system of connected LDQBDs. In Section 7.4 wegive an algorithm which determines if the successive censoring algorithm can beapplied for a given Markov chain. We perform a complexity analysis in Section 7.5and the algorithm is demonstrated with an example in Section 7.6. Section 7.7 givesconcluding remarks.


We consider a three-dimensional Markov chain X , describing a system of connectedLevel Dependent Quasi-Birth-and-Death processes (LDQBD), with states (s, l, p)where s denotes the set, l denotes the level and p denotes the phase of a singlestate. Let Q be its infinitesimal generator. Each set ωs, s = 1, . . . , S, has Ls levelslabelled ψsl , l = 1, . . . , Ls, and each level ψsl in ωs has Ps,l phases labelled ξs,lp , p =1, . . . , Ps,l. The transitions within each set ωs, s = 1, . . . , S, constitute a transientLDQBD and are described by the transient generator Qs,s, a submatrix of Q, inwhich the states are ordered lexicographically, i.e., (s, 1, 1),(s, 1, 2),. . .,(s, 1, Ps,1),(s, 2, 1),. . .,(s, 2, Ps,2),. . .,(s, Ls, 1),. . .,(s, Ls, Ps,Ls). The (transient) generator Qs,s

has a tri-diagonal block structure and is given by

Qs,s =

L(1)

s F (1)

s 0 · · · · · · 0

B(2)

s L(2)

s F (2)

s

. . ....

0 B(3)

s

. . .. . .

. . ....

.... . .

. . .. . .

. . . 0...

. . .. . . L(Ls−1)

s F (Ls−1)

s

0 · · · · · · 0 B(Ls)

s L(Ls)

s

. (7.1)

Here, F (l)

s describes the transitions from ψsl to ψsl+1, B(l)

s describes the transitionsfrom ψsl to ψsl−1, and L(l)

s describes the transitions within ψsl .The transitions between two sets ωi and ωj , i 6= j, are governed by two sets

of conditions, labelled direct and indirect conditions. These conditions ensure that


the tri-diagonal structure within each set is maintained throughout the successivecensoring algorithm that is introduced in Section 7.3. A specific part of the algorithmwill require the determination of the inverse of the transient generator Qs,s. If thetri-diagonal block structure is maintained throughout the algorithm, this inverse canbe obtained using the results by Shin [92] on the fundamental matrix of a transientLDQBD, ot the results by Choi et al. [21] on the fundamental matrix of a transientQBD. These results are also presented in Chapter 5.

We denote by Qi,j , i, j = 1, . . . , S, the submatrix of Q with transitions from ωito ωj , and by

[Qi,j

]a,b

, a = 1 . . . , Li and b = 1 . . . , Lj , the submatrix of Qi,j with

transitions from ψia to ψjb .

Definition 7.1. The direct conditions describe the one step transitions betweenωi and ωj . We define six sets of transitions that can occur between ωi and ωj for(i < j):

T1(z): Transitions from any level ψil , l = 1, . . . , Li, to only ψjz and ψjz+1, andback, i.e., [

Qi,j

]a,b

= 0, if b 6= z and b 6= z + 1,[Qj,i

]b,a

= 0, if b 6= z and b 6= z + 1,

for z = 1, . . . , Lj − 1.

T2(z): Transitions from any level ψil , l = 1, . . . , Li, to only ψjz, and transitions

from levels ψjz−1, ψjz, and ψjz+1 to any level ψil , l = 1, . . . , Li, i.e.,[Qi,j

]a,b

= 0, if b 6= z,[Qj,i

]b,a

= 0, if b 6= z − 1, b 6= z and b 6= z + 1,

for z = 2, . . . , Lj − 1.

T3(z): Transitions from any level ψil , l = 1, . . . , Li, to only ψjz−1, ψjz, and ψjz+1,and transitions from level ψjz to any level ψil , l = 1, . . . , Li, i.e.,[

Qi,j

]a,b

= 0, if b 6= z − 1, b 6= z and b 6= z + 1,[Qj,i

]b,a

= 0, if b 6= z,

for z = 2, . . . , Lj − 1.

T4: Only transition from ωi to ωj , i.e., Qj,i = 0.

T5: Only transition from ωj to ωi (reversed T4 transition), i.e., Qi,j = 0.

T6: No transitions between ωi and ωj , i.e., Qi,j = 0 and Qj,i = 0.


Note that T4 and T5 are mutually exclusive, except for the trivial case of allzero, but that other sets may have a non-empty intersection, for example T2(x) andT3(y) for x = y, and T1(x) and T4, etc.

These six sets of transitions are shown in an example in Figure 7.1. In this smallexample we consider a network of connected LDQBDs and focus on ωi and ωj , eachwith 5 levels, and their one-step transitions. For each of the six sets of transitionsfrom Definition 7.1 we present a schematic view of the generator. In this schematicview we depict a (possibly) non-zero submatrix by a light gray square. The darkgray squares depict the one-step transitions between ωi and ωj . The white squaresdepict zero-submatrices. In Figure 7.1 it is shown that in a T1(z) transition thereare transitions from any level in ωi to ψjz and ψjz+1, and back. Figure 7.1 also showsthat a T5 transition only consists of transitions from ωj to ωi. Figure 7.1 makesit easy to visualise how the intersection of T1(z) and T5, with transitions from ψjzand ψjz+1 to any level in ωi but none back, looks like. Finally, observe that a T6transition is the trivial all-zero intersection of the other five sets of transitions.

ψjz

ψjz

ωi

ωj

ωi ωj

T1(z)

ψjz

ψjz

ωi

ωj

ωi ωj

T2(z)

ψjz

ψjz

ωi

ωj

ωi ωj

T3(z)

ωi

ωj

ωi ωj

T4

ωi

ωj

ωi ωj

T5

ωi

ωj

ωi ωj

T6

Figure 7.1: Schematic representation of the generators corresponding to each of thesix types of transitions between ωi and ωj .

Definition 7.2. Indirect conditions describe the multiple step paths between ωi andωj . We define a lower path from ωi to ωj as a path from ωi to ωj only passing


through sets with index less than min i, j. Based on the one step transitions be-tween ωi and ωj , (i < j) in Definition 7.1, we define the following indirect conditions

i. If there is a T1(z) transition between ωi and ωj , z = 1, . . . , Lj − 1, then:

a. Each lower path from ωi to ωj must end with a T1(z) transition, and,

b. Each lower path from ωj to ωi must start with a T1(z) transition.

ii. If there is a T2(z) transition between ωi and ωj , z = 2, . . . , Lj − 1, then:



iii. If there is a T3(z) transition between ωi and ωj , z = 2, . . . , Lj − 1, then:



iv. If there is a T4 transition between ωi and ωj , then there cannot be a lower pathfrom ωj to ωi.

v. If there is a T5 transition between ωi and ωj , then there cannot be a lower pathfrom ωi to ωj .

vi. If there is a T6 transition between ωi and ωj , then either:

a. For some z ∈ 1, . . . , Lj − 1, all lower paths from ωj to ωi start witha T1(z) transition and all lower paths from ωi to ωj end with a T1(z)transition, or,

b. For some z ∈ 2, 3, . . . , Lj − 1, all lower paths from ωj to ωi start witha T2(z) transition and all lower paths from ωi to ωj end with a T2(z)transition, or,

c. For some z ∈ 2, 3, . . . , Lj − 1, all lower paths from ωj to ωi start witha T3(z) transition and all lower paths from ωi to ωj end with a T3(z)transition, or,

d. There can be one or more lower paths from ωi to ωj , but none from ωj toωi, or,

e. There can be one or more lower paths from ωj to ωi, but none from ωi toωj , or,

f. There are no lower paths between ωi and ωj .

There are many lower paths from ωi to ωj which makes it difficult to checkwhether or not Definition 7.2 holds for a given Markov chain. Algorithm 7.2 isa simplified version of the successive censoring algorithm, given by Algorithm 7.1,which quickly checks if Definition 7.2 holds for a given Markov chain.


Remark 7.1 (Special cases). We will briefly discuss the relation between our modeland the models discussed in Katehakis and Smit [52] and Gaver, Jacobs and La-touche [34]. In Gaver, Jacobs and Latouche [34], a successive censoring algorithmis presented to find the stationary distribution of a LDQBD. Assuming S = 1 werestrict our model to a single set and we obtain a LDQBD. In this special case, oursuccessive censoring algorithm is the same as the successive censoring algorithm ofGaver, Jacobs and Latouche [34].

In Katehakis and Smit [52] a successive lumping procedure is presented for aspecial class of Markov chains. Important is that the state space can be partitionedinto sets and that in each set there is only one single entrance state, a state throughwhich the set is entered. Note that there are no restrictions for the transitionswithin a set. By assuming that all levels consist of a single phase, and by restrictingto transitions from and to a single level in ωj , i.e., the intersection of a T2(x) andT3(x) transition, x = 1, . . . , Lj , for each set ωj , j = 1, . . . , S, we obtain a specialcase of both our model and the model by Katehakis and Smit [52].

7.3 Successive censoring algorithm

Let π = [ π1 π2 · · · πS ] denote the stationary distribution of the three-dimensio-nal Markov chain X such that πQ = 0 and πe = 1 and let πi denote the probabilityvector of being in ωi, i = 1, . . . , S. We obtain π by using a successive censoring algo-rithm based on the censoring algorithm in Kemeny and Snell [54]. In the censoringalgorithm the state space of an arbitrary Markov chain Y is first split into subsetsA and B such that its generator T and stationary distribution ν can be partitionedfollowing:

T =

[TA TAB

TBA TB

], ν = [ νA νB ] .

Then a reduction step occurs in which transitions from B to B via A are projectedonto transitions within B creating the generator T ∗B

T ∗B = TB + TBA [−TA]−1TAB . (7.2)

During an intermediate step the stationary distribution νB is determined by solving:

νBT∗B = 0,

and is used in the expansion step the determine νA

νA = νBTBA [−TA]−1. (7.3)

The successive censoring algorithm consists of S−1 reduction steps (7.2), one in-termediate step, and S−1 expansion steps (7.3). In reduction step k, k = 1, . . . , S−1,the generatorQk is reduced toQk+1 by removing ωk from the state space (censoring).Observe that following this definition, Q1 = Q. In the intermediate step, the station-ary distribution of QS is determined. Next, in expansion step k, k = 1, . . . , S−1, the

7.3 Successive censoring algorithm 83

stationary distribution is expanded by adding ωS−k, the set with highest index stillcensored, back to the state space. Finally, by normalising the resulting vector, weobtain the stationary distribution π. The successive censoring algorithm is formallydescribed in Algorithm 7.1.

Algorithm 7.1 (The Successive Censoring Algorithm).

1. Reduce the state space in S − 1 reduction steps. In reduction step k, ωkis removed from the state space and the generator Qk is reduced to Qk+1

following

Qk+1i,j = Qk

i,j +Qki,k

[−Qk

k,k

]−1

Qkk,j . (7.4)

2. Determine the stationary distribution P S of QS , such that P SQS = 0 andP Se = 1.

3. Expand P S in S − 1 expansion steps. In expansion step k, ωS−k is added tothe state space and the vector P S−k+1 =

[pk+1 pk+2 · · · pS

]is expanded

to P S−k where the vector pk is obtained following

pk =

k∑i=1

pS−k+iQS−kS−k+i,S−k

[−QS−k

S−k,S−k

]−1

. (7.5)

4. Normalise P 1 to obtain the stationary distribution π of the Markov chain X ,i.e.,

π =P 1∣∣P 1∣∣ .

EachQj,j , j = 1, . . . , S describes a transient LDQBD and due to the irreducibility

assumption the negative inverse[−Qj,j

]−1, or fundamental matrix of Qj,j , exists

and describes the sojourn time in ωj before transition to some other ωi, i 6= j.

Let[−Qj,j

]−1

a,b, a, b = 1 . . . , Lj , denote the submatrix of

[−Qj,j

]−1describing the

average time spent in ψjb before the Markov process leaves ωj , given that it enteredωj through ψja. The fundamental matrix of Qj,j is given by Shin in [92] and isdiscussed in detail in Section 5.5. It will be proven in Theorem 7.1 and Theorem 7.2that Qk

j,j , k = 1, . . . , S − 1 and j = k, . . . , S, is also a transient LDQBD such thatthe results of Shin in [92] on the fundamental matrix of a transient LDQBD can beused throughout the algorithm.

In the special case where Qkj,j would describe a Quasi-Birth-and-Death process

(QBD), the results on the fundamental matrix of Choi et al. in [21] and in Section 5.4can be applied.

The remainder of this section discusses the first three steps of Algorithm 7.1 indetail.


7.3.1 Reduction step k

In reduction step k, the generator Qk is reduced to Qk+1 by removing ωk from thestate space. Observe that ωk is the set with the smallest index in Qk. Following thereduction step (7.2) we obtain for i, j > k

Qk+1i,j = Qk

i,j +Qki,k

[−Qk

k,k

]−1

Qkk,j .

Decomposing these submatrices by their levels, for i = j > k, gives:

[Qk+1i,i

]x,y

=[Qki,i

]x,y

+

Lk∑a=1

Lk∑b=1

[Qki,k

]x,a

[−Qk

k,k

]−1

a,b

[Qkk,i

]b,y. (7.6)

In this reduction step transitions from ωi to ωi via ωk are projected onto transitionswithin ωi. For example, a T1(z) transition from ωk to ωi is projected onto transitionswithin and between ψiz and ψiz+1 and in this case (7.6) can be reduced since[

Qk+1i,i

]x,y

=[Qki,i

]x,y

, if x, y /∈ z, z + 1.

We rewrite (7.6) as

[Qk+1i,i

]x,y

=

[Qki,i

]x,y

+∑Lka=1

∑Lkb=1

[Qki,k

]x,a

[−Qk

k,k

]−1

a,b

[Qkk,i

]b,y,

if x ∈ R1 and y ∈ R2,[Qki,i

]x,y

, otherwise. (7.7)

Here, R1, R2 ⊆ 1, . . . , Lk depend on the type of transition between ωk and ωi andare given in Table 7.1. When R1 = R2 = z, z + 1 all transitions between ωk andωi are projected onto transitions within and between ψiz and ψiz+1. Observe thatT4, T5 and T6 transitions are not projected onto transitions within ωi since thereare no transitions from ωi to ωi via ωk and[

Qk+1i,i

]x,y

=[Qki,i

]x,y

.

A similar decomposition as (7.6) applies for transitions between two sets ωi and ωjwith k < i < j

[Qk+1i,j

]x,y

=

[Qki,j

]x,y

+∑Lka=1

∑Lkb=1

[Qki,k

]x,a

[−Qk

k,k

]−1

a,b

[Qkk,j

]b,y,

if x ∈ S1 and y ∈ S2,[Qki,j

]x,y

, otherwise, (7.8)


T1(z) T2(z) T3(z)

R1 = z, z + 1 R1 = z R1 = z − 1, z, z + 1

R2 = z, z + 1 R2 = z − 1, z, z + 1 R2 = z

T4 T5 T6

R1 = ∅ R1 = ∅ R1 = ∅

R2 = ∅ R2 = ∅ R2 = ∅

Table 7.1: The subsets R1 and R2 for each type of transition from ωk to ωi (k < i).

and

[Qk+1j,i

]x,y

=

[Qkj,i

]x,y

+∑Lka=1

∑Lkb=1

[Qkj,k

]x,a

[−Qk

k,k

]−1

a,b

[Qkk,i

]b,y,

if x ∈ T1 and y ∈ T2,[Qkj,i

]x,y

, otherwise. (7.9)

The subsets S1, T2 ⊆ 1, . . . , Li and S2, T1 ⊆ 1, . . . , Lj depend on the transi-tions between ωk and ωj and between ωk and ωi. For i < j these ranges are given inTable 7.2 and Table 7.3. For example, suppose there are T1(x) transitions from ωkto ωi and T4 transitions from ωk to ωj (i < j). In reduction step k, these transitions

will be projected onto transitions from ψix and ψix+1 to any level ψjl , l = 1, . . . , Lj(S1 = x, x + 1 and S2 = 1, . . . , Lj), and no transitions from ωj to ωi (T1 = ∅and T2 = ∅).

Note that during reduction step k the transitions from (or via) ωk are projectedonto existing transitions (including T6 transitions) between sets ωi and ωj , i, j >k. Using this we can now formulate the following theorem relating the indirectconditions in Definition 7.2 to the direct conditions in Definition 7.1.

Theorem 7.1. The indirect conditions in Definition 7.2 ensure that the direct con-ditions in Definition 7.1 are preserved in each reduction step.

Proof. Observe that a lower path from ωi to ωj , i < j, is projected onto a directtransition from ωi to ωj in reduction steps 1, . . . , i−1. Therefore, following the orderin Definition 7.2, we can easily state that after reduction step i− 1:

i. The lower paths in Def. 7.2.i.a. and Def. 7.2.i.b. will be projected onto transi-


Transition from ωk to ωj (k < j)

T1(y) T2(y) T3(y)

Tra

nsi

tion

fromωk

toωi

(k<i)

T1(x)

S1 = x, x + 1 S1 = x, x + 1 S1 = x, x + 1

S2 = y, y + 1 S2 = y − 1, y, y + 1 S2 = y

T1 = y, y + 1 T1 = y T1 = y − 1, y, y + 1

T2 = x, x + 1 T2 = x, x + 1 T2 = x, x + 1

T2(x)

S1 = x S1 = x S1 = x

S2 = y, y + 1 S2 = y − 1, y, y + 1 S2 = y

T1 = y, y + 1 T1 = y T1 = y − 1, y, y + 1

T2 = x − 1, x, x + 1 T2 = x − 1, x, x + 1 T2 = x − 1, x, x + 1

T3(x)

S1 = x − 1, x, x + 1 S1 = x − 1, x, x + 1 S1 = x − 1, x, x + 1

S2 = y, y + 1 S2 = y − 1, y, y + 1 S2 = y

T1 = y, y + 1 T1 = y T1 = y − 1, y, y + 1

T2 = x T2 = x T2 = x

T4

S1 = ∅ S1 = ∅ S1 = ∅

S2 = ∅ S2 = ∅ S2 = ∅

T1 = y, y + 1 T1 = y T1 = y − 1, y, y + 1

T2 = 1, . . . , Li T2 = 1, . . . , Li T2 = 1, . . . , Li

T5

S1 = 1, . . . , Li S1 = 1, . . . , Li S1 = 1, . . . , Li

S2 = y, y + 1 S2 = y − 1, y, y + 1 S2 = y

T1 = ∅ T1 = ∅ T1 = ∅

T2 = ∅ T2 = ∅ T2 = ∅

Table 7.2: The subsets S1, S2, T1 and T2 for different types of transition between ωkand ωj and between ωk and ωi for k < i < j.


Transition from ωk to ωj (k < j)

T4 T5T

ran

siti

on

from

ωk

toωi

(k<i)

T1(x)

S1 = x, x + 1 S1 = ∅

S2 = 1, . . . , Lj S2 = ∅

T1 = ∅ T1 = 1, . . . , Lj

T2 = ∅ T2 = x, x + 1

T2(x)

S1 = x S1 = ∅

S2 = 1, . . . , Lj S2 = ∅

T1 = ∅ T1 = 1, . . . , Lj

T2 = ∅ T2 = x − 1, x, x + 1

T3(x)

S1 = x − 1, x, x + 1 S1 = ∅

S2 = 1, . . . , Lj S2 = ∅

T1 = ∅ T1 = 1, . . . , Lj

T2 = ∅ T2 = x

T4

S1 = ∅ S1 = ∅

S2 = ∅ S2 = ∅

T1 = ∅ T1 = 1, . . . , Lj

T2 = ∅ T2 = 1, . . . , Li

T5

S1 = 1, . . . , Li S1 = ∅

S2 = 1, . . . , Lj S2 = ∅

T1 = ∅ T1 = ∅

T2 = ∅ T2 = ∅

Table 7.3: The subsets S1, S2, T1 and T2 for different types of transition between ωkand ωj and between ωk and ωi for k < i < j.


tions from any level ψil , l = 1, . . . , Li, to ψjz and ψjz+1, and back, i.e.,[Qi−1i,j

]a,b

= 0, if b 6= z and b 6= z + 1,[Qi−1j,i

]b,a

= 0, if b 6= z and b 6= z + 1,

for z = 1, . . . , Lj − 1, preserving the T1(z) transitions.

ii. The lower paths in Def. 7.2.ii.a. and Def. 7.2.ii.b. will be projected ontotransitions from any level ψil , l = 1, . . . , Li, to ψjz−1, ψjz and ψjz+1, and fromψjz to any level ψil , l = 1, . . . , Li, i.e.,[

Qi−1i,j

]a,b

= 0, if b 6= z,[Qi−1j,i

]b,a

= 0, if b 6= z − 1, b 6= z and b 6= z + 1,


iii. The lower paths in Def. 7.2.iii.a. and Def. 7.2.iii.b. will be projected ontotransitions from any level ψil , l = 1, . . . , Li, to ψjz, and from ψjz−1, ψjz and ψjz+1

to any level ψil , l = 1, . . . , Li, i.e.,[Qi−1i,j

]a,b

= 0, if b 6= z − 1, b 6= z and b 6= z + 1,[Qi−1j,i

]b,a

= 0, if b 6= z,


iv. There are no lower paths from ωk to ωj so Qj−1k,j = 0 and T4 transitions are

preserved.

v. There are no lower paths from ωj to ωk so Qj−1j,k = 0 and T5 transitions are

preserved.

vi. Following the same reasoning as above we immediately state that the lowerpaths are projected onto:

a. a T1(z) transition.

b. a T2(z) transition.

c. a T3(z) transition.

d. a T4 transition.

e. a T5 transition.

f. a T6 transition.

Since T6 transitions can be considered as special cases of the other five transitionswe can conclude that the direct conditions are maintained in each reduction step bythe indirect conditions.


Theorem 7.1 ensures that the six types of transitions in Definition 7.1 are main-tained through all the reduction steps. We can therefore state the following relationbetween the direct conditions and the tri-diagonal block structure of each set.

Theorem 7.2. The direct conditions between ωi and ωj, i < j, in Definition 7.1ensure that the original tri-diagonal block structure of ωj is preserved in reductionstep i. Moreover, these six sets of transitions are the only transitions that preservethe tri-diagonal block structure.

Proof. From (7.6) and Table 7.1 it can be seen that T1(z), T2(z) and T3(z) tran-sitions are projected onto transitions within ψjz or onto transition to and from oneof the adjacent levels, i.e., ψjz−1 and ψjz+1 (assuming these levels exist). It also fol-lows from Table 7.1 that the remaining three types of transitions are not projectedonto transitions in ωj and we conclude that the tri-diagonal block structure of ωj ispreserved by the direct conditions.

Suppose there exist a T7 transition which is not included in any of the six sets inDefinition 7.1. A T7 must have transitions in both directions, otherwise it is merelya special case of a T4 or a T5 transition. Note that since ωi is removed from thestate space in reduction step i, we can assume that transitions occur from, and to,any level in ωi, i.e., ψil , l = 1, . . . , Li. To preserve the tri-diagonal block structure, aT7 transition must be projected onto transitions within a certain level and betweendirectly adjacent levels, meaning that the transitions from ωj to ωi cannot originatefrom more than three levels. Similarly, transitions from any level in ωi cannot go tomore than three (adjacent) levels in ωj .

Suppose that a T7 transition contains transitions from ψjz−1, ψjz, and ψjz+1, forsome z ∈ 2, . . . , Li − 1, to any level in ωi. Then, the only possibility to preservethe tri-diagonal block structure is to only allow transitions from any level in ωi toψjz, making it a T2(z) transition.

Next, suppose that a T7 transitions contains transition from ψja and ψjb , b 6= a,to any level in ωi. If |b− a| > 1 the two levels are not adjacent and to preserve thetri-diagonal block structure |b − a| = 2 must hold. In this case a T7 transition isagain a special case of a T2(z) transition. If |b− a| = 1 the levels are adjacent and,to preserve the tri-diagonal structure, transitions may occur from any level in ωi toψja and ψjb , |b− a| = 1, making it a T1(z) transition.

Finally suppose that a T7 transitions contains transition from ψjz, for some z ∈1, . . . , Li, to any level in ωi then, to preserve the tri-diagonal block structure,transition can occur from any level in ωi to only three levels (two levels in case z = 1or z = Li) in ωj , namely, ψjz−1, ψjz, and ψjz+1. Even if some of these transitions arezero, we find that T7 must be a T1(z), or a T3(z) transition.

We can thus conclude that the six sets of transitions in Definition 7.1 are theonly types that preserve the tri-diagonal block structure of ωj .

Theorem 7.2 guarantees that Qki,i, for i = k, . . . , S, has a tri-diagonal block

structure after reduction step k − 1.


Remark 7.2 (Ordering of the sets). Note that in Definition 7.1 and Theorem 7.2each type of transition specifies that there are transition from any level in ωi tosome levels in ωj , and/or transitions from some levels in ωj to any level in ωi, withi < j. This ordering of sets if important since it determines whether or not thetri-diagonal block structure is preserved. If, for instance, a T1(z) transition wouldbe reversed, i.e., there are transitions from any level in ωj to ψiz and ψiz+1, and back,with i < j, then in reduction step i, this T1(z) transition will be projected ontotransitions within and between all levels in ωj , making Qi+1

j,j a full matrix insteadof a tri-diagonal block structured matrix. It is therefore important that the sets areordered correctly such that Definition 7.1 holds.

7.3.2 Intermediate step

Theorem 7.2 guarantees that QS describes a finite LDQBD of LS levels:

QS =

L(1)

S F (1)

S 0 · · · · · · 0

B(2)

S L(2)

S F (2)

S

. . ....

0 B(3)

S

. . .. . .

. . ....

.... . .

. . .. . .

. . . 0...

. . .. . . L

(LS−1)

S F(LS−1)

S

0 · · · · · · 0 B(LS)

S L(LS)

S

.

The stationary distribution pS =[pS1 pS2 · · · pSLS

]of QS , i.e., pS such that

pSQS = 0 and pSe = 1, is given by Gaver, Jacobs and Latouche [34], see alsoSection 5.5,

C1 = L(1)

S ,

Ci = L(i)

S +B(i)

S [−Ci−1]−1F (i−1)

S , 2 ≤ i ≤ LS .

and

pSLSCLS = 0,

pSi = pSi+1B(i+1)

S [−Ci]−1, 1 ≤ i ≤ LS − 1,

such that

LS∑i=1

pSi e = 1.


T1(z) T2(z) T3(z)

U = z, z + 1 U = z U = z − 1, z, z + 1

T4 T5 T6

U = ∅ U = 1, . . . , LS−k+i U = ∅

Table 7.4: Subset U for each type of transition from ωS−k to ωS−k+i.

7.3.3 Expansion step k

Let P S−k =[pS−k pS−k+1 · · · pS

]be the vector obtained after expansion step

k. By normalising this vector we obtain the stationary distribution of QS−k. Let pijdenote the subvector of pi corresponding to ψij . Following the expansion step (7.3)we obtain

pS−k =[pS−k+1 · · · pS

]QS−kS−k+1,S−n

...

QS−nS,S−k

[−QS−kS−k,S−k

]−1

=

k∑i=1

pS−k+iQS−kS−k+i,S−k

[−QS−k

S−k,S−k

]−1

.

By decomposing the submatrices by their levels gives

pS−kj =

k∑i=1

LS−k∑b=1

LS−k+i∑a=1

pS−k+ia

[QS−kS−k+i,S−k

]a,b

[−QS−k

S−k,S−k

]−1

b,j. (7.10)

Utilising the type of transition between ωS−k+i and ωS−k we can write the innersum as ∑

a∈U

[pS−k+i

]a

[QS−kS−k+i,S−k

]a,b,

where U ⊆ 1, . . . , LS−k+i follows from the type of transition and is given in Ta-ble 7.4. The stationary distribution π of Q is obtained by normalising the vectorobtained after expansion step S − 1.

7.3.4 Inverse of −Qkk,k

In reduction step k and expansion step S − k the negative inverse of the transientgeneratorQk

k,k needs to be determined. It follows from Theorem 7.1 and Theorem 7.2

that Qkk,k has a tri-diagonal structure for k = 1, . . . , S, and, moreover, that it is


described by a LDQBD. In Shin [92], direct formulas are given to determine thefundamental matrix of a transient LDQBD, see also Section 5.5.

7.4 Simplified successive censoring algorithm

In Section 7.2 we introduced the three-dimensional Markov chain X with states(s, l, p). Next, we have shown that under the conditions stated in Definition 7.1and Definition 7.2, the stationary distribution can be obtained using the successivecensoring algorithm in Algorithm 7.1. The direct conditions in Definition 7.1 are easyto check for the Markov chain X but checking the indirect conditions in Definition 7.2is a difficult task since all lower paths must be checked. To solve this problem, weintroduce a simplified version of the successive censoring algorithm which tells if theindirect conditions hold and consequently, if the successive censoring algorithm canbe applied.

Let Ck(i, j), i < j, denote the collection of transition types from ωi to ωj afterreduction step k − 1 (in which ωk−1 is removed). For example, suppose that afterreduction step k−1, ψj3, ψj4, and ψj5 can be reached from ωi, i < j, but ωi cannot bereached from ωj , then the transitions between ωi and ωj are a special case of botha T3(4) and T4 transition and

Ck(i, j) = T3(4), T4.

Next, we define the iteration

Ck+1(i, j) = Ck(i, j) ∩W[Ck(k, i)× Ck(k, j)

], i < j < k, (7.11)

where the Cartesian product Ck(k, i)×Ck(k, j) consists of all ordered pairs describingthe type of transitions from ωk to ωi and from ωk to ωj . Upon removing ωk thesetransitions are projected to transitions from ωi to ωj according to Table 7.5. Thefunction W is then the union of the projections of each pair in Ck(k, i) × Ck(k, j),given in Table 7.5.

Recall that a T6 transition is the trivial all-zero special case of all other transi-tions. Therefore, we state that if there is a T6 transitions from ωi to ωj , then Ck(i, j)is the union of all possible transition types, i.e.,

Ck(i, j) =

Lj−1⋃z=1

T1(z)

∪Lj−1⋃z=2

T2(z)

∪Lj−1⋃z=2

T3(z)

∪ T4 ∪ T5

.

In case of a T6 transition, the intersection in (7.11) comes down to

Ck(i, j) ∩ T6 = Ck(i, j).

To demonstrate the iteration in (7.11) we use the following example.

7.4 Simplified successive censoring algorithm 93

Ck(k,j

),k<j

T1(y

)T

2(y

)T

3(y

)T

4T

5T

6

Ck(k,i),k<i

T1(x

)T

1(y

)T

2(y

)T

3(y

)T

4T

5T

6

T2(x

)T

1(y

)T

2(y

)T

3(y

)T

4T

5T

6

T3(x

)T

1(y

)T

2(y

)T

3(y

)T

4T

5T

6

T4

T1(y

),T

3(y

),T

1(y−

1),T

1(y

),

T3(y

),T

5T

6T

5T

6T

3(y

+1),T

5T

2(y),T

3(y−

1),

T3(y),T

3(y

+1),T

5

T5

T1(y

),T

2(y

),

T2(y

),T

4

T1(y−

1),T

1(y

),

T4

T6

T6

T2(y

+1),T

4

T2(y−

1),T

2(y

),

T2(y

+1),T

3(y

),T

4

T6

T6

T6

T6

T6

T6

T6

Table

7.5

:P

roje

ctio

ns

onto

transi

tions

fromωi

toωj,k<i<j.


Example 7.1. Suppose that Ck(i, j) = T3(4), T4, Ck(k, i) = T2(3), T5, andCk(k, j) = T3(4) then

W[Ck(k, i)× Ck(k, j)

]=W [T5, T3(4), T2(3), T3(4)]= T1(3), T1(4), T2(3), T2(4), T2(5), T3(4), T4 ∪ T3(4)= T1(3), T1(4), T2(3), T2(4), T2(5), T3(4), T4,

and

Ck+1(i, j) = T3(4), T4 ∩ T1(3), T1(4), T2(3), T2(4), T2(5), T3(4), T4= T3(4), T4.

Theorem 7.3. If Ck+1(i, j) = ∅ for any two sets ωi and ωj, i < j, after reductionstep k (k = 1, . . . , ω − 1), then the direct conditions in Definition 7.1 are violatedand the successive censoring algorithm can no longer be applied.

Proof. Observe that the first term in (7.11) describes the possible types of transitionsfrom ωi to ωj before removing ωk, whereas the second term describes the projectionas a result of removing ωk. The direct conditions only hold if these projectionscorrespond to the existing types of transitions. If Ck+1(i, j) = ∅ the projections aredifferent than the existing transitions and the direct conditions are violated afterreduction step k.

We close this section by presented the simplified successive censoring algorithm

Algorithm 7.2 (Simplified successive censoring algorithm).

1. Determine C1(i, j), i < j, by charaterising the types of transitions in theMarkov chain X .

2. Perform the iteration in (7.11) until CS−1(S − 1, S) is obtained. Determine ineach iteration step

a. the Cartesian product Ck(k, i) × Ck(k, j) specifying all possible combina-tions of transitions between ωk and ωi, and between ωk and ωj , i < j < k,

b. the resulting projection of each element in Ck(k, i) × Ck(k, j) using Ta-ble 7.5,

c. the union of these projection, i.e., determining W[Ck(k, i)× Ck(k, j)

],

i < j < k,

d. and finally the intersection of the original transitions, Ck(i, j), and theunion of the projections.

3. If CS−1(S − 1, S) 6= ∅ then the successive censoring algorithm can be appliedto obtain the stationary distribution. If, on the other hand, in some iterationCk(i, j) = ∅, for i < j < k, the successive censoring algorithm cannot beapplied.

7.5 Complexity analysis 95

7.5 Complexity analysis

Let us consider a system of connected LDQBDs with S sets, each with L levels(L = maxk Lk) of P phases (P = maxj,k Pj,k) phases each. The successive censor-ing algorithm consists of S − 1 reduction steps, one intermediate step, and S − 1expansion steps. Since the intermediate step is performed only once while both thereduction and expansion steps are performed S − 1 times, we can ignore the effectof the intermediate step on the complexity. Furthermore, some of the operationsneeded in the reduction steps are also needed in the expansions steps, the productQki,k[−Qk

k,k]−1 for example, is used in both the reduction step as well as the expan-sion step. However, in reduction step k, we need to multiply this product with amatrix (on the right) (S − k)2 times, while in expansion step S − k we must mul-tiply this product with a vector (on the left) S − k times. Therefore, the reductionstep requires more computations than an expansion step and we can focus on thereduction steps alone to determine the complexity of the algorithm.

The complexity of each reduction step depends on the type of transitions be-tween the sets. Obviously, a T6 transition does not have any projections and willnot contribute to the complexity, therefore, we will assume that in this worst-casescenario there are no T6 transitions. Furthermore, a T1(z) requires 4 projections,whereas censoring a T2(z) or T3(z) would only require 3 projections, making T1(z)the worst-case among these three transitions. Next, we show that by reordering thesets, any T5 transition can be changed into a T4 transition without disobeying theindirect conditions.

Corollary 7.1. For any system of connected LDQBDs which satisfies both the directand indirect conditions with T4 and T5 transitions, the sets can be reordered suchthat there are only T4 transitions.

Proof. Consider the (schematically represented) Markov chain in Figure 7.2(a) witha T5 transition from ωi to ωj , i < j. Recall that this T5 transition only containstransitions from ωj to ωi, therefore, we depict is with a black arrow from ωj to ωi.The blocks D1, D2, and D3 are collections of sets with appropriate index, e.g., subsetD1 contains all sets ωy with y < i.

Suppose |j − i| = 1, then D2 = ∅. By definition, there is no lower path from ωjto ωi and the sets can be switched without violation Definition 7.2.

Next, suppose D2 6= ∅, then |j − i| > 1. Since both the direct and indirectconditions hold there cannot be a lower path from ωi to ωj . If there would be alower path, then after reduction step j− 1 there would be transitions from any levelin ωj (due to the T5 transition) to an some level in ωj (due to the lower path fromωi to ωj). This suggests that D2 can be partitioned into the subsets

• D2a, with sets ωx, i < x < j, with at least one lower path from ωx to ωj andnone from ωi to ωx,

• D2b, with sets ωy, i < y < j, with at least one lower path from ωi to ωy andnone from ωy to ωj ,


1 2a

j

2b

i

3

(b)

(a)

2

ji

1 3

Figure 7.2: Schematic representation of system of connected LDQBDs, before (a) andafter (b) reordering of the sets.

• D2c, with sets ωz, i < z < j, with no lower paths from ωz to ωj and no lowerpaths from ωi to ωz.

Note that there cannot be any lower paths between D2c and D2a or D2b since thiswould imply that there would be a lower path from D2c to ωj or from ωi to D2c,which is not possible. Furthermore, note that there can be lower paths from ωj toany set in D2a, D2b, or D2c, also, there can be lower paths from any set in D2a, D2b,or D2c to ωi.

We can switch ωi and ωj and relocate the subsets D2a, D2b, and D2c such thatthe Markov chain in Figure 7.2(b) is formed. Let σ(j) be the index of ωj in this newMarkov chain, then σ(j) < σ(i) and the T5 transition from ωi to ωj is transformedin a T4 transition from ωσ(j) to ωσ(i). Since there are no lower paths from ωito ωj , there are no lower paths from ωσ(i) to ωσ(j) and Definition 7.2 still holds.Furthermore, by making sure that D2a is placed before ωσ(j) and D2b is placed afterωσ(i), no new lower paths were created that might violate Definition 7.2.

Due to Corollary 7.1 we will consider a system of connected LDQBDs with onlyT1(z) and T4 transitions. By only considering non-zero transitions, we can concludethat a projection of 2 T1(z) transitions is a vector-matrix-vector multiplication withO(L2P 3), and that a projection of a T1(z) and a T4 transition is a vector-matrix-matrix multiplication, also with O(L2P 3). So to determine the complexity of thealgorithm we must maximise the number of projections made in each step (insteadof the size of the projections). Since the projection of 2 T1(z) transitions resultsin 4 projections while the projection of a T1(z) and a T4 transition results in 2projections, we will consider a system of connected LDQBD-processes with onlyT1(z) transitions, for some z ∈ 1, . . . , L.

In reduction step k, we must perform (S−k)2 projections of 2 T1(z) transitions.A single projection of 2 T1 transitions is a vector-matrix-vector multiplication withO(L2P 3). Therefore the total complexity, including the inverse following [92], ofreduction step k is O(S2L2P 3). Finally, there are S − 1 reduction steps resulting in

7.6 Demonstration of the successive censoring algorithm 97

a complexity of the successive censoring algorithm of O(S3L2P 3). Comparing thisto solving πQ = 0 with O(S3L3P 3) we conclude that we decrease complexity by afactor L.

7.6 Demonstration of the successive censoring al-gorithm

To demonstrate the successive censoring algorithm we will use Algorithm 7.1 toobtain the stationary distribution for a Markov chain with generator Q in (7.12).

Example 7.2. Let us consider a Markov chain with generator Q in (7.12). Here,S = 3, L1 = L2 = L3 = 4, and all levels consists of a single phase.

Q =

−1 1

9 −16 2 5

8 −19 3 8

7 −11 4

2 −6 4

3 6 −14 5

5 −11 6

4 −8 4

−7 7

3 3 3 −17 8

3 2 −14 9

3 3 1 −7

.

(7.12)In (7.12), the solid lines denote three distinctive sets, each representing a LDQBDand Q is partitioned as follows

Q = Q1 =

Q11,1 Q1

1,2 Q11,3

Q12,1 Q1

2,2 Q12,3

Q13,1 Q1

3,2 Q13,3

.First we determine, using Algorithm 7.2, if the successive censoring algorithm isapplicable. In (7.12) we see that

C1(1, 2) = T1(1), C1(1, 3) = T3(2), C1(2, 3) = T3(2).

The iteration (7.11) and Table 7.5 now give us

C2(2, 3) = T3(2) ∩W [T1(1), T3(2)] = T3(2).


From which we conclude that Algorithm 7.1 can be applied.

Next, we perform 2 reduction steps (7.2) to remove ω1 and ω2 from the statespace. The generators obtained in these 2 reduction step are Q2

Q2 =

− 24857

1228285 0 0 0 0 4

95 013619 − 1176

95 5 0 0 0 2195 0

0 5 −11 6 0 0 0 0

0 0 4 −8 0 0 4 0

0 0 0 0 −7 7 0 04719

4495 3 0 3 −17 766

95 02219

15495 0 0 0 2 − 1309

95 91419

9895 0 3 0 0 212

95 −7

,

and Q3

Q3 =

−7 7 0 0

3 −17 13 0

0 2 −11 9

0 0 7 −7

.The intermediate step gives us the stationary distribution p3 of Q3

p3 =[

3122

7122

49122

63122

].

Finally, we perform 2 expansion step (7.3) to obtain p2

p2 =[

2043279873520

280371218380

402143676

3861943676

].

and p1

p1 =[

1755243109190

553077436760

4382787352

2425587352

].

Normalising the vector[p1 p2 p3

]gives π =

[π1 π2 π3

]with

π1 =1

21443881

[14041944 1106154 438270 242550

],

π2 =1

21443881

[2043279 1121484 804300 772380

],

π3 =1

21443881

[21480 50120 350840 451080

].

It is now easy to check that πQ = 0.



We introduced a successive censoring algorithm to find the stationary distributionof a general class of Markov chains consisting of multiple Level Dependent Quasi-Birth-and-Death processes (LDQBD) connected by special types of transitions. Thesuccessive censoring algorithm consists of reduction steps, in which the state space isreduced by removing a LDQBD in each step, an intermediate step, in which the sta-tionary distribution of the reduced Markov chain is determined, and expansion steps,in which the stationary distribution is expanded by adding a (previously removed)LDQBD back to the state space. By applying the results of Shin [92] we determinethe inverse of the transient LDQBD-generator required in both the reduction andexpansion steps.

In Section 7.5 we have shown that the complexity of the successive censoringalgorithm is O(S3L2P 3). The successive censoring algorithm is applied to Markovchain with three sets in Section 7.6.


CHAPTER 8

An iterative aggregation method for a

queueing network with finite buffers

8.1 Introduction

We consider a tandem queueing network with finite buffers as depicted in Figure 8.1.Customers arrive at the first queue according to a Markovian Arrival Process (MAP )which depends on the number of customers in the first queue. At each queue,customers are served by a single server according to a Markovian Service Process(MSP ) depending on both the length of the queue in front of the server, and thelength of the downstream queue. The MSP of the last server only depends on thelength of the last queue. We assume a Blocking Before Service (BBS) protocol,i.e., a server becomes blocked, and is unable to serve customers, if its departingcustomer fills the downstream queue. External arrivals that find the first queue full,are rejected and lost.

MAP N1 MSP1 N2 MSP2 N3 MSP3 N4 MSP4 N5 MSP5

Queue 1 Queue 2 Queue 3 Queue 4 Queue 5

Figure 8.1: A tandem queue with M = 5 servers.

One of the first studies on approximating the marginal queue length distributionby decomposing a queueing network with finite buffers into smaller sized subnetworkswas performed by Altiok [2]. In [2] Altiok analyses a k-tandem queueing networkwith finite buffers, exponential service times and Poisson arrivals to the first queue.Altiok approximates each queue in the network by an M/C2/1/N queue, assumingPoisson arrivals and a Coxian service distribution of 2 phases to incorporate blockingby the downstream queue. The parameters of the Coxian distribution are determinedby the marginal queue length distribution of the downstream buffer and are obtainedwith a backward iteration method, starting with queue k. This backward iterationmethod is also used in the paper by Perros and Altiok [85] in which the work ofAltiok [2] is extended by approximating each queue by an M/Ck/1/Ni queue incor-porating blocking by multiple downstream queues. Merging and splitting topologies

102 8. An iterative aggregation method

are considered by Altiok and Perros in [4]. The results above are combined in thepaper by Altiok and Perros [5] in which arbitrary networks with exponential servicetimes are approximated. In [3] Altiok extends his backward iteration method byincorporating Phase-Type service distributions.

The backward iteration of Altiok [2] was extended by Gershwin in [35] by combin-ing a backward iteration with a forward iteration. Gershwin considers a k-tandemqueueing network in discrete time with unreliable geometric servers and interruptedgeometric arrivals. He describes a backward iteration to incorporate blocking and aforward iteration to incorporate starvation, therefore extending the work of [2]. Dur-ing the iterations, the service distributions are adjusted to incorporate both blockingand starvation using previously adjusted service distributions of the neighbouringqueues. The iterations continue until an equilibrium in service distributions is ob-tained. The approximation algorithm of Gershwin was extended to continuous timeand Phase-Type distributions by Altiok and Ranjan in [6]. A specific Phase-Typedistribution, namely C2, for both the arrival and service distribution was consideredby Helber [45]. Further extensions towards general arrival and service distributionsand multi-server queues were performed by Van Vuuren, Adan and Resing-Sassenin [97]. In Van Vuuren and Adan [96] and in Bierbooms, Adan and Van Vuuren[13], the authors argue if at a service completion the downstream buffer is full it islikely that at the next service completion the downstream buffer is again full. Theytherefore extend Gershwin’s method to include correlated service time distributions.An improvement of Gershwin’s method, providing accuracy and proof of conver-gence, was given by Li and Meerkov [63]. This model is able to approximate severalperformance measures such as throughput or work in progress for a great varietyof queueing networks [64, 65]. For surveys on aggregation methods for queueingnetworks with finite buffers, we refer to Dallery and Gershwin [26] and Li et al. [62].

All the techniques described above focus on a specific queue in the network andapproximate both the upstream and downstream network behaviour. However, themajority of these aggregation techniques result in a queue with both an uncorrelatedarrival process and an uncorrelated service process. Only two papers discuss anapproximation technique that result in a queue with correlated service process.

In this chapter we propose an iterative aggregation method to approximate themarginal queue length distribution of a single queue in a tandem network with finitebuffers, with state dependent Markovian Arrival Processes (MAP s) and with statedependent Markovian Service Processes (MSP s) at each server. We approximatean arbitrary queue i, i = 1, . . . ,M , by a state dependent MAP/MSP/1/Ni whichincorporates both a correlated arrival process as well as a correlated service process.In our iterative aggregation method we first construct a state dependent MAP forqueue i by aggregating all upstream network behaviour, i.e., queue 1 up to (andincluding) queue i−1. Second, we aggregate all downstream network behaviour, i.e.,queue i+1 up to queue M , to form the state dependent MSP for queue i. Instead ofaggregating the upstream (downstream) network in one single aggregation step, weaggregate the upstream (downstream) network iteratively, aggregating one queue ineach aggregation step. The resulting state dependent MAP/MSP/1/Ni is a Level


Dependent Quasi-Birth-and-Death process (LDQBD) and its marginal queue lengthdistribution is obtained using Matrix analytic methods.

Section 8.2 presents the queueing network and gives the generator of a singlequeue in the network, modelled as a state dependent MAP/MSP/1/Ni-queue. InSection 8.3 we first present an aggregation method for a Markovian Arrival Process(MAP ) and a Markovian Service Process (MSP ), next, we focus on the iterativeprocedure, and finally, we obtain the marginal queue length distribution of the re-sulting state dependent MAP/MSP/1/Ni queue using Matrix analytic methods.Numerical results are presented in Section 8.4. We give concluding remarks in Sec-tion 8.5.


We consider a tandem queueing network of M single server queues with finite buffers.New customers arrive at the first queue according to a Markovian Arrival Process(MAP ) depending on the number of customers in the first queue. Arriving customersthat find the first queue to be full are rejected and lost. After arrival, customers areserver at queue i, i = 1, . . . ,M , according to a Markovian Service Process (MSP )depending on both the length of queue i as well as the length of queue i + 1. TheMSP of queue M depends on the length of queue M alone. We assume a BlockingBefore Service (BBS) protocol, i.e., a server becomes blocked, and unable to servecustomers, if its departing customer fills the downstream queue.

The tandem network is a Markov chain with states (s0, n1, s1, n2, s2, . . . , nM , sM ),where ni = 0, . . . , Ni denotes the queue length at queue i and si, i = 1, . . . ,M , de-notes the state of the MSP of queue i. We denote by s0 the state of the MAP ofqueue 1. Let An1

0 (1) and An11 (1) define the MAP of queue 1 when its queue length

is n1 and let SnM0 (M) and SnM1 (M) define the MSP of queue M when its queuelength is nM . Furthermore, let us define the MSP of queue i, i = 2, . . . ,M − 1, bythe matrices T

ni,ni+1

0 (i) and Tni,ni+1

1 (i). Next, let In1

A (1), InMS (M) and Ini,ni+1

T (i)denote identity matrices of the same size as An1

0 (1), SnM0 (M), and Tni,ni+1

0 (i) re-spectively and let e denote a vector of ones of appropriate size. Finally, let ⊗ and ⊕be the Kronecker product and Kronecker sum respectively, see Chapter 5 for moredetails.

The marginal queue length distribution of, say, queue i can be obtained bymodelling queue i as a state dependent MAP/MSP/1/Ni queue. This queue isa Level Dependent Quasi-Birth-and-Death process (LDQBD) and its queue lengthdistribution can be obtained using Matrix analytic methods. However, the arrivalprocess of queue i is determined by the upstream network behaviour with states(s0, n1, s1, n2, s2, . . . , ni−1, si−1), and the service process of queue i is determined bythe downstream network behaviour with states (si, ni+1, si+1, ni+2, si+2, . . . , nM , sM ).In Section 8.3 we present an iterative aggregation method to approximate both thearrival process and service process of queue i. To this end, we extend the above nota-tion by defining the MAP of queue i, i = 1, . . . ,M , which is obtained by combiningall its upstream network behaviour, by Ani

0 (i) and Ani1 (i) and we define the MSP


of queue i, i = 1, . . . ,M , after combining all its downstream network behaviour by

Sni0 (i) and Sni1 (i). Next, we define by Ani

0 (i) and Ani

1 (i) the aggregated MAP of

queue i, and by Sni

0 (i) and Sni

1 (i) the aggregated MSP of queue i. Finally, wedenote by IniA (i), Ini

A(i), IniS (i), and Ini

S(i) the identity matrix of the same size as

Ani0 (i), A

ni

0 (i), Sni0 (i), and Sni

0 (i) respectively.

8.3 Iterative aggregation method

In this section we will present an iterative aggregation method to obtain the marginalqueue length distribution of an arbitrary queue i in the tandem queueing networkof Figure 8.1. Recall that the arrival process to queue i depends on the state ofall upstream queues. The size of the state space of this MAP is too large to givetractable results. Therefore, we aggregate the state space. The service process de-pends on the downstream queues and also needs to be aggregated. We first introducean aggregation method to reduce the size of a Markovian Arrival Process (MAP ) orMarkovian Service Process (MSP ). Second, we discuss the procedure to iterativelyaggregate all upstream and downstream queues of queue i. Last, we discuss how themarginal queue length distribution is obtained for a MAP/MSP/1/N queue withstate dependent MAP s and MSP s.

8.3.1 Aggregation of a Markovian Arrival Process

Consider the MAP X, defined by A0 and A1, with state space SX consisting of nstates. First, we form m subsets Ωi, i = 1, . . . ,m, such that Ωi ∩ Ωj 6= Ωi, i 6= j,and

⋃mi=1 Ωi = SX . The states in Ωi, i = 1, . . . ,m, are chosen such that the total

arrival rate in each state is Λi, or, [A1e]k = Λi, if k ∈ Ωi. This implies that if k ∈ Ωiand [A1e]k 6= [A1e]l, then l /∈ Ωi. Furthermore, if k ∈ Ωi and k ∈ Ωj , i 6= j, thenΛi = Λj . Next, we aggregate each Ωi into a single exponential state ωi, aggregating

the original n-state MAP to an m-state MAP called Y , defined by A0 and A1,with state space SY and m ≤ n.

The transition rates in Y are chosen such that the mean sojourn time in a partic-ular state ωi is the same as the mean sojourn time in the set Ωi. Also, the transitionprobability from ωi to ωj in Y , is the same as the transition probability from Ωito Ωj in X. As a result, the average arrival rate to the queue is the same for bothMAP X and the aggregated MAP Y , i.e., suppose pX is the stationary distributionof MAP X and pY is the stationary distribution of MAP Y , then

pX(A0 +A1) = 0, pY (A0 + A1) = 0, and pXA1 = pY A1.

Each set Ωi is a finite transient Markov chain and both the sojourn time in theset and the probability of changing to another set depend on the state through whichthe Ωi was entered. We denote these so-called entrance states by σi. If |σi| = 1,there is only one entrance state and the probability that Ωi was entered through thisstate is 1. However, when |σi| > 1, there are multiple entrance states and we must

8.3 Iterative aggregation method 105

(a) (b)

1 2 3 4

1 2a 3a

2b 3b 4

λ λ λ

µ µ µ

λ λ

λ

λ λ

µ µ

µ

µ µ

Ω1

Ω2

Figure 8.2: Enlarging the state space of A0 +A1 by introducing duplicate states.

determine the probability that Ωi is entered through a specific entrance state. Todetermine these probabilities we introduce the jump chain P . We define a jump tobe the event at which the MAP leaves one set and enters another. The submatrixP i,j of P then describes the transition probabilities from Ωi to Ωj . In order toobtain P we define the matrices A0(i, j) and A1(i, j). These |Ωi|× |Ωj | matrices areconstructed from A0 and A1 and correspond to specific transitions of the MAP . Inparticular, Ak(i, i) corresponds to transitions within set Ωi with (k = 1) or without(k = 0) the occurrence of an arrival. In addition, A0(i, i) also contains the negativediagonal elements of A0 of the states in Ωi. The matrices Ak(i, j), i 6= j, describethe transitions from Ωi to the entrance states σj of Ωj . The rates to non-entrancestates are 0, i.e., the element [Ak(i, j)]a,b = 0, ∀a ∈ Ωi, if b /∈ σj . In Example 8.1we show how the matrices A0(i, j) and A1(i, j) are obtained for a 4-state MAP .

Example 8.1. Consider the 4 state MAP with

A0 =

−λ λ 0 0

0 −λ− µ λ 0

0 0 −λ− µ λ

0 0 0 −µ

, A1 =

0 0 0 0

µ 0 0 0

0 µ 0 0µ2 0 µ

2 0

, A1e =

0

µ

µ

µ

.

We form three subsets: Ω1 = 1, Ω2 = 2, 3, and Ω3 = 3, 4 with entrance statesσ1 = 1, σ2 = 2, and σ3 = 4 and Λ1 = 0, and Λ2 = Λ3 = µ. Construction ofA0(i, j) and A1(i, j) resembles enlarging the state space by creating duplicates forthose states that are included in multiply sets, in this example state 3. Figure 8.2(a)shows the Markov chain with generator A0 +A1. Enlarging the state space resultsin the Markov chain in Figure 8.2(b) in which duplicates of state 3 is incorporated.Transitions between Ω1, Ω2, and Ω3 occur via their entrance states.


We now obtain

A0(1, 1) A0(1, 2) A0(1, 3)

A0(2, 1) A0(2, 2) A0(2, 3)

A0(3, 1) A0(3, 2) A0(3, 3)

=

−λ λ 0 0 0

0 −λ− µ λ 0 0

0 0 −λ− µ 0 λ

0 0 0 −λ− µ λ

0 0 0 0 −µ

,

A1(1, 1) A1(1, 2) A1(1, 3)

A1(2, 1) A1(2, 2) A1(2, 3)

A1(3, 1) A1(3, 2) A1(3, 3)

=

0 0 0 0 0

µ 0 0 0 0

0 µ 0 0 0

0 µ 0 0 0µ2 0 0 µ

2 0

.

The submatrix P i,j of the jump chain P by is given by

P i,j = − [A0(i, i) +A1(i, i)]−1

[A0(i, j) +A1(i, j)] , i, j = 1, . . . ,m, and i 6= j.(8.1)

The negative inverse on the right hand side of (8.1) is the fundamental matrix of thetransient generator A0(i, i) + A1(i, i) and describes the mean sojourn time in Ωi,while the second part describes the transition rates from Ωi to σj ∈ Ωj . The productof the two gives the probability that, starting in some state in Ωi, the process entersΩj upon leaving Ωi, see Kemeny and Snell [54].

The transition matrix P of the jump chain has stationary distribution p, withp = [ p1 p2 · · · pm ] and pi = [ pi(1) pi(2) · · · pi(|Ωi|) ] such that pP = pand pe = 1.

Remark 8.1 (Reducing size of P ). Note that the transition matric P can be reducedin size by removing all non-entrance states. Since these states cannot be visited aftera jump, the columns corresponding to non-entrance states will be all-zero columnsand pi(j) = 0 if j /∈ σi.

The stationary distribution p is the probability that after a jump between sets,the Markov chain enters a specific state, e.g.,

pi(j) = P[The Markov chain enters state j in Ωi after a jump between sets].

We use this distribution to find the probability that, given that the set Ωi is entered,it is entered through state j ∈ Ωi. This conditional probability is denoted by pi, withpi = [ pi(1) pi(2) · · · pi(|Ωi|) ] and is obtained by renormalising the distributionpi within the set Ωi

pi(j) =pi(j)∑|Ωi|k=1 pi(k)

, j = 1, . . . , |Ωi|, and i = 1, . . . ,m. (8.2)


The numerator is the probability that after a jump, state j ∈ Ωi is entered, whilethe denominator gives the probability that after a jump, set Ωi is entered.

We are now ready to state the following aggregation theorem.

Theorem 8.1. Let A0(i, j) and A1(i, j) denote the transitions from Ωi to Ωj in the

original MAP X constructed from A0 and A1, and let A0(i, j) and A1(i, j) denotethe transition rates from ωi to ωj in the aggregated MAP Y , then

Ak(i, j) =pi [−A0(i, i)−A1(i, i)]

−1Ak(i, j)e

pi [−A0(i, i)−A1(i, i)]−1e

, k = 1, 2, and i, j = 1, . . . ,m,

(8.3)where e is a column vector of ones of appropriate size.

Proof. The proof of Theorem 8.1 is based on the interpretation of Equation (8.1).

First consider the denominator in (8.3). The element [(−A0(i, i)−A1(i, i))−1

]a,b isthe mean sojourn time in state b ∈ Ωi before leaving Ωi, given that Ωi was enteredthrough the state a, see Kemeny and Snell [54]. Taking into account the probabilitythat Ωi was entered through a specific state, p, and summing up of all states b, weobtain the mean sojourn time in Ωi. Since these states are aggregated into ωi weobtain the rate at which Ωi is left

A0(i, i) + A1(i, i) =−1

pi [−A0(i, i)−A1(i, i)]−1e. (8.4)

Suppose i 6= j, then the numerator in (8.3) gives, according to the interpretationof (8.1), the probability that Ωj is entered after leaving Ωi. This transition can occurduring an arrival (k = 1), or without an arrival, (k = 0). Therefore the rate at which

ωi changes to ωj , i.e. Ak(i, j), i 6= j, is given by (8.3).

Now suppose that k = 1. Since∑j

A1(i, j) = Λi, and∑j

A1(i, j)e = Λie,

we find

A1(i, i) = Λi −∑j 6=i

A1(i, j)

= Λi −∑j 6=i

pi [−A0(i, i)−A1(i, i)]−1A1(i, j)e

pi [−A0(i, i)−A1(i, i)]−1e

=pi [−A0(i, i)−A1(i, i)]

−1A1(i, i)e

pi [−A0(i, i)−A1(i, i)]−1e

.


Finally, using (8.4) we obtain

A0(i, i) =−1

pi [−A0(i, i)−A1(i, i)]−1e− A1(i, i)

=−1

pi [−A0(i, i)−A1(i, i)]−1e− pi [−A0(i, i)−A1(i, i)]

−1A1(i, i)e

pi [−A0(i, i)−A1(i, i)]−1e

=pi [−A0(i, i)−A1(i, i)]

−1A0(i, i)e

pi [−A0(i, i)−A1(i, i)]−1e

,

which completes the proof.

Example 8.2. Consider an M/M/1/k → •/M/1/∞ tandem queueing system. Cus-tomers arrive with rate λ = 3 and are served in both queue with rate µ1 = µ2 = 5.In this example we will aggregate the Markovian Arrival Process formed by queue 1to form an aggregated MAP for queue 2. We will aggregate this MAP to a 3-stateMAP with sets and entrance states Ω1 = 0, σ1 = 0, Ω2 = 1, 2, . . . , k − 1,σ2 = 1, Ω3 = 2, 3, . . . , k and σ3 = k.

In this particular example we use k = 5 for ease of notation, but we emphasizethat the method holds for arbitrary k <∞. Before aggregation the MAP of queue2 is defined by

A0 =

−3 3 0 0 0 0

0 −8 3 0 0 0

0 0 −8 3 0 0

0 0 0 −8 3 0

0 0 0 0 −8 3

0 0 0 0 0 −5

, A1 =

0 0 0 0 0 0

5 0 0 0 0 0

0 5 0 0 0 0

0 0 5 0 0 0

0 0 0 5 0 0

0 0 0 0 5 0

.

First we determine A0(i, j) and A1(i, j), for i, j = 1, 2, 3. For instance, A0(2, 2) andA1(3, 2) are given by

A0(2, 2) =

−8 3 0 0

0 −8 3 0

0 0 −8 3

0 0 0 −8

, A1(3, 2) =

5 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

.

Proceeding, we find the 3×3 transition matrix P of the jump chain and its stationary


distribution p

P =

0 1 0 0 0 0 0 0 013601441 0 0 0 0 0 0 0 81

144112251441 0 0 0 0 0 0 0 216

144110001441 0 0 0 0 0 0 0 441

14416251441 0 0 0 0 0 0 0 816

1441

0 1 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0

, pT =

1360288214412882

0

0

0

0

0

081

2882

.

Normalising p within each set gives

p =[p1 p2 p3

]=[

1 1 0 0 0 0 0 0 1

].

This result could have been obtained directly since each set has only one entrancestate, therefore the probability that the set is entered through this state must be 1.

Finally, using (8.3) we obtain the aggregated MAP for queue 2 defined by

A0 =

−3 3 0

0 − 2671518

81518

0 0 −5

, A1 =

0 0 0680259

615259 0

0 625842

3585842

.

Remark 8.2 (Aggregation of an MSP ). In this section we have described a methodto aggregate a Markovian Arrival Process. However, the same method applies foran arbitrary Markovian Service Process defined by matrices S0 and S1.

8.3.2 Iterative procedure

The iterative aggregation method consists of one forward iteration phase, in whichthe aggregated MAP of queue i is constructed, and one backward iteration phase, inwhich the aggregated MSP of queue i is constructed. First we focus on the forwarditeration phase in detail, after which we discuss the backward iteration phase. In theforward iteration phase we aggregate the network upstream of queue i, denoted bythe states (s0, n1, s1, n2, s2, . . . , ni−1, si−1), to construct a state dependent MAP forqueue i with states s∗i−1. Instead of aggregating the entire upstream network in onestep, we use i − 1 iterations steps, each of which aggregates a single queue. In thefirst step we aggregate the states (s0, n1, s1) to an aggregated MAP for queue 2 withstates s∗1. Next, in iteration j, j = 2, . . . , i− 1, we aggregate the states (s∗j−1, nj , sj)to construct the aggregated MAP for queue j+ 1 with states s∗j . The final iteration


step in the forward iteration phase gives the aggregated MAP for queue i with statess∗i−1.

Before the MAP of queue j + 1 with states (s∗j−1, nj , sj) can be aggregated, weneed to describe it first. Suppose the length of queue j + 1 is nj+1, then queue j isa MAP/MSP/1/N queue with states (s∗j−1, nj , sj). In particular, the generator for

this queue is Anj+1

0 (j+ 1) +Anj+1

1 (j+ 1). Since Anj+1

1 (j+ 1) corresponds to arrivalsto queue j+ 1, and thus departures from queue j. Therefore, the generator of queuej can be split up to give

Anj+1

0 (j + 1) =

Y0 Z0 0 · · · 0

0 Y1 Z1. . .

......

. . .. . .

. . . 0...

. . . YNj−1 ZNj−1

0 · · · · · · 0 YNj

, (8.5)

Anj+1

1 (j + 1) =

0 · · · · · · · · · 0

X1. . .

...

0 X2. . .

......

. . .. . .

. . . 0

0 · · · 0 XNj 0

,

with

Xk = Tk,nj+1

1 (j)⊗ IkA

(j), k = 1, . . . , Nj ,

Yk = Tk,nj+1

0 (j)⊕ Ak

0(j), k = 0, . . . , Nj , (8.6)

Zk = Ik,nj+1

T (j)⊗ Ak

1(j), k = 0, . . . , Nj − 1,

where Xk describes departures from queue j, Zk describes arrivals to queue j and Ykdescribes state changes without arrivals or departures occurring. The MAP defined

by Anj+1

0 (j + 1) and Anj+1

1 (j + 1) in (8.5) is now aggregated to obtain Anj+1

0 (j + 1)

and Anj+1

1 (j + 1), the aggregated MAP of queue j + 1 with states s∗j .In the backward iteration phase we aggregated the network behaviour down-

stream of queue i with states (si, ni+1, si+1, . . . , nM , sM ) to an aggregated MSP forqueue i with states s∗i . Similar to the forward iteration phase, we aggregate onequeue per iteration step, resulting in M − i iteration steps. In the first step, theMSP with states (sM−1, nM , sM ) is aggregated to a MSP for queue M − 1 withstates s∗M−1. Next, in iteration step j, j = 2, . . . ,M − i, the MSP with states(sM−j , nM−j+1, s

∗M−j+1) is aggregated to a MSP with states s∗M−j . In step M − i,

the aggregated MSP of queue i with states s∗i is obtained.


Assuming the length of queue j − 1 is nj−1, the MSP with states (sj−1, nj , s∗j )

is given by

Snj−1

0 (j − 1) =

Y0 0 · · · · · · 0

X1 Y1. . .

...

0 X2. . .

. . ....

.... . .

. . . YNj−1 0

0 · · · 0 XNj YNj

, (8.7)

Snj−1

1 (j − 1) =

0 Z0 0 · · · 0...

. . . Z1. . .

......

. . .. . . 0

.... . . ZNj−1

0 · · · · · · · · · 0

,

with

Xk = Sk

1(j)⊗ Inj−1,kT (j − 1), k = 1, . . . , NM−j+1,

Yk = Sk

0(j)⊕ T nj−1,k0 (j − 1), k = 0, . . . , NM−j+1, (8.8)

Zk = IkS

(j)⊗ T nj−1,k1 (j − 1), k = 0, . . . , NM−j+1 − 1.

Finally, we aggregate Snj−1

0 (j−1) and Snj−1

1 (j−1) in (8.7) to obtain the aggregatedMSP of queue j − 1 with states s∗j−1.

8.3.3 Marginal distribution

The iterative aggregation method approximates a single queue in the tandem networkby a state dependent MAP/MSP/1/N queue with generator

Q =

S00 ⊕A0

0 I0S ⊗A0

1 0 · · · · · · 0

S11 ⊗ I1

A S10 ⊕A1

0 I1S ⊗A1

1

. . ....

0 S21 ⊗ I2

A

. . .. . .

. . ....

.... . .

. . .. . .

. . . 0...

. . .. . . SN−1

0 ⊕AN−10 IN−1

S ⊗AN−11

0 · · · · · · 0 SN1 ⊗ INA SN0 ⊕AN0

.

(8.9)


This queue is a Level Dependent Quasi-Birth-and-Death process (LDQBD) inwhich the levels are represented by the number of customers in the queue and thephases are represented by the states of both the MAP and the MSP . The stationaryqueue length distribution π = [ π0 π1 · · · πN ], such that πQ = 0 and πe = 1, isobtained using Matrix analytic methodes, see Latouche and Ramaswami [58]. Here,we use the algorithm by Gaver, Jacobs and Latouche [34], see also Section 5.5,

C0 = S00 ⊕A0

0,

Cj = Sj0 ⊕Aj0 −

[Sj1 ⊗ IjA

][Cj−1]

−1[Ij−1S ⊗Aj−1

1

], j = 1, . . . , N,

and

πiNCN = 0,

πij = −πij+1

[Sj+1

1 ⊗ Ij+1A

][Cj ]

−1, j = 0, . . . , N − 1,

such that

N∑j=0

πije = 1.

Remark 8.3 (Special cases). The algorithm by Gaver, Jacobs and Latouche [34] isdesigned to give the stationary distribution of an LDQBD. A queue without statedependent service and arrival processes can be modelled as a Quasi-Birth-and-Deathprocess (QBD) for which the stationary distribution is a mixture of geometric terms,see Hajek [38]. The stationary distribution of a queue controlled by a thresholdpolicy can be obtained using the successive censoring algorithm in Baer, Al Hanbali,Boucherie and van Ommeren [7] and Chapter 7.

8.4 Numerical results

In this section we use the iterative aggregation method to approximate the marginalqueue length distribution and the mean sojourn time in a single queue in a tan-dem network and we compare these results to simulation results. We consider threedifferent tandem networks: a tandem network of M/M/1/N queues, a tandem net-work of two-stage M/M/1/N threshold queues, and a tandem network of four-stageM/M/1/N feedback threshold queues. In these three tandem networks, there arePoisson arrivals to the first queue and exponentially distributed service times. In thetandem network of two-stage M/M/1/N threshold queues the service rates of eachqueue are controlled by a threshold policy based on the queue length. In the tandemnetwork of four-stage M/M/1/N feedback threshold queues, both the service ratesand the arrival rates are controlled by a similar threshold policy. In Sections 8.4.2and 8.4.3 we will explain these policies in detail.

8.4 Numerical results 113

For comparison of our approximation results, we creates a discrete event simula-tion for each of the three tandem networks. For each scenario for each network weran 25 simulations, which only deviated in the third or fourth decimal compared tothe average value. Therefore, we will not include any confidence intervals.

For each of these tandem networks we consider multiple scenario’s for the systemparameters and we consider multiple MAP aggregations, i.e., the sets, Ωi, andentrance states, σi, required in the iterative aggregation method in Section 8.3.1.

8.4.1 A tandem of M/M/1/N queues

We consider a tandem network of M/M/1/N queues. The arrival rate to the firstqueue is λ, the service rate of queue i is µ(i), and the buffer size is Ni. In this tandemnetwork of M/M/1/N queues, si = 1, for i = 1, . . . ,M . We approximate a singlequeue in the network using three different MAP aggregations, namely a two-stateaggregation, a four-state aggregation, and a so-called buffer-sized aggregation.

Two-state aggregation In the two-state aggregation we aggregate an upstreamqueue by an ON/OFF source. To do this, we create 2 sets, Ω1 and Ω2, representingthe OFF-state and the ON-state, respectively. Suppose queues 1 to j−1 are alreadyaggregated to an ON/OFF source that feeds queue j, and we want to aggregate queuej to obtain an ON/OFF MAP for queue j + 1. Recall that before aggregation, thestates of the MAP of queue j + 1 are (s∗j−1, nj , 1). This MAP can be considered tobe OFF when there are no customers in queue j, and ON when there are customerin queue j. Therefore, combining the states of the MAP of queue j, we define thesets Ω1 and Ω2 as follows

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, Ω2 =

⋃s∗j−1

Nj⋃nj=1

(s∗j−1, nj , 1)

.

The set Ω1 can only be entered from Ω2 through the states with nj = 0, and similarly,Ω2 can only be entered from Ω1 through the states with nj = 1. Therefore, we definethe entrance states to be

σ1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

.

This results in a two-state MAP describing an ON/OFF arrival process. Wheneverqueue j is empty, the aggregated MAP will be in state 1 with Λ1 = 0 and whenthere are customers in queue j, the aggregated MAP is in state 2 and Λ2 = µ(j).Similarly, we construct an ON/OFF server as two-state aggregation for a MSP .Before aggregation, the MSP of queue j−1 has states (1, nj , s

∗j ) which we aggregate


by choosing the sets and entrance states according to

Ω1 =⋃s∗j

Nj−1⋃nj=0

(1, nj , s

∗j ), σ1 =

⋃s∗j

(1, Nj − 1, s∗j )

,

Ω2 =⋃s∗j

(1, Nj , s

∗j ), σ2 =

⋃s∗j

(1, Nj , s

∗j ).

In this aggregation, the ON state is represented by Ω1 which denote that queue j isnot full. The OFF state is represented by Ω2 which denotes that queue j is full andqueue j − 1 is blocked. The entrance states indicate that Ω1 can only be enteredfrom Ω2 through the states with nj = Nj − 1, similarly, Ω2 can only be enteredthrough the states with nj = Nj .

Four-state aggregation In the four-state aggregation we partition the state spaceof the MAP of queue j + 1 into three subsets, namely states with nj = 0, stateswith nj = Nj , or states with 0 < nj < Nj . Since the third subset can be enteredeither through states with nj = 1, or nj = Nj − 1. To distinguish between these twopossibilities we define four sets with entrance states as follows

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

,

Ω2 =⋃s∗j−1

Nj−1⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,

Ω3 =⋃s∗j−1

Nj−1⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Nj − 1, 1)

,

Ω4 =⋃s∗j−1

(s∗j−1, Nj , 1)

σ4 =

⋃s∗j

(s∗j−1, Nj , 1)

.

Here, Ω1 can only be entered through states with nj = 0, and Ω4 can only be enteredthrough states with nj = Nj . Note that Ω2 = Ω3, but that Ω2 corresponds to atransition from Ω1 (via states with nj = 1) and that Ω3 corresponds to a transitionfrom Ω4 (via states with nj = Nj − 1). The MSP of queue j − 1 is aggregatedfollowing the same approach, but with

Ω1 =⋃s∗j

(1, 0, s∗j )

, σ1 =

⋃s∗j

(1, 0, s∗j )

,

Ω2 =⋃s∗j

Nj−1⋃nj=1

(1, nj , s

∗j ), σ2 =

⋃s∗j

(1, 1, s∗j )

,

Ω3 =⋃s∗j

Nj−1⋃nj=1

(1, nj , s

∗j ), σ3 =

⋃s∗j

(1, Nj − 1, s∗j )

,


Ω4 =⋃s∗j

(1, Nj , s

∗j )

σ4 =⋃s∗j

(1, Nj , s

∗j ).

Remark 8.4 (Redundant sets). Note that when Nj = 2, then Ω2 = Ω3 and σ2 = σ3.In the aggregation steps we take this into account and remove any redundant sets.If Nj = 1, then Ω1 and Ω3 would coincide, as well as Ω2 and Ω4. The four-stateaggregation would then reduce to a two-state aggregation.

Buffer-sized aggregation In the buffer-sized aggregation, we aggregate queue jto an (Nj + 1)-state MAP . This sets and entrance states are chosen such that afteraggregating queue j, the aggregated MAP will have the same size and structure asqueue j. For each state in queue j, regardless of the state of its MAP , a new setΩi is created. Since sj = 1 this would results in an aggregated MAP of (Nj + 1)states. The sets and entrance states are now obtained by taking the union over allstates s∗j−1, i.e., the states of the MAP of queue j

Ωi =⋃s∗j−1

(s∗j−1, i, 1)

, σi =

⋃s∗j−1

(s∗j−1, i, 1)

.

Here, Ωi corresponds to the states with nj = i, e.g., Ω0 corresponds to an emptyqueue j, and ΩNj corresponds to a full queue j. Similarly, we can aggregate queuej to an (Nj + 1)-state MSP according to

Ωi =⋃s∗j

(1, i, s∗j )

, σi =

⋃s∗j

(1, i, s∗j )

.

These three different settings are used to form five different approximations for queuei. In the first three approximations we analyse queue i as a single MAP/MSP/1/Niqueue where the MAP and MSP are formed using one of the three settings. In thelast two approximations we analyse the 3-tandem network of queue i − 1, queue iand queue i+ 1 and use the two-state and four-state aggregations to form the MAPof queue i−1 and the MSP of queue i+ 1. We will denote these five approximationby two-state single, four-state single, buffer-sized single, two-state tandem, and four-state tandem and we refer to these approximations by the abbreviations 2SS, 4SS,Buf, 2ST, and 4ST respectively.

Next, we consider 8 different tandem networks of M/M/1/N queues given inTable 8.1. The terms used in this table refer to the properties of the queues in thetandem network, e.g., if the service rates are balanced, all service rates are equal,but if the service rates are increasing, we mean that µ(j) ≥ µ(k), if j > k, and ifthe service rates are decreasing, µ(j) ≤ µ(k), if j > k. Cases 1 and 2 consider anetwork with balanced service rates and buffer sizes, but the service rates in Case2 are higher to decrease the server utilisation ρ = λ

µ in the network. Cases 3 and


Service Rates (µ) Buffer (N)

Case 1 Balanced (low) Balanced

Case 2 Balanced (high) Balanced

Case 3 Decreasing Balanced

Case 4 Decreasing Increasing

Case 5 Increasing Balanced

Case 6 Increasing Decreasing

Case 7 Alternating (high-low) Alternating (low-high)

Case 8 Alternating (low-high) Alternating (high-low)

Table 8.1: 8 different cases for the tandem network of M/M/1/N queues

4 both consider a tandem network with decreasing service rates, i.e., µ(j) < µ(k) ifj > k. The buffer sizes in Case 4 are increasing in size while the buffer sizes in Case3 are balanced and equal to the buffer size of queue M in Case 4. Cases 5 and 6 bothconsider a tandem network with increasing service rates. The buffer sizes in Case 6are decreasing in size, while the buffer sizes in Case 5 are balanced and equal to thebuffer size of queue 1 in Case 6. Finally, Cases 7 and 8 describe a tandem networkwith alternating service rates and buffer sizes, starting with high service rates andsmall buffers in Case 7 and low service rates and large buffers in Case 8.

Figures 8.5 and 8.6 show the actual errors and relative errors made in the ap-proximations compared to simulation results for a tandem network of M = 5 queues.These figures show that for Cases 1 through 6, the approximations are better forhigher number queues. Figure 8.6 shows that approximations 2SS and 4SS show analternating relative error in Cases 7 and 8, while the approximations Buf, 2ST, and4ST result in a (slightly) alternating actual error compared to the simulation results.In general, the approximations based on the buffer-sized aggregation perform best.

Focusing on the approximations based on the buffer-sized aggregation and com-paring cases pairwise, we see that based on Cases 1 and 2, the approximations arebetter for a low server utilisation. Cases 3 and 4 show that an increasing buffersize gives better results than balanced buffer size when service rates are decreasing.Furthermore, balanced buffer sizes outperforms decreasing buffer sizes when the ser-vice rates are increasing, as is shows by Cases 5 and 6. Finally, Cases 7 and 8 showthat the orientation of an alternating tandem network, i.e. starting with high or lowservice rate, has little influence on the approximations. The best approximationsfor a tandem network of M = 5 M/M/1/N queues are obtained in Cases 2 and 5,i.e., a tandem network with balanced service rates and buffer sizes, but with low


0 1 · · · L − 1 L · · · U

L · · · U U + 1 · · · N − 1 N


µL


Figure 8.3: Transition Diagram for a single two-stage M/M/1/N threshold queue.

server utilisation (Case 2), and a tandem network with increasing service rates andbalanced buffer sizes (Case 5).

Figure 8.7 shows the actual and relative errors made by the approximation basedon buffer-sized aggregation compared to simulation results for a tandem network ofM = 10 and M = 20 queues. It is shown that the approximations become better forCase 3 when the size of the network increases. For other cases, the approximationsbecome less accurate when the size of the network increases.

8.4.2 A tandem of two-stage M/M/1/N threshold queues

We consider a tandem queueing network of two-stage M/M/1/N threshold queues.The arrival rate to the first queue is λ and the buffer size of queue i is Ni. Theservice rate of queue i depends on its stage. The stage, si, of queue i is controlledby a threshold policy based on the queue length ni. If queue i is in stage si anda departure or an arrival causes the queue length to drop below Li,si or to exceedUi,si , the stage changes. In a two-stage M/M/1/N threshold queue, there are twostages and the changes between these stages are depicted in the transition diagramin Figure 8.3. We assume

Li,1 = 0, Li,2 = Li, Ui,1 = Ui, Ui,2 = Ni.

and0 < Li ≤ Ui < Ni.

The stage dependent service rates of queue i are given in Table 8.2.

Stage Service Rate

si = 1 µH

si = 2 µL

Table 8.2: The stage dependent service rates for queue i.

We approximate a single queue in the tandem network using three different MAPaggregations, namely a four-state aggregation, a six-state aggregation, and a buffer-sized aggregation. The sets and entrance states of these aggregations are given in


detail in Appendix 8.A, here we will focus on general idea of the three aggregationand give some details.

Four-state aggregation The four-state aggregation is the simplest of the threeaggregation discussed in this section, it is comparable to the two-state aggregationfor the tandem of M/M/1/N queues. We partition the state space of the MAP ofqueue j+ 1, with states (s∗j−1, nj , sj), into three subsets. The first subsets describesan empty queue j (starvation) and contains all states with nj = 0 and sj = 1. Thesecond subset describes the queue containing customers and having service rate µHand contains all states with 1 ≤ nj ≤ Uj and sj = 1. The third and final subsetcontains all states with Lj ≤ nj ≤ Nj and sj = 2 and describes a queue withcustomers and having service rate µL.

Note that the second subset can be entered through states with nj = 1, andstates with nj = Lj − 1. We distinguish these two possibilities by making a four-state aggregation with sets and entrance states as in equation (8.10)

A similar approach is used to aggregate the MSP of queue j − 1. We partitionits state space in three subsets:

• states with 0 ≤ nj ≤ Uj and sj = 1,

• states with Lj ≤ nj ≤ Nj − 1andsj = 2,

• and states with nj = Nj and sj = 2 (blocking).

Next, we note that the second subset can be entered through Uj + 1 or Nj − 1 anddistinguish between these to possibilities by defining a four-state aggregation. Thesets and entrance states are given in equation (8.11)

Six-state aggregation The six-state aggregation combines both four-state aggre-gations described above. We partition the state space of the MAP of queue j+1 (orthe MSP of queue j-1) into four subsets:

• states with nj = 0 and sj = 1,

• states with 1 ≤ nj ≤ Uj and sj = 1,

• states with Lj ≤ nj ≤ Nj = 1 and sj = 2,

• and states with nj = Nj and sj = 2,

where the first subset would indicate starvation of queue j (MAP ), and the lastsubset would indicate blocking by queue j (MSP ). We note that the second subsetcan be entered through states with nj = 1, and states with nj = Lj − 1, and thatthe third subset can be entered through states with nj = Uj + 1, and states withnj = Nj − 1. We distinguish between these possibilities and define the six-stateaggregation by the sets and entrance states for the MAP in (8.12) and the MSP in(8.13).


Buffer-sized aggregation Here, we use the same strategy as the buffer-sized ag-gregation for a tandem of M/M/1/N queues. We aggregate the MAP of queue j+1(or MSP of queue j-1) to MAP (or MSP ) which had the same size and structure asqueue j. After aggregation, this MAP (MSP ) will be a state-dependent two-stageM/M/1/Nj threshold queue. The sets and entrance states are defined in (8.14) forMAP and in (8.15) for the MSP .

These three different settings are used to form five different approximations forqueue i. In the first three approximations we analyse queue i as a single two-stageMAP/MSP/1/Ni threshold queue where the MAP and MSP are formed usingone of the three settings. In the last two approximations we analyse the 3-tandemnetwork of queue i− 1, queue i and queue i+ 1 and use the four-state and six-stateaggregations to form the MAP of queue i − 1 and the MSP of queue i + 1. Wewill denote these five approximation by four-state single, six-state single, buffer-sizedsingle, four-state tandem, and six-state tandem and we refer to these approximationsby the abbreviations 4SS, 6SS, Buf, 4ST, and 6ST respectively.

Next, we consider 8 different tandem networks of two-stage M/M/1/N thresholdqueues given in Table 8.3. The terms used in this table refer to the properties of thequeues in the tandem network, e.g., if the buffer sizes are balanced, then Nj = Nk,j 6= k, and if the buffer sizes are increasing, we mean that Nj ≥ Nk, if j > k. Cases 1and 2 consider a tandem network with balanced system parameters, but the servicerates is Case 2 are higher to decrease the server utilisation in the network. Cases 3and 4 both consider a tandem network with decreasing service rates and increasingupper thresholds and buffer sizes. In Case 4 the lower threshold are balanced whilethe lower threshold in Case 3 are increasing. Cases 5 and 6 both consider a tandemnetwork with increasing service rates and decreasing upper thresholds and buffersizes. In Case 6 the lower thresholds are balanced while the lower thresholds inCase 5 are decreasing. Finally, Cases 7 and 8 describe a tandem network withalternating service rates, lower and upper thresholds, and buffer sizes, starting withhigh service rates, low thresholds, and small buffers in Case 7 and low service rates,high thresholds, and large buffers in Case 8.

Figures 8.8 and 8.9 show the actual and relative errors made in the approxima-tions compared to the discrete event simulation for a tandem network of M = 5queues. These figures show that for Cases 1 to 6, the approximations are betterfor higher number queues. In general, the approximations based on the buffer-sizedaggregation perform best.

Focusing on the approximations based on the buffer-sized aggregation and com-paring cases pairwise, we see that based on Cases 1 and 2, the approximations arebetter for a high server utilisation (Case 1). Cases 3 and 4 show that increasing lowerthresholds (Case 3) give better results than balanced lower thresholds (Case 4) whenservice rates are decreasing. Furthermore, balanced lower thresholds (Case 6) out-perform decreasing lower thresholds (Case 5) when the service rates are increasing,as is shown by Cases 5 and 6. Finally, Cases 7 and 8 show that the orientation ofan alternating tandem network, i.e. starting with high or low service rate, has little


Service Rates(µH , µL)

LowerThresholds (L)

UpperThresholds (U)

Buffer (N)

Case 1 Balanced (low) Balanced Balanced Balanced

Case 2 Balanced (high) Balanced Balanced Balanced

Case 3 Decreasing Increasing Increasing Increasing

Case 4 Decreasing Balanced Increasing Increasing

Case 5 Increasing Decreasing Decreasing Decreasing

Case 6 Increasing Balanced Decreasing Decreasing

Case 7 Alternating(high-low)

Alternating(low-high)



Case 8 Alternating(low-high)

Alternating(high-low)



Table 8.3: 8 different cases for the tandem network of two-stage M/M/1/N thresholdqueues.

influence on the approximations. The best approximations for a tandem network ofM = 5 two-stage M/M/1/N threshold queues are obtained in Cases 1, 7, and 8, i.e.,a tandem network with alternating servers (Cases 7 and 8), and a tandem networkwith balanced servers but high utilisation (Case 1).

8.4.3 A tandem of four-stage M/M/1/N feedback thresholdqueues

We consider a tandem queueing network of four-stage M/M/1/N feedback thresholdqueues in which queue i has buffer size Ni. Each queue is controlled by a thresholdpolicy which determines the stage, sj , of the queue based on its queue length. Ifqueue i is in stage si and a departure or an arrival causes the queue length to dropbelow Li,si or to exceed Ui,si , the stage of changes. The stage sj , j = 2, . . . , Nj ,determines both the service rate of queue j as well as the service rate of queue j−1.The stage of queue 1 determines both the service rate of queue 1 as well as the arrivalrate to queue 1. In a four-stage M/M/1/N feedback threshold queue, there are fourstages and the changes between these stages are depicted in the transition diagramin Figure 8.4. We assume

Li,1 = 0, Li,2 = Li,4 = Li,µ, Li,3 = Li,λ,

andUi,1 = Ui,3 = Ui,µ, Ui,2 = Ui,λ, Ui,4 = Ni,


with0 < Li,λ < Li,µ ≤ Ui,µ < Ui,λ < Ni.

The arrival rate to the first queue depends on the stage of queue 1 and is givenin Table 8.4. The service rates of queue i, i = 2, . . . ,M − 1, depend on their own

Stage of queue 1 Arrival Rate

s1 = 1 λH

s1 = 2 λH

s1 = 3 λL

s1 = 4 λL

Table 8.4: The stage dependent arrival rates to queue 1.

stage, as well as the stage of the downsteam queue. In Table 8.5 we give the servicerates of queue i, i = 2, . . . ,M − 1. Finally, the service rates of the last queue onlydepend on the stage of queue M and are given in Table 8.6.

We approximate a single queue in the tandem network using three differentMAP aggregations, namely an eight-state aggregation, a ten-state aggregation, anda buffer-sized aggregation.

Eight-state aggregation The eight-state aggregation is primarily based on par-titioning the state space in four subsets according to its (four) stages, sj = 1, . . . , 4.In case of a MAP we would also want to distinguish the empty queue (starvation)giving a total if five subsets. Next we distinguish between the entrance states bynoting that sj = 1 can be entered through states with nj = 1, nj = Lj,λ − 1, ornj,mu − 1 and that sj = 4 can be entered through states with nj = Uj,µ + 1 andnj = Uj,λ+ 1. Taking all possibilities into account we obtain the 8 sets and entrancestates defined in (8.16) for a MAP .

For the MSP we distinguish the full queue (blocking) instead of an empty queueand obtain the aggregation defined by the 8 sets and entrance states in (8.17).

Ten-state aggregation The ten-state aggregation combines both eight-state ag-gregation by distinguish both the empty queue, as well as the full queue. It is definedby the ten sets and entrance states in (8.18) and (8.19) for the MAP and MSP ,respectively.

Buffer-sized aggregation Here, we use the same strategy as the buffer-sized ag-gregation for a tandem of M/M/1/N queues. We aggregate the MAP of queue j+1(or MSP of queue j-1) to MAP (or MSP ) which had the same size and structure asqueue j. After aggregation, this MAP (MSP ) will be a state-dependent four-stage


s=

10

1···

Lλ

−1

Lλ

···Lµ

−1

Lµ

···Uµ

s=

2Lµ

···Uµ

Uµ

+1

···Uλ

s=

3Lλ

···Lµ

−1

Lµ

···Uµ

s=

4Lµ

···Uµ

Uµ

+1

···Uλ

Uλ

+1

···N

−1

N

Fig

ure

8.4

:T

ransitio

nD

iagra

mfo

rth

ea

single

four-sta

geM/M

/1/N

feedback

thresh

old

queu

e.


Stage of queue i Stage of queue i + 1 Service rate

si = 1 or si = 3 si+1 = 1 or si+1 = 2 µHH

si = 1 or si = 3 si+1 = 3 or si+1 = 4 µHL

si = 2 or si = 4 si+1 = 1 or si+1 = 2 µLH

si = 2 or si = 4 si+1 = 3 or si+1 = 4 µLL

Table 8.5: The stage dependent service rates of queue i, i = 2, . . . ,M − 1.

Stage of queue 3 Service rate

s3 = 1 µH

s3 = 2 µL

s3 = 3 µH

s3 = 4 µL

Table 8.6: The stage dependent service rates of queue M .

M/M/1/Nj feedback threshold queue. The sets and entrance states are defined in(8.20) for MAP and in (8.21) for the MSP .

With these three settings we approximate queue i as a single MAP/MSP/1/Nifeedback threshold queue. We will denote the three approximations by eight-statesingle, ten-state single and buffer single and we refer to these approximations by theabbreviations 8SS, 10SS, and Buf respectively.

Next, we consider 6 different tandem networks of four-stage M/M/1/N feedbackthreshold queues given in Table 8.7. The terms used in this table refer to theproperties of the queues in the tandem network, e.g., if the buffer sizes are balanced,then Nj = Nk, j 6= k, and if the buffer sizes are increasing, we mean that Nj ≥ Nk,if j > k. Cases 1, 2, 3 and 4 consider a tandem network with balanced systemparameters, but with high service rates in Cases 1 and 3, and low service rates inCases 2 and 4. Furthermore, Cases 1 and 2 consider small buffer sizes and lowthresholds, while Cases 3 and 4 consider large buffer sizes and high thresholds. Case5 considers a network with decreasing service rates, and with increasing lower andupper thresholds and buffer sizes. Case 6 considers a network with increasing servicerates, and with decreasing lower and upper thresholds and buffer sizes.

Figures 8.10 and 8.11 show the actual errors and relative errors made in theapproximations compared to the discrete event simulation for a tandem network of


Service Rates(µHH , µHL,µLH , µLL)

LowerThresholds(Lλ, Lµ)

UpperThresholds(Uλ, Uµ)

Buffer (N)

Case 1 Balanced (high) Balanced Balanced Balanced (low)

Case 2 Balanced (low) Balanced Balanced Balanced (low)

Case 3 Balanced (high) Balanced Balanced Balanced (high)

Case 4 Balanced (low) Balanced Balanced Balanced (high)

Case 5 Decreasing Increasing Increasing Increasing

Case 6 Increasing Decreasing Decreasing Decreasing

Table 8.7: 6 different cases for the tandem network of four-stage M/M/1/N feedbackthreshold queues.

M = 5 queues. These figures show that the approximations are better for highernumber queues. In general, the approximations based on the buffer-sized aggregationperform best.

Focusing on the approximations based on the buffer-sized aggregation and com-paring cases pairwise, we see that based on Cases 1 and 2, and Cases 3 and 4, that theapproximations are better for a high server utilisation (Cases 2 and 4). ComparingCases 1 and 3, and Cases 2 and 4, we see that the size buffer sizes have little effect onthe relative errors. Finally, comparing Cases 5 and 6 we see that the approximationsare better when the service rates are increasing (Case 6). The best approximationsfor a tandem network of M = 5 four-stage M/M/1/N feedback threshold queues areobtained in Cases 2, 4 and 6, i.e., a tandem network with increasing service rates(Case 6), and a tandem network with balanced servers and high server utilisation(Cases 2 and 4).


Case 1

Case 2

Case 3

Case 4

1 2 3 4 5−0.1

−0.05

0

0.05

0.1

0.15

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

75%

100%

125%

150%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

75%

100%

Queue

Rel.Error

2SS4SSBuf2ST4ST

Figure 8.5: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 M/M/1/N queues


Case 5

Case 6

Case 7

Case 8

1 2 3 4 5−0.05

0

0.05

0.1

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

Queue

Rel.Error

2SS4SSBuf2ST4ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

Queue

Error

2SS4SSBuf2ST4ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

2SS4SSBuf2ST4ST

Figure 8.6: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 M/M/1/N queues


10-Tandem

10-Tandem

20-Tandem

20-Tandem

1 2 3 4 5 6 7 8 9 10−0.1

−0.05

0

0.05

0.1

0.15

Queue

Error

Case 1Case 2Case 3Case 4

1 2 3 4 5 6 7 8 9 10−25%

0%

25%

50%

75%

Queue

Rel.Error


1 2 3 4 5 6 7 8 9 10−0.05

0

0.05

0.1

0.15

Queue

Error


1 2 3 4 5 6 7 8 9 10−25%

0%

25%

50%

75%

Queue

Rel.Error


2 4 6 8 10 12 14 16 18 20−0.1

−0.05

0

0.05

0.1

0.15

0.2

Queue

Error


2 4 6 8 10 12 14 16 18 20−25%

0%

25%

50%

75%

100%

Queue

Rel.Error


2 4 6 8 10 12 14 16 18 20−0.1

0

0.1

0.2

0.3

Queue

Error


2 4 6 8 10 12 14 16 18 20−25%

0%

25%

50%

75%

100%

Queue

Rel.Error


Figure 8.7: Actual and relative error compared to simulation results for a tandemnetwork of M = 10 and M = 20 M/M/1/N queues


Case 1

Case 2

Case 3

Case 4

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

0.25

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

100%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 50

0.02

0.04

0.06

0.08

0.1

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 50%

25%

50%

75%

100%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 50

0.02

0.04

0.06

0.08

0.1

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

100%

Queue

Rel.Error

4SS6SSBuf4ST6ST

Figure 8.8: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 two-stage M/M/1/N threshold queues


Case 5

Case 6

Case 7

Case 8

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

0.25

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

4SS6SSBuf4ST6ST

1 2 3 4 5−0.1

0

0.1

0.2

0.3

0.4

0.5

Queue

Error

4SS6SSBuf4ST6ST

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

4SS6SSBuf4ST6ST

Figure 8.9: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 two-stage M/M/1/N threshold queues


Case 1

Case 2

Case 3

1 2 3 4 5−0.1

−0.05

0

0.05

0.1

0.15

0.2

0.25

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

8-SS10-SSBuf

1 2 3 4 5−0.05

0

0.05

0.1

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

8-SS10-SSBuf

1 2 3 4 5−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

75%

100%

125%

Queue

Rel.Error

8-SS10-SSBuf

Figure 8.10: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 four-stage M/M/1/N feedback threshold queues.


In this chapter, we have presented an aggregation method to reduce the size ofMarkovian Arrival Processes (MAP) and Markovian Service Processes (MSP). Thisaggregation method will approximate a number of states from the MAP by a singleexponential state, reducing its size. With these MAP aggregations we presented aniterative aggregation method to approximate the marginal queue length distributionof, and mean sojourn time in, an arbitrary queue in a tandem queueing network. The


Case 4

Case 5

Case 6

1 2 3 4 5−0.1

−0.05

0

0.05

0.1

0.15

0.2

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

75%

Queue

Rel.Error

8-SS10-SSBuf

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

75%

100%

Queue

Rel.Error

8-SS10-SSBuf

1 2 3 4 5−0.05

0

0.05

0.1

0.15

0.2

0.25

Queue

Error

8-SS10-SSBuf

1 2 3 4 5−25%

0%

25%

50%

Queue

Rel.Error

8-SS10-SSBuf

Figure 8.11: Actual and relative error compared to simulation results for a tandemnetwork of M = 5 four-stage M/M/1/N feedback threshold queues.

service and arrival rates in this network are exponential but can be state dependentor controlled by a threshold policy.

The results given by the iterative aggregation method are compared to simulationresults for a tandem network of M/M/1/N queues, a tandem network of two-stageM/M/1/N threshold queues, and a tandem network of four-stage M/M/1/N feed-back threshold queues. Results show that the approximation give better results forhigher numbered queues compared to lower numbered queue. The iterative aggre-gation method behaves well for tandem networks with balanced service rates, or


tandem network with increasing service rates. The iterative aggregation methodbecomes less accurate in larger tandem networks of M/M/1/N queues.

The best results were obtained using the so-called buffer aggregation, however,the approximation might be improved by using other MAP aggregations.

8.A Sets and Entrance states

8.A.1 A tandem of M/M/1/N queues

Two-state aggregation In the two-state aggregation, the MAP of queue j + 1,with states (s∗j−1, nj , sj), is aggregated to a two-state MAP by defining the followingsets and entrance states

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

,

Ω2 =⋃s∗j−1

Nj⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

.

Similarly, the MSP of queue j − 1, with states (sj−1, nj , s∗j ), is aggregated to a

two-state MSP with

Ω1 =⋃s∗j

Nj−1⋃nj=0

(1, nj , s

∗j ), σ1 =

⋃s∗j

(1, Nj − 1, s∗j )

,

Ω2 =⋃s∗j

(1, Nj , s

∗j ), σ2 =

⋃s∗j

(1, Nj , s

∗j ).

Four-state aggregation In the four-state aggregation, we aggregate queue j toa four-state MAP with

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

,

Ω2 =⋃s∗j−1

Nj−1⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,

Ω3 =⋃s∗j−1

Nj−1⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Nj − 1, 1)

,

Ω4 =⋃s∗j−1

(s∗j−1, Nj , 1)

σ4 =

⋃s∗j

(s∗j−1, Nj , 1)

.

or a four-state MSP with

Ω1 =⋃s∗j

(1, 0, s∗j )

, σ1 =

⋃s∗j

(1, 0, s∗j )

,

8.A Sets and Entrance states 133

Ω2 =⋃s∗j

Nj−1⋃nj=1

(1, nj , s

∗j ), σ2 =

⋃s∗j

(1, 1, s∗j )

,

Ω3 =⋃s∗j

Nj−1⋃nj=1

(1, nj , s

∗j ), σ3 =

⋃s∗j

(1, Nj − 1, s∗j )

,

Ω4 =⋃s∗j

(1, Nj , s

∗j )

σ4 =⋃s∗j

(1, Nj , s

∗j ).

Remark 8.5 (Redundant sets.). Note that whenNj = 2, then Ω2 = Ω3 and σ2 = σ3.In the aggregation steps we take this into account and remove any redundant sets.If Nj = 1, then Ω1 and Ω3 would coincide, as well as Ω2 and Ω4. The four-stateaggregation would then reduce to a two-state aggregation.

Buffer-sized aggregation In this last case, we aggregate queue j to an (Nj + 1)-state MAP by choosing the sets and entrance states according to

Ωi =⋃s∗j−1

(s∗j−1, i, 1)

, σi =

⋃s∗j−1

(s∗j−1, i, 1)

,

or an (Nj + 1)-state MSP with

Ωi =⋃s∗j

(1, i, s∗j )

, σi =

⋃s∗j

(1, i, s∗j )

.

8.A.2 A tandem of two-stage M/M/1/N threshold queues

Four-state aggregation The MAP of queue j + 1, with states (s∗j−1, nj , sj), isaggregated to a four-state MAP by defining the following sets and entrance states

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

, (8.10)

Ω2 =⋃s∗j−1

Uj⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,

Ω3 =⋃s∗j−1

Uj⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Lj − 1, 1)

,

Ω4 =⋃s∗j−1

Nj⋃nj=Lj

(s∗j−1, nj , 2)

, σ4 =

⋃s∗j−1

(s∗j−1, Uj + 1, 2)

.


Similarly, the MSP of queue j − 1, with states (sj−1, nj , s∗j ), is aggregated to a

four-state MSP with

Ω1 =⋃s∗j

Uj⋃nj=0

(1, nj , s

∗j ), σ1 =

⋃s∗j

(1, Lj − 1, s∗j )

, (8.11)

Ω2 =⋃s∗j

Nj−1⋃nj=Lj

(2, nj , s

∗j ), σ2 =

⋃s∗j

(2, Uj + 1, s∗j )

,

Ω3 =⋃s∗j

Nj−1⋃nj=Lj

(2, nj , s

∗j ), σ3 =

⋃s∗j

(2, Nj − 1, s∗j )

,

Ω4 =⋃s∗j

(2, Nj , s

∗j ), σ4 =

⋃s∗j

(2, Nj , s

∗j ).

Six-state aggregation In the six-state aggregation, we aggregate queue j to asix-state MAP with

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

, (8.12)

Ω2 =⋃s∗j−1

Uj⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,

Ω3 =⋃s∗j−1

Uj⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Lj − 1, 1)

,

Ω4 =⋃s∗j−1

Nj−1⋃nj=Lj

(s∗j−1, nj , 2)

, σ4 =

⋃s∗j−1

(s∗j−1, Uj + 1, 2)

,

Ω5 =⋃s∗j−1

Nj−1⋃nj=Lj

(s∗j−1, nj , 2)

, σ5 =

⋃s∗j−1

(s∗j−1, Nj − 1, 2)

,

Ω6 =⋃s∗j−1

(s∗j−1, Nj , 2)

, σ6 =

⋃s∗j−1

(s∗j−1, Nj , 2)

,

or a six-state MSP using

Ω1 =⋃s∗j

(1, 0, s∗j )

, σ1 =

⋃s∗j

(1, 0, s∗j )

, (8.13)

Ω2 =⋃s∗j

Uj⋃nj=1

(1, nj , s

∗j ), σ2 =

⋃s∗j

(1, 1, s∗j )

,


Ω3 =⋃s∗j

Uj⋃nj=1

(1, nj , s

∗j ), σ3 =

⋃s∗j

(1, Lj − 1, s∗j )

,

Ω4 =⋃s∗j

Nj−1⋃nj=Lj

(2, nj , s

∗j ), σ4 =

⋃s∗j

(2, Uj + 1, s∗j )

,

Ω5 =⋃s∗j

Nj−1⋃nj=Lj

(2, nj , s

∗j ), σ5 =

⋃s∗j

(2, Nj − 1, s∗j )

,

Ω6 =⋃s∗j

(2, Nj , s

∗j ), σ6 =

⋃s∗j

(2, Nj , s

∗j ).

Remark 8.6 (Redundant sets). Note that in both the four-state aggregation andthe six-state aggregation, sets and entrance states may coincide. For example in thesix-state aggregation, if Lj = 2, then Ω2 = Ω3 and σ2=σ3, and if Uj = Nj − 2, thenΩ4 = Ω5 and σ4 = σ5. In the aggregation steps we take this into account and removeany redundant sets.

Buffer-sized aggregation In this last case, we aggregate queue j to a MAP bychoosing the sets and entrance states according to

Ωi,k =⋃s∗j−1

(s∗j−1, i, k)

, σi,k =

⋃s∗j−1

(s∗j−1, i, k)

, (8.14)

or a MSP with

Ωi,k =⋃s∗j

(k, i, s∗j )

, σi,k =

⋃s∗j

(k, i, s∗j )

. (8.15)

8.A.3 A tandem of four-stage M/M/1/N feedback thresholdqueues

Eight-state aggregation In the eight-state aggregation, the MAP of queue j+1,with states (s∗j−1, nj , sj), is aggregated into an eight-state MAP by choosing the setsand entrance states according to

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

, (8.16)

Ω2 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,


Ω3 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Lj,λ − 1, 1)

,

Ω4 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ4 =

⋃s∗j−1

(s∗j−1, Lj,µ − 1, 1)

,

Ω5 =⋃s∗j−1

Uj,λ⋃nj=Lj,µ

(s∗j−1, nj , 2)

, σ5 =

⋃s∗j−1

(s∗j−1, Uj,µ + 1, 2)

,

Ω6 =⋃s∗j−1

Uj,µ⋃nj=Lj,λ

(s∗j−1, nj , 3)

, σ6 =

⋃s∗j−1

(s∗j−1, Lj,µ − 1, 3)

,

Ω7 =⋃s∗j−1

Nj⋃nj=Lj,µ

(s∗j−1, nj , 4)

, σ7 =

⋃s∗j−1

(s∗j−1, Uj,µ + 1, 4)

,

Ω8 =⋃s∗j−1

Nj⋃nj=Lj,µ

(s∗j−1, nj , 4)

, σ8 =

⋃s∗j−1

(s∗j−1, Uj,λ + 1, 4)

.

Similarly, the MSP of queue j − 1, with states (sj−1, nj , s∗j ), is aggregated to an

eight-state MSP with

Ω1 =⋃s∗j

Uj,µ⋃nj=0

(1, nj , s

∗j ), σ1 =

⋃s∗j

(1, Lj,λ − 1, s∗j )

, (8.17)

Ω2 =⋃s∗j

Uj,µ⋃nj=0

(1, nj , s

∗j ), σ2 =

⋃s∗j

(1, Lj,µ − 1, s∗j )

,

Ω3 =⋃s∗j

Uj,λ⋃nj=Lj,µ

(2, nj , s

∗j ), σ3 =

⋃s∗j

(2, Uj,µ + 1, s∗j )

,

Ω4 =⋃s∗j

Uj,µ⋃nj=Lj,λ

(3, nj , s

∗j ), σ4 =

⋃s∗j

(3, Lj,µ − 1, s∗j )

,

Ω5 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ5 =

⋃s∗j

(4, Uj,µ + 1, s∗j )

,

Ω6 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ6 =

⋃s∗j

(4, Uj,λ + 1, s∗j )

,


Ω7 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ7 =

⋃s∗j

(4, Nj − 1, s∗j )

,

Ω8 =⋃s∗j

(4, Nj , s

∗j ), σ8 =

⋃s∗j

(4, Nj , s

∗j ).

Ten-state aggregation In the ten-state aggregation, we aggregate queue j to aten-state MAP with

Ω1 =⋃s∗j−1

(s∗j−1, 0, 1)

, σ1 =

⋃s∗j−1

(s∗j−1, 0, 1)

, (8.18)

Ω2 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ2 =

⋃s∗j−1

(s∗j−1, 1, 1)

,

Ω3 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ3 =

⋃s∗j−1

(s∗j−1, Lj,λ − 1, 1)

,

Ω4 =⋃s∗j−1

Uj,µ⋃nj=1

(s∗j−1, nj , 1)

, σ4 =

⋃s∗j−1

(s∗j−1, Lj,µ − 1, 1)

,

Ω5 =⋃s∗j−1

Uj,λ⋃nj=Lj,µ

(s∗j−1, nj , 2)

, σ5 =

⋃s∗j−1

(s∗j−1, Uj,µ + 1, 2)

,

Ω6 =⋃s∗j−1

Uj,µ⋃nj=Lj,λ

(s∗j−1, nj , 3)

, σ6 =

⋃s∗j−1

(s∗j−1, Lj,µ − 1, 3)

,

Ω7 =⋃s∗j−1

Nj−1⋃nj=Lj,µ

(s∗j−1, nj , 4)

, σ7 =

⋃s∗j−1

(s∗j−1, Uj,µ + 1, 4)

,

Ω8 =⋃s∗j−1

Nj−1⋃nj=Lj,µ

(s∗j−1, nj , 4)

, σ8 =

⋃s∗j−1

(s∗j−1, Uj,λ + 1, 4)

,

Ω9 =⋃s∗j−1

Nj−1⋃nj=Lj,µ

(s∗j−1, nj , 4)

, σ9 =

⋃s∗j−1

(s∗j−1, Nj − 1, 4)

,

Ω10 =⋃s∗j−1

(s∗j−1, Nj , 4)

, σ10 =

⋃s∗j−1

(s∗j−1, Nj , 4)

.

or a ten-state MSP using

Ω1 =⋃s∗j

(1, 0, s∗j )

, σ1 =

⋃s∗j

(1, 0, s∗j )

, (8.19)


Ω2 =⋃s∗j

Uj,µ⋃nj=1

(1, nj , s

∗j ), σ2 =

⋃s∗j

(1, 1, s∗j )

,

Ω3 =⋃s∗j

Uj,µ⋃nj=1

(1, nj , s

∗j ), σ3 =

⋃s∗j

(1, Lj,λ − 1, s∗j )

,

Ω4 =⋃s∗j

Uj,µ⋃nj=1

(1, nj , s

∗j ), σ4 =

⋃s∗j

(1, Lj,µ − 1, s∗j )

,

Ω5 =⋃s∗j

Uj,λ⋃nj=Lj,µ

(2, nj , s

∗j ), σ5 =

⋃s∗j

(2, Uj,µ + 1, s∗j )

,

Ω6 =⋃s∗j

Uj,µ⋃nj=Lj,λ

(3, nj , s

∗j ), σ6 =

⋃s∗j

(3, Lj,µ − 1, s∗j )

,

Ω7 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ7 =

⋃s∗j

(4, Uj,µ + 1, s∗j )

,

Ω8 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ8 =

⋃s∗j

(4, Uj,λ + 1, s∗j )

,

Ω9 =⋃s∗j

Nj−1⋃nj=Lj,µ

(4, nj , s

∗j ), σ9 =

⋃s∗j

(4, Nj − 1, s∗j )

,

Ω10 =⋃s∗j

(4, Nj , s

∗j ), σ10 =

⋃s∗j

(4, Nj , s

∗j ).

Remark 8.7 (Redundant sets). Note that both the eight-state aggregation and theten-state aggregation may contain redundant sets for certain system parameters, seealso Remark 8.6. In the aggregation steps we take this into account and remove anyredundant sets.

Buffer aggregation In this last case, we aggregate queue j to a MAP by choosingthe sets and entrance states according to

Ωi,k =⋃s∗j−1

(s∗j−1, i, k)

, σi,k =

⋃s∗j−1

(s∗j−1, i, k)

, (8.20)

or a MSP with

Ωi,k =⋃s∗j

(k, i, s∗j )

, σi,k =

⋃s∗j

(k, i, s∗j )

. (8.21)

Concluding remarks and Further research

Concluding remarks

The literature on traffic models is vast, but it lacks a good model capturing modern-day traffic behaviour. In this thesis we have introduced two queueing models thatcapture the hysteretic behaviour of highway traffic: the two-stage M/M/1 thresholdqueue and the four-stage M/M/1 feedback threshold queue. Both queueing modelswere validated using empirical data of highway traffic in Denmark. These resultsshow that both queueing systems capture the fundamental diagram of modern-daytraffic.

The threshold queues were extended to form the PH/PH/1 multi-thresholdqueue for which the stationary distribution was obtained. Results show that furhterfine-tuning of the fundamental diagram can be achieved by assuming that the ar-rival process and service process follow a Phase-Type distributions instead of theexponential distributions.

Next, both queueing systems were extended to tandem networks to give a bet-ter representation of highway traffic flows. Unfortunately, the state space of thesetandem models grows exponentially in the number of queues and it becomes ex-tremely difficult to obtain the stationary queue length distribution needed for thefundamental diagram. The fundamental diagram was obtained but was not fitted toempirical data due to the state space explosion. The fundamental diagrams obtainedfor the tandem networks show the characteristic capacity drop, but the results areless impressive than the results obtained with the single queue traffic models.

A new solution method was introduced to circumvent the problem of state spaceexplosion, namely the successive censoring algorithm. First, this algorithm signifi-cantly reduces the size of the Markov chain using a censoring technique. Next, thestationary distribution of the reduced Markov chain was obtained. This stationarydistribution is expanded in the final step of the algorithm to obtain the station-ary distribution of the original Markov chain. The successive censoring algorithmenables the analysis of three queue tandem networks and obtain the fundamentaldiagram of each individual queue.

In order to handle even larger tandem networks, a iterative aggregation methodwas introduced. The iterative aggregation method focuses on a single queue in atandem network and approximates its marginal queue length distribution and itsmean sojourn time. It iteratively aggregates the upstream network behaviour into aMarkovian Arrival Process, and the downstream network behaviour into a MarkovianService Process. Results show that the iterative aggregation method is more effective

140 Concluding remarks and Further research

for small networks than for large networks. Also, the iterative aggregation methodperforms better when queues in the tandem network are balanced or when servicerates are increasing from queue to queue.

Overall, we conclude that threshold queueing models describe modern-day high-way traffic; both the single queue traffic models and the tandem networks show thecharacteristic shape of the fundamental diagram. However, the tandem networks aremore difficult to analyse than the single queues and the results are less impressive.Therefore, we conclude that modern-day traffic is best modelled by the two-stagethreshold queue and the four-stage feedback threshold queue.

Further research

Our queueing models were validated with empirical data of highway traffic in theCopenhagen area in Denmark. These data sets showed the characteristic shape of thefundamental diagram but contained only few data points corresponding to extremecongested traffic. It would be advisable to see how our traffic models would performfor high density traffic.

The threshold queues are able to mimic modern-day traffic, but lack a physicalinterpretation of the parameters. With a physical interpretation at hand, it wouldnot be necessary to fit the queueing models with empirical data.

In this thesis, we have only used the iterative aggregation method to aggregate atandem network. The approximations provided by the aggregation method heavilydepend on the MAP aggregation that is used. By finding a good MAP aggregation,these approximations might be improved. Also, we focused on tandem networkwith Poisson arrivals and exponentially distribution service times. The iterativeaggregation method could be extended to more complex networks including mergingand splitting or by including Phase-Type distributions.

Bibliography

[1] K.I. Ahmed, M.E. Ben-Akiva, H.N. Koutsopoulos, and R.G. Mishalani. Modelsof freeway lane changing and gap acceptence behavior. In Proceedings of the13th International Symposium on Transportation and Traffic Theory, 1996.

[2] T. Altiok. Approximate analysis of exponential tandem queues with blocking.European Journal of Operational Research, 11:390–398, 1982.

[3] T. Altiok. Approximate analysis of queues in series with phase-type servicetimes and blocking. Operation Research, 37(4):601–610, 1989.

[4] T. Altiok and H.G. Perros. Open network of queue with blocking: Split andmerge configurations. IIE Transactions, 18(3):251–261, 1986.

[5] T. Altiok and H.G. Perros. Approximate analysis of arbitraty configurationsof open queueing networks with blocking. Annals of Operations Research,9:481–509, 1987.

[6] T. Altiok and R. Ranjan. Analysis of production lines with general servicetimes and finite buffers: A two-node decomposition approach. EngineeringCosts and Production Economics, 17:155–165, 1989.

[7] N. Baer, A. Al Hanbali, R.J. Boucherie, and J.C.W. van Ommeren. A suc-cessive censoring algorithm for a system of connected qbd-processes. Mem-orandum 2030, Department of Applied Mathematics, University of Twente,Enschede, The Netherlands, 2013.

[8] N. Baer, A. Al Hanbali, R.J. Boucherie, and J.C.W. van Ommeren. An iter-ative aggregation method for a tandem queueing network with finite buffers.Unpublished work, 2015.

[9] N. Baer, R.J. Boucherie, and J.C.W. van Ommeren. Threshold queueing de-scribes the fundamental diagram of uninterrupted traffic. Memorandum 2000,Department of Applied Mathematics, University of Twente, Enschede, TheNetherlands, 2012.

[10] N. Baer, R.J. Boucherie, and J.C.W. van Ommeren. The PH/PH/1 multi-threshold queue, volume 8499 of Lecture Notes in Computer Science, pages95–109. Springer International Publishing, 2014.

142 Bibliography

[11] M. Baykal-Gursoy, W. Xiao, and K. Ozbay. Modeling traffic flow interruptedby incidents. European Journal of Operational Research, 195:127–138, 2009.

[12] N. Bellomo and C. Dogbe. On the modeling of traffic and crowds: A survey ofmodels, speculations, and perspectives. SIAM Review, 53(3):409–463, 2011.

[13] R. Bierbooms, I.J.B.F. Adan, and M. van Vuuren. Approximate analysis ofsingle-server tandem queues with finite buffers. Annals of Operations Research,209:67–84, 2013.

[14] M. Boon. Polling Models - From theory to traffic intersections. PhD thesis,Eindhoven University of Technology, 2011.

[15] M. Brackstone and M. McDonald. Car-following: A historical review. Trans-portation Research - Part F, 2:181–196, 1999.

[16] L. Bright and P.G. Taylor. Calculating the equilibrium distribution in leveldependent quasi-birth-and-death processes. Communications in Statistics -Stochastic Models, 11(3):497–525, 1995.

[17] D.J. Buckley. A semi-poisson model of traffic flow. Transportation Science,2(2):107–133, 1968.

[18] W. Cao and W.J. Stewart. Iterative aggregation/dissagregation techniques fornearly uncoupled markov chains. Journal of the Association for ComputingMachinery, 32(3):702–719, 1985.

[19] S.R. Chakravarthy. A multi-server queueing model with markovian arrivals andmultiple thresholds. Asia-Pacific Journal of Operational Research, 24(2):223–243, 2007.

[20] R.E. Chandler, R. Herman, and E.W. Montroll. Traffic dynamics: Studies incar following. Operations Research, 6(2):165–184, 1958.

[21] S.H. Choi, B. Kim, K. Sohraby, and B.D. Choi. On matrix-geometric solutionof nested QBD chains. Queueing Systems, 43:5–28, 2003.

[22] C.F. Chou, L. Golubchik, and J.C.S. Lui. Multiclass multiserver thresholdbased systems: A study of noninstantaneous server activation. IEEE Trans-actions on Parallel and Distributed Systems, 18(1):96–110, 2007.

[23] D. Chowdhury, L. Santen, and A. Schadschneider. Statistical physics of vehic-ular traffic and some related systems. Physics Report, 329:199–329, 2000.

[24] F.R.B. Cruz and J.M. Smith. Approximate analysis of M/G/c/c state-dependent queueing networks. Computers & Operations Research, 34:2332–2344, 2007.

Bibliography 143

[25] F.R.B. Cruz, J.M. Smith, and R.O. Medeiros. An M/G/c/c state-dependentnetwork simulation model. Computers & Operations Research, 32:919–941,2005.

[26] Y. Dallery and S.B. Geshwin. Manufacturing flow line systems: A review ofmodels and analytical results. Queueing Systems, 12:3–94, 1992.

[27] J.M. Del Castillo and F.G. Benitez. On the functional form of the speed-densityrelationship I: General theory. Transportation Research - Part B, 29B(5):373–389, 1995.

[28] J.M. Del Castillo and F.G. Benitez. On the functional form of the speed-density relationship II: Empirical investigation. Transportation Research - PartB, 29B(5):391–406, 1995.

[29] J.S. Drake, J.L. Schofer, and A.D. May. A statistical analysis of speed-densityhypotheses. Highway Research Record, 156:53–87, 1967.

[30] D.R. Drew. Traffic Flow Theory and Control. McGraw-Hill, New York, 1968.

[31] J.C. Edie. Car-following and steady-state theory for noncongested traffic. Op-eration Research, 9(1):66–76, 1961.

[32] B.N. Feinberg and S.S. Chiu. A method to calculate steady-state distributionsof large markov chains by aggregating states. Operations Research, 35(2):282–290, 1987.

[33] W. Feng, K. Adachi, and M. Kowada. A two-queue and two-server modelwith a threshold-based control service policy. European Journal of OperationalResearch, 137:593–611, 2002.

[34] D.P. Gaver, P.A. Jacobs, and G. Latouche. Finite birth-and-death models inrandomly changing environments. Advances in Applied Probability, 16(4):715–731, 1984.

[35] S.B. Gershwin. An efficient decomposition method for the approximate eval-uation of tandem queues with finite storage space and blocking. OperationResearch, 35(2):291–305, 1987.

[36] H. Greenberg. An analysis of traffic flow. Operation Research, 7:79–85, 1959.

[37] B.D. Greenshields. A study of traffic capacity. In Proceedings of the 14thAnnual Meeting of the Highway Research Board, 1935.

[38] B. Hajek. Birth-and-death processes on the integers with phases and generalboundaries. Journal of Applied Probability, 19:488–499, 1982.

[39] M. Haviv. Aggregation/disaggregation methods for computing the station-ary distribution of a markov chain. SIAM Journal on Numerical Analysis,24(4):952–966, 1987.

144 Bibliography

[40] D. Heidemann. Queue length and delay distributions at traffic signals. Trans-portation Research - Part B, 28B(5):377–389, 1994.

[41] D. Heidemann. A queueing theory approach to speed-flow-density relation-ships. In Proceedings of the 13th International Symposium on Transportationand Traffic Theory, 1996.

[42] D. Heidemann. Non-stationary traffic flow from a queueing theory viewpoint.In Proceedings of the 14th International Symposium on Transportation andTraffic Theory, 1999.

[43] D. Heidemann. A queueing theory model of nonstationary traffic flow. Trans-portation Science, 35(4):405–412, 2001.

[44] D. Heidemann. Mathematical analysis of non-stationary queues and waitingtimes in traffic flow with particular consideration of the coordinate transfor-mation technique. In Proceedings of the 15th International Symposium onTransportation and Traffic Theory, 2002.

[45] S. Helber. Analysis of flow lines with Cox-2-distributed processing times andlimited buffer capacity. OR Spectrum, 27:221–242, 2005.

[46] D. Helbing. Traffic and related self-driven many-particle systems. Reviews ofModern Physics, 73, 2001.

[47] S.P. Hoogendoorn and P.H.L. Bovy. State-of-the-art of vehicular traffic mod-eling. J. Syst. Control Eng., 215:283–303, 2001.

[48] A. Hordijk and R. Schassberger. Weak convergence for generalized semi-markov processes. Stochastic Processes and their Applications, 12:271–291,1982.

[49] O.C. Ibe and J. Keilson. Multi-server threshold queues with hysteresis. Per-formance Evaluation, 21:185–213, 1995.

[50] R. Jain and J.M. Smith. Modeling vehicular traffic flow using M/G/c/c statedependent queueing models. Transportation Science, 31(4):324–336, 1997.

[51] H.D. Kafeety, C.D. Meyer, and W.J. Stewart. A general framework for itera-tive aggregation/disaggregation methods. In Proceedings of the Fourth CopperMountain Conference on Iterative Methods, 1992.

[52] M.N. Katehakis and L.C. Smit. A successive lumping procedure for a classof markov chains. Probability in the Engineering and Informational Sciences,26(4):483–508, 2012.

[53] M.N. Katehakis, L.C. Smit, and F. Spieksma. Explicit solutions for a class ofquasi-skip free in one direction processes. 2013.

[54] J.G. Kemeny and J.L. Snell. Finite Markov Chains. Springer, 1960.

Bibliography 145

[55] D.S. Kim and R.L. Smith. An exact aggregation algorithm for a special classof markov chains. Technical report, University of Michigan, 1989.

[56] R.D. Kuhne and M.B. Rodiger. Macroscopic simulation model for freewaytraffic with jams and stop-start waves. In W.D. Kelton B.L. Nelson and G.M.Clark, editors, Proceedings of the 1991 Winter Simulation Conference, pages762–770, 1991.

[57] G. Latouche and V. Ramaswami. A logarithmic reduction algorithm for quasi-birth-death processes. Journal of Applied Probability, 30:650–674, 1993.

[58] G. Latouche and V. Ramaswami. Introduction to Matrix Analytic Methods inStochastic Modelling. ASA-SIAM Series on Statistics and Applied Probability.SIAM, Philadelphia, PA., 1999.

[59] L.M. Le Ny. Probabilit stationnaire d’une file d’attente 2 seuils. Technicalreport, IRISA, 1987.

[60] L.M. Le Ny. Exact analysis of a threshold-based queue with hysteresis and adelayed additional server. Technical report, IRISA, 2000.

[61] L.M. Le Ny and B. Tuffin. A simple analysis of heterogeneous multi-serverthreshold queues with hysteresis. In Proceedings of the Applied Telecommuni-cation Symposium, 2002.

[62] J. Li, D.E. Blumenfeld, N. Huang, and J.M. Alden. Throughput analysis ofproduction systems: Recent advances and future topics. International Journalof Production Research, 47(14):3823–3851, 2009.

[63] J. Li and S.M. Meerkov. Production Systems Engineering. Springer, 2009.

[64] J. Li, S.M. Meerkov, and L. Zhang. Production systems engineering: Problems,solutions, and applications. Annual Reviews in Control, 34:73–88, 2010.

[65] J. Li, S.M. Meerkov, and L. Zhang. Production systems engineering: Mainresults and recommendations for management. International Journal of Pro-duction Research, 51(23-24):7209–7234, 2013.

[66] S. Li and H. Sheng. Generalized folding algorithm for sojourn time analy-sis of finite qbd processes and its queueing applications. Communications inStatistics - Stochastic Models, 12(3):507–522, 1996.

[67] M.J. Lighthill and G.B. Whitham. On kinematic waves. I. Flood movement inlong rivers. Proceedings of the Royal Society of London, Part A, 229(1178):281–316, 1955.

[68] M.J. Lighthill and G.B. Whitham. On kinematic waves. II. A theory of trafficflow on long crowded roads. Proceedings of the Royal Society of London, PartA, 229(1178):317–345, 1955.

146 Bibliography

[69] J.D.C. Little. A proof for the queueing formula: l = λw. Operation Research,9(3):383–387, 1961.

[70] J.C.S. Lui and L. Golubchik. Stochastic complement analysis of multi-serverthreshold queues with hysteresis. Performance Evaluation, 35:185–213, 1999.

[71] M.J. MacNicholas. A simple and pragmatic representation of traffic flow. InSymposium on The Fundamental Diagram: 75 years, pages 161–177. Trans-portation Research Board, 2008.

[72] H.S. Mahmassani, J. Dong, J. Kim, R.B. Chen, and B. Park. Incorporatingweather impacts in traffic estimation and prediction systems. Technical ReportFHWA-JPO-09-065, U.S. Department of Transportation, 2009.

[73] A.D. May. Traffic Flow Fundamentals. Prentice-Hall, Inc., 1990.

[74] P.K. Munjal and L.A. Pipes. Propogation of on-ramp density perturbationson uni-directional and two- and three-lane freeways. Transportation Research,5:241–255, 1971.

[75] K. Nagel. Particle hopping models and traffic flow theory. Physical Review E,53(5):4655–4672, 1996.

[76] M.F. Neuts. Matrix-Geometric Solutions in Stochastic Models - An AlgorithmicApproach. Dover Publications, Inc., New York, 1981.

[77] M.F. Neuts and B.M. Rao. On the design of a finite-capacity queue with phase-type service times and hysteretic control. European Journal of OperationalResearch, 62:221–240, 1992.

[78] G.F. Newell. Nonlinear effects in the dynamics of car following. OperationResearch, 9(2):209–229, 1961.

[79] G.F. Newell. A simplified theory of kinematic waves in highway traffic, PartI: General theory. Transportation Research - Part B, 27(4):281–287, 1993.

[80] G.F. Newell. A simplified theory of kinematic waves in highway traffic, Part II:Queueing at freeway bottlenecks. Transportation Research - Part B, 27(4):289–303, 1993.

[81] G.F. Newell. A simplified theory of kinematic waves in highway traffic, PartIII: Multi-destination flows. Transportation Research - Part B, 27(4):305–313,1993.

[82] D. Ni. Multiscale modeling of traffic flow. Mathematica Aeterna, 1(1):27–54,2011.

[83] D. Ni. A unified perspective on traffic flow theory, Part II: The unified diagram.Applied Mathematical Sciences, 7(40):1947–1963, 2013.

Bibliography 147

[84] H.J. Payne. Models of freeway traffic and control. Mathematical Models ofPublic Systems. Simulation Councils, Inc., 1971.

[85] H.G. Perros and T. Altiok. Approximate analysis of open networks of queueswith blocking: Tandem configurations. IEEE Trans. On Software Engineering,SE-12(3):450–461, 1986.

[86] N. Prameswari and C. Giacomo Prato. DTU Denmark, 2013. Private commu-nication.

[87] I. Prigogine and F.C. Andrews. A Boltzmann-like approach for traffic flow.Operations Research, 8(6):789–797, 1960.

[88] I. Prigogine and R. Herman. Kinetic Theory of Vehicular Traffic. AmericanElsevier, New York, 1971.

[89] H. Rakha and B. Crowther. Comparison of Greenshields, Pipes, and VanAerde car-following and traffic stream models. Transportation Research Board,1802:248–262, 2002.

[90] P.I. Richards. Shock waves on the highway. Operations Research, 4(1):42–51,1956.

[91] D.F. Rogers and R.D. Plante. Estimating equilibrium probabilities for banddiagonal markov chains using aggregation and disaggregation techniques. Com-puters Operations Research, 20(8):857–877, 1993.

[92] Y.W. Shin. Fundamental matrix of transient QBD generator with finite stateand level dependent transitions. Asia-Pacific Journal of Operations Research,26(5):697–714, 2009.

[93] Y. Sugiyama, M. Fukui, M. Kikuchi, K Hasebe, A. Nakayama, K. Nishinari,S. Tadaki, and S. Yukawa. Traffic jams without bottlenecks - experimentalevidence for the physical mechanism of the formation of a jam. New Journalof Physics, 10(3), 2008.

[94] R.T. Underwood. Speed, volume, and density relationship: Quality and theoryof traffic flow. Yale Bureau of Highway Traffic, pages 141–188, 1961.

[95] M. Van Aerde. Single regime speed-flow-density relationship for congested anduncongested highways. In The 74th TRB Annual Conference. TransportationResearch Board, 1995.

[96] M. van Vuuren and I.J.B.F. Adan. Performance analysis of tandem queueswith small buffers. IIE Transactions, 41(10):882–892, 2009.

[97] M. van Vuuren, I.J.B.F. Adan, and S.A.E. Resing-Sassen. Performance analysisof multi-server tandem queues with finite buffers and blocking. OR Spectrum,27:315–338, 2005.

148 Bibliography

[98] T. van Woensel. Models for uninterrupted traffic flows - A queueing approach.Uninterrupted traffic flows, Universiteit Antwerpen, 2003.

[99] T. van Woensel and N. Vandaele. Empirical validation of a queueing approachto uninterrupted traffic flows. 4OR, 4:59–72, 2006.

[100] T. van Woensel and N. Vandaele. Modeling traffic flows with queueing models:A review. Asia-Pacific Journal of Operational Research, 24(4):435–461, 2007.

[101] T. van Woensel, B. Wuyts, and N. Vandaele. Validating state-dependentqueueing models for uninterrupted traffic flows using simulation. 4OR, 4:159–174, 2006.

[102] N. Vandaele, T. van Woensel, and A. Verbruggen. A queueing based trafficflow model. Transportation Research - Part D, 5(2):121–135, 2000.

[103] H. Wang, J. Li, Q.Y. Chen, and D. Ni. Speed-density relationship: From de-terministic to stochastic. In Proceedings of the 88th annual meeting of Trans-portation Research Board, 2009.

[104] H. Wang, J. Li, Q.Y. Chen, and D. Ni. Logistic modeling of the equilib-rium speed-density relationship. Transportation Research - Part A, 45:554–566, 2011.

[105] R.W. Wolff. Stochastic Modelling and the Theory of Queues. Prentice-Hall,Englewood Cliffs, New Jersey, 1989.

[106] J. Ye and S. Li. Folding algorithm: A computational method for finiteQBD processes with level-dependent transitions. IEEE Trans. On Comm.,42(2):625–639, 1994.

[107] S.J. Yuhaski Jr. and J.M. Smith. Modeling circulation systems in buildingsusing state dependent queueing models. Queueing Systems, 4:319–338, 1989.

[108] Y. Zhu, P. Yang, and K. Basu. Overload control in finite buffer message systemwith general distributed service time. In Proceedings Internation Conferenceon Communication Technology, volume 1, pages 408–411, 1996.

Summary

Traffic jams are everywhere, some are caused by constructions or accidents but alarge portion occurs naturally. These “natural” traffic jams are a result of variabledriving speeds combined with a high number of vehicles. To prevent these trafficjams, we must understand traffic in general, and to understand traffic we mustunderstand the relations between the three key parameters of highway traffic, speed,the average speed of a vehicle, flow, the number of vehicles passing a reference point,and density, the number of vehicles on the road, where flow equals the product ofspeed and density. Queueing theory offers new insights in the remaining relationbetween these three parameters.

In this thesis we have developed queueing models that are able to capture modern-day highway traffic behaviour, and we have developed solution methods enabling theanalysis of these queueing models. This thesis is organised in two parts, the first partcovers traffic models based on queueing systems, while the second part discusses thetheory behind the queueing models used.

Chapter 1 provides a general introduction to the thesis. It indicates what wewant to achieve with our traffic models and motivates our choice of queueing mod-els. Furtermore, it explains the general idea behind our queueing models, i.e., thehystertic behaviour of highway traffic. The chapter concludes with an outline of thethesis.

Chapter 2 is the introductory chapter to Part I. The chapter gives a historicaloverview of traffic models used to create the fundamental diagram of highway traffic.It discusses both single-regime traffic models and multi-regime traffic models aris-ing in literature. Furthermore, it gives a literature review on traffic models basedon queueing theory. Two main queueing theoretic approaches can be identified tomodel highway traffic: the queue with waiting room of Heidemann, and the queuewith blocking by Jain and Smith. The queueing models in this thesis are basedHeidemann’s queueing model.

In Chapter 3 we introduce two queueing systems to model the a single highwaysection: the two-stage M/M/1 threshold queue, and the four-stage M/M/1 feedbackthreshold queue. The service rates in the two-stage M/M/1 threshold queue arecontrolled by a threshold policy, based on its queue length. This queueing systemmodel the hysteretic behaviour of traffic on a single highway section. Since thishysteretic behavious is not limited to a single highway section we introduce the four-stage M/M/1 feedback threshold queue. In this queueing system, both the arrivalrates and the service rates are controlled by a threshold policy modelling both thehysteretic behaviour of traffic on a single highway section, as well as its progression

150 Summary

to the upstream highway section. Both queueing models are validated with empiricaldata on highway traffic obtained in Denmark. A sensitivity analysis was performedto investigate the effects of small changes in the parameters on the shape of thefundamental diagram.

In Chapter 4 we model highway traffic on a sequence of highway sections. Tothis end, we extend the single queue models from Chapter 3 to a tandem networkof two-stage M/M/1 threshold queues and a tandem network of four-stage M/M/1feedback threshold queues. In a tandem network of two-stage M/M/1 thresholdqueues, the service rate of each queue is controlled by a threshold policy based on itsqueue length. This tandem network assumes that the hysteretic behaviour of trafficis confined to a single highway section. The tandem of four-stage M/M/1 feedbackthreshold queues also assumes a hysteretic relation between consecutive queues. Inthis tandem network, the service rates are controlled by the threshold policies oftwo consecutive queues. Both queueing models were solved numerically and thefundamental diagram of each individual queue was obtained. Next, a sensitivityanalysis was performed to investigate the effects of small changes in the parameterson the shape of the fundamental diagram.

Chapter 5 is the introductory chapter to Part II. It presents known results fromthe field of Matrix analytic methods on Phase-Type distributions, Markovian ArrivalProcess and Markovian Service Processes, regular Quasi-Birth-and-Death processesand Level Dependent Quasi-Birth-and-Death processes,

Chapter 6 extends the single queue traffic models of Chapter 3 to a more generalqueueing model, the PH/PH/1 multi-threshold queue. In this queueing models,the arrival process and service process are given by a Phase-Type distribution andcontrolled by an arbitrary threshold policy. The PH/PH/1 multi-threshold queueis modelled as a Level Dependent Quasi-Birth-and-Death process and the stationaryqueue length distribution is obtained by decomposing the different stages.

Chapter 7 discusses a system of connected Level Dependent Quasi-Birth-and-Death processes, in which the Markov chain can be divided into subsets, each de-scribing a Level Dependent Quasi-Birth-and-Death process. We provide a successivecensoring algorithm to obtain the stationary distribution of such a system and in-vestigate the possible connections between different subsets.

In Chapter 8 we present an iterative aggregation method which gives an approx-imation of a single queue in a larger tandem network. While focusing on a singlequeue in the network, the aggregation method aggregates all upstream network be-haviour into a Markovian Arrival Process, and all downstream network behaviourinto a Markovian Service Process. This is done in an iterative fashion, aggregatingone queue in each iteration, until all upstream (or downstream) queues are aggre-gated. The resulting queueing model is then analysed using results from the field ofMatrix analytic methods. The iterative aggregation method is compared to simula-tion results of a tandem network of M/M/1/k queues, a tandem network of two-stageM/M/1/k threshold queues, and a tandem network of four-stage M/M/1/k feedbackthreshold queues.

Chapter 9 gives concluding remarks and possibilities for further research.

Samenvatting

Files zijn overal, sommige worden veroorzaakt door wegwerkzaamheden of onge-lukken, maar een groot deel van de files ontstaan zonder enige aanleiding. Deze“natuurlijke” files zijn een resultaat van verschillende rijstijlen op drukke snelwegen.Om deze files te voorkomen moeten we het snelwegverkeer begrijpen, en om snelweg-verkeer te begrijpen moeten we de onderlinge relatie tussen de drie parameters vanverkeer, snelheid, de gemiddelde snelheid van een voertuig, intensiteit, het aantalvoertuigen per uur, en dichtheid, het aantal voertuigen op de weg, begrijpen. Eenelementaire relatie is dat intensiteit het product is van snelheid en dichtheid. Metbehulp van wachtrijtheorie is het mogelijk om inzicht te krijgen in de overgeblevenrelatie tussen deze drie parameters.

In dit proefschrift hebben we wachtrijmodellen ontwikkeld die het gedrag vanhedendaags verkeer kunnen nabootsen. Daarnaast hebben wij oplossingsmethodenontwikkeld die het mogelijk maken deze wachtrijmodellen te analyseren. Dit proef-schrift is onderverdeeld in twee delen, het eerste deel beschrijft de verkeersmodellendie voortkomen uit onze wachtrijmodellen, terwijl het tweede deel de achterliggendetheorie in the analyse van deze wachtrijmodellen behandeld.

Hoofdstuk 1 geeft een algemene introductie tot het proefschrift. Het geeft aanwat we willen bereiken met onze verkeersmodellen en motiveert onze keuze voorwachtlijnmodellen. Bovendien geeft het de achterliggende gedachte van onze wacht-rijmodellen, namelijk het hysteretische gedrag van snelwegverkeer. Het hoofdstukwordt afgesloten met een overzicht van het proefschrift.

Hoofdstuk 2 is het inleidende hoofdstuk van Deel I. Het hoofdstuk geeft een his-torisch overzicht van verkeersmodellen die het fundamenteel diagram van snelweg-verkeer creren. Het bespreekt zowel enkel-regime verkeersmodellen en meervoudig-regime verkeersmodellen uit de literatuur. Bovendien geeft het een literatuurover-zicht van verkeersmodellen gebaseerd op wachtrijtheorie. De twee belangrijke wacht-rijmodellen zijn het wachtrijmodel met wachtruimte van Heidemann, en het wacht-rijmodel met blokkering van Jain en Smith. De wachtrijmodellen in dit proefschriftzijn gebaseerd Heidemann’s wachtrijmodel.

In Hoofdstuk 3 introduceren we twee wachtrij systemen die elk een deel van desnelweg modelleren, namelijk de twee-fase M/M/1 threshold wachtrij, en de vier-faseM/M/1 feedback threshold wachtrij. De bedieningstijden in the twee-fase M/M/1threshold wachtrij worden beınvloed door een drempelwaarde systeem welke geba-seerd is op het aantal klanten in de wachtrij. Wanneer het aantal klanten in dewachtrij een van deze drempelwaardes overschrijdt, wordt de snelheid waarmee klan-ten bedient worden aangepast. Wanneer de wachtrij slinkt, en kleiner wordt dan een

152 Samenvatting

andere drempelwaarde, zal de snelheid weer hersteld worden. Dit wachtrijsysteemmodelleert het hysteretische gedracg van snelwegverkeer op een enkele sectie van desnelweg. Aangezien het hysteretische niet beperkt zal blijven tot een enkele sectie,introduceren we ook de vier-fase M/M/1 feedback threshold wachtrij. In dit wacht-rijsysteem worden zowel de aankomstintensiteiten als de bedieningstijden beınvloeddoor een drempelwaarde systeem. Dit wachtrijsysteem modelleert zowel het hyste-rische gedrag van verkeer op een enkel deel van de snelweg, alsmede de hysteretischerelatie tussen twee opeenvolgende secties. Beide wachtrijmodellen zijn gevalideerdmet empirische gegevens over snelwegverkeer in Denemarken. Een gevoeligheidsana-lyse is uitgevoerd om het effect van kleine veranderingen in de systeem parametersop de vorm van de fundamentele diagram te onderzoeken.

In Hoofdstuk 4 modelleren we snelwegverkeer op meerdere opeenvolgende delenvan de snelweg. Hiervoor hebben we de wachtrijmodellen uit Hoofdstuk 3 uitge-breid naar tandem netwerken van wachtrijen, namelijk een tandem netwerk vantwee-fase M/M/1 threshold wachtrijen, en een tandem netwerk van vier-fase vier-fase M/M/1 feedback threshold wachtrijen. In het tandem netwerk van twee-faseM/M/1 threshold wachtrijen, wordt de bedieningsduur van elke wachtrij beınvloeddoor een drempelwaarde systeem welke gebaseerd is op het aantal mensen in dewachtrij. Dit tandem netwerk neemt aan dat het hysteretische gedrag van het ver-keer is beperkt tot een enkel deel van de snelweg. Het tandem netwerk van vier-faseM/M/1 feedback threshold wachtrijen veronderstelt ook een hysteretische relatietussen opeenvolgende wachtrijen. Beide wachtrijmodellen zijn numeriek opgelost enhet fundamenteel diagram van elke individuele wachtrij is verkregen. Door middelvan een gevoeligheidsanalyse is het effect van kleine veranderingen in de systeemparameters op de vorm van het fundamentele diagram onderzocht.

Hoofdstuk 5 is het inleidende hoofdstuk van Deel II. Het geeft bestaande resul-taten over Phase-Type verdelingen, Markovisch Aankomst Processes en MarkovischBedienings Processes, over regulier Quasi-Geboorte-en-Sterfte processen en over Le-vel Afhankelijke Quasi-Geboorte-en-Sterfte processen.

In Hoofdstuk 6 wordt het verkeersmodel uit Hoofdstuk 3 uitgebreid tot een al-gemener wachtrijmodel, de PH/PH/1 multi-threshold wachtrij. In deze wachtrij-modellen zijn zowel het aankomstprocess als het bedieningsprocess verdeeld volgenseen Phase-Type verdeling en worden zij beıvloed door een arbitraire drempelwaardesysteem. De PH/PH/1 multi-threshold wachtrij wordt gemodelleerd als een LevelAfhankelijke Quasi-Geboorte-en-Sterfte proces en de stationaire verdeling van dewachtrijlengte wordt verkregen door het ontbinden van de verschillende fasen.

Hoofdstuk 7 behandelt een systeem van verbonden Level Afhankelijke Quasi-Geboorte-en-Sterfte processen, waarin de Markov keten kan worden onderverdeeldin subgroepen, die elk een Level Afhankelijke Quasi-Geboorte-en-Sterfte proces be-schrijven. Wij geven een opeenvolgend censurerings algoritme om de stationaireverdeling van de wachtrijlengte van een dergelijk systeem te verkrijgen. Daarnaastonderzoeken we de mogelijke verbindingen tussen de verschillende subgroepen.

In Hoofdstuk 8 presenteren we een iteratieve aggregatie methode waarin een en-kele wachtrij in een tandem netwerk wordt benaderd. De methode focust op een

Samenvatting 153

enkele wachtrij en aggregeert alle voorgaande wachtrijen in een Markovisch Aan-komst Proces en alle opvolgende wachtrijen in een Markovisch Bedienings Proces.Dit wordt gedaan op een iteratieve wijze, waarin een enkele wachtrij per iteratiewordt geaggregeerd. Het resulterende wachtrijmodel wordt vervolgens geanalyseerddoor middel van Matrix analytische methoden. De iteratieve aggregatie methodewordt vergeleken met simulatieresultaten van een tandem netwerk van M/M/1/kwachtrijen, een tandem netwerk twee-fase M/M/1/k threshold wachtrijen en eentandem netwerk van vier-fase M/M/1/k feedback threshold wachtrijen.

Hoofdstuk 9 geeft een slotbeschouwing en mogelijkheden voor verder onderzoek.

About the Author

Niek Baer was born in Lelystad, the Netherlands, on July 29, 1986. In 2004, Niekobtained his VWO diploma at Almere College in Kampen. In 2008, he earned hisBachelor of Science degree in Applied Mathematics from the University of Twente.For his Bachelor thesis, Niek and two other math students developed a mathematicalmodel for the Dutch Film Fund to determine which films would be funded. In2010, Niek obtained his Master of Science degree in Applied Mathematics from theUniversity of Twente. For his master thesis, Niek analysed the patient flows of thepain rehabilitation department at Rehabilitation Centre “Het Roessingh”. Usingtreatment plans, developed by Het Roessingh, Niek determined the bottlenecks inthe patient flows and determined the number of practitioners required to treat allincoming patients.

After obtaining his Master of Science degree, Niek started his Ph.D. studies atthe University of Twente, under supervision of prof. dr. Richard Boucherie anddr. Jan-Kees van Ommeren. In the autumn of 2013, Niek spent three months atthe Technical University of Denmark to work with dr. techn. Bo Friis Nielsen inCopenhagen. His Ph.D. research led to this thesis.

On the 1st of January, 2015, Niek will start as a Consultant at Quintiq in ’sHertogenbosch, the Netherlands.

Documents

Niek Ba er - University of Twente Research Information · Voorwoord Ruim vier jaar geleden begon ik met het onderzoek voor dit proefschrift en er zijn momenten geweest dat ik niet