610

Applications of Markov Chains in Chemical Engineering

Embed Size (px)

Citation preview

Page 1: Applications of Markov Chains in Chemical Engineering
Page 2: Applications of Markov Chains in Chemical Engineering

PREFACE

Markov chains enable one to predict the future state of a system from its

present state ignoring its past history.

Surprisingly, despite the widespread use of Markov chains in many areas of

science and technology such as: Polymers, Biology, Physics, Astronomy,

Astrophysics, Chemistry, Operations Research, Economics, Communications,

Computer Networks etc., their applications in Chemical Engineering has been

relatively meager.

A possible reason for this phenomenon might be that books containing

material on this subject have been written in such a way that the simplicity of

Markov chains has been shadowed by the tedious mathematical derivations. This

caused one to abandon the book, thus loosing a potential tool for handling his

problems.

There are many advantages, detailed in Chapter 1, of using the discrete

Markov-chain model in Chemical Engineering. Probably, the most important

advantage is that physical models can be presented in a unified description via state

vector and a one-step transition probability matrix. Consequently, a process is

demonstrated solely by the probability of a system to occupy a state or not to

occupy it. William Shakespeare profoundly stated this in the following way: " to

be (in a state) or not to be (in a state), that is the question".

I believe that Markov chains have not yet acquired their appropriate status in

the Chemical Engineering textbooks although the method has proven very effective

and simple for solving complex processes. Thus, the major objective of writing

this book has been to try to change this situation. The book has been written in an

easy and understandable form where complex mathematical derivations are

abandoned. The demonstration of the fundamentals of Markov chains in Chapter 2

has been done with examples from the bible, art and real life problems. The

majority of the book contains an extremely wide collection of examples viz..

Page 3: Applications of Markov Chains in Chemical Engineering

VI

reactions, reactors, reactions and reactors as well as combined processes, including

their solution and a graphical presentation of it. All this, to my opinion,

demonstrates the usefulness of applying Markov chains in Chemical Engineering.

Bearing all the above in mind, leads me also to suggest this book as a useful

textbook for a new course entitled Applications of Markov chains in Chemical

Engineering,

Abraham Tamir

Beer Sheva, Israel

May 1, 1998

Page 4: Applications of Markov Chains in Chemical Engineering

ACKNOWLEDGMENTS

A few persons have contributed either directly or indirectly to this book; I

would like to mention them by name.

Professor Arie Dubi, a great teacher and scientist, deserves special thanks.

He was the one who skillfully polished my knowledge in Markov chains to such a

level which made it possible for me to write this book.

Mr. Moshe Golden, a personal friend and a talented progranuner, deserves

many thanks. He assisted me in all technical problems which developed in

producing the book in a camera ready copy form.

Professor T.Z.Fahidy, of Waterloo University, was extremely influential in

the creation of this book, in reviewing part of it; I deeply thank him.

To Ms. Stella Zak, an extremely talented artist, many thanks for helping

design the book cover.

The most significant impact, however, has been that of my graduate students

who participated in my course related to Markov chains. Their proclivity to ask

'why'? has forced me to rethink, recognize and rewrite many parts of the book

again and again. In particular, many thanks are due to my student Adi Wolfson,

who reviewed Chapter 2.

Also thanks are due to Ben Gurion University, which provided generous

assistance and a pleasant atmosphere in which to write this book.

Finally, since I have no co-authors, I must accept responsibility for all errors

in this book.

Abraham Tamir

Page 5: Applications of Markov Chains in Chemical Engineering

Table of Contents

Ch. 0 Biblical Origins and Artistic Demonstrations of Markov Chains - a

Humorous Introduction 1

Ch. 1 Why Write this Book? 6

Ch. 2 Fundamentals of Markov Chains 11

2.1 Markov Chains Discrete in Time and Space 11

2.2 Markov Chains Discrete in Space and Continuous in Time 132

2.3 Markov Chains Continuous in Space and Time 170

2.4 Concluding Remarks 180

2.5 Artistic Ending of the Chapter 180

Ch. 3 Applications of Markov Chains in Chemical Reactions 186

3.1 Modeling the Probabilities in Chemical Reactions 187

3.2 Application and Verification of the Modeling 193

3.3 Major Conclusions and General Guidelines for Applying the

Modeling 204

3.4 Application of Kinetic Models to Artistic Paintings 204

3.5 Introduction to Modeling of Chemical Reactions 210

3.6 Single Step Irreversible Reaction 213

3.7 Single Step Reversible Reactions 219

3.8 Consecutive Irreversible Reactions 228

3.9 Consecutive Reversible Reactions 237

3.10 Parallel Reactions Single and Consecutive Irreversible Reaction

Steps 250

3.11 Parallel Reactions Single and Consecutive Reversible Reaction

Steps 287

3.12 Chain Reactions 301

Page 6: Applications of Markov Chains in Chemical Engineering

3.13 Oscillating Reactions [55-69] 305

3.14 Non-Existing Reactions with a Beautiful Progression Route 323

Ch. 4 Applications of Markov Chains in Chemical Reactors 334

4.1 Modeling The Probabilities in Flow Systems 335

4.2 Application of the Modeling and General Guidelines 349

4.3 Perfectly Mixed Reactor Systems 353

4.4 Plug Flow-Perfectly Mixed Reactor Systems 406

4.5 Impinging-Stream Systems 462

Ch. 5 Applications of Markov Chains in Chemical Processes 498

5.1 Modeling of the Probabilities 498

5.2 Application of the Modeling and General Guidelines 521

Nomenclature 590

References 599

Page 7: Applications of Markov Chains in Chemical Engineering

Chapter 0

BIBLICAL ORIGINS AND ARTISTIC DEMONSTRATIONS OF MARKOV CHAINS

A HUMOROUS INTRODUCTION

The origin of Markov chains, a probabihstic model for predicting the future

given atv^ present of a process and ignoring iXspast, goes back to biblical times, i.e.

to the Book of Books. This we know thanks to what has been said in Exodus 28,

verse 13-14, "Make gold rosettes and two CHAINS of pure gold worked into a

form of ropes, and fix them on the rosettes". A thorough investigation of this verse

led to the conclusion that the word CHAINS is an abbreviation of MARKOV

CHAINS. Thus, it turns out that Markov chains is a very old subject and, as said

in Ecclesiastes 1 verse 9,"... And there is nothing new under the sun".

It is also surprising that available books [2-8, 15-18] related to the subject

matter do not refer at all to biblical Markov processes. Such a process, for

example, can be generated on the basis of Genesis 1 and is related to the order of

the days of the week in the Creation. According to verse 27, man was created on

Friday. The Bible describes this event very nicely as follows: " And God created

man in His image, in the image of God he created him ... And there was evening

and there was morning, the sixth day." Independent of the past history, i.e.

Sunday to Friday, the probability that man will occupy a Saturday on the next day

is 100%. In other words, since the present state is known, namely, Friday, and the

probabihty of moving to the next state is also known, 100%, it is possible to predict

Saturday as the future state of man with respect to the days of the week. The above

example, elaborated later in example 2.11, demonstrates for the first time the

essence of Markov chains proposed by Markov only in 1906 [1], much later than

Biblical times.

Page 8: Applications of Markov Chains in Chemical Engineering

An additional example of a Markov process is related to the states day and

night in Genesis 1 verse 4-5. The creation of these complicated states is described

simply as: " God saw that Ught was good, and God separated the light from the

darkness. God called the light day, and the darkness He called night. And there

was evening and there was morning, a first day." The occurrence of the state night

(or day^ depends only on the previous state unless something unexpected happens

in the universe.

The last Biblical example of a Markov process is concerned with the famous

trial of king Solomon [1 Kings 3]. The story develops as follows (verse 16-22):

"Then came there two women, that were harlots, unto the king, and stood before

him. And the one women said: "Oh, my lord, I and this woman dwell in one

house; and I was delivered of a child with her in the house. And it came to pass the

third day after I was delivered, that this woman was delivered also; and we were

together; there was no stranger with us in the house, save we two in the house.

And this woman's child died in the night; because she overlay it. And she arose at

midnight, and took my son from beside me, while thy handmaid slept, and laid her

dead child in my bosom. And when I arose in the morning to give my child suck,

behold, it was dead; but when I had looked well at it in the morning, behold, it was

not my son, whom I did bear". And the other woman said: "Nay; but the living is

my son, and the dead is thy son". And this said: "No; but the dead is thy son, and

the living is my son. Thus they spoke before the king."

King Solomon was faced with an extremely hard human problem of how to

find out to whom does the living child belong? In order to resolve the problem,

king Solomon made a wise decision described in verse 24-25 as: And the king said:

"Fetch me a sword." And they brought a sword before the king. And the king said:

"Divide the living child in two, and give half to the one, and half to the other." The

above example generates the following Markov process. There are two states here,

namely, that of a living child and that of a divided child. By his brave decision,

king Solomon fixed the probability of moving from the first state to the final one to

be 100%. Consequently, if his verdict would have been materialized, an ultimate

state known in Markov chains as dead state would have been reached. Fortunately,

according to verse 26-27 the woman the child belonged to said, for her heart

yearned upon her son: ... "Oh, my lord, give her the living child, and in no wise

Page 9: Applications of Markov Chains in Chemical Engineering

slay it" while the other woman said: ... "It shall be neither mine nor thine; divide it." Then the king answered and said: "Give her the living child, and in no wise slay it: she is the mother thereof." In this way, the terrible result predicted by the Markov process was avoided.

The following example of ^ifrog in a lily pond was mentioned by Howard in 1960 in the opening of his book [3, p.3] as a graphic example of a Markov process.

Fig.0-1. Escher-Howard Markov process (M.C.Escher "Frog" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

Surprisingly, M.C.Escher, the greatest graphic artist (1898-1972), probably unfamiliar with Markov processes, has already demonstrated in 1931 the same situation in his woodcut Frog [10, p.231]. This is reproduced in Fig.0-1. As time goes by, the frog, system, jumps from one lily pad, state, to another according to

Page 10: Applications of Markov Chains in Chemical Engineering

his whim of the moment. The latter assures a true Markov process since the effect

of the past history is assumed to be negUgible. The state of the system is the

number of the pad currently occupied by the frog; the state transition is, of course,

his leap. If the number of lily pad is finite, then we have a finite-state process.

In 1955,Escher prepared the lithograph Convex and Concave [10, p.308]

which is reproduced in Fig.0-2. It is interpreted below as a Markov process and

elaborated in Chapter 2 in example 2.14.

Fig.0-2. Demonstration of Escher*s Markov process

("Convex and Concave", © 1955 M.C.Escher Foundation ® - Baam, the Netherlands. AH rights

reserved)

It is interesting, first of all, to explore some interesting phenomena in the

lithograph, which is a visual shock. The columns appearing in the picture can be

seen as either concave or convex. On the right-hand side, the solid floor underfoot

Page 11: Applications of Markov Chains in Chemical Engineering

can become the ceiling overhead, and that one may once climb the selfsame

staircase safely, and after some time, while climbing, suddenly be falling down

because the stairs seem upside down. Similarly is the situation with the woman

with the basket walking down the stairs. The upper floor in the middle of the

lithograph, with the flute player, may be seen as convex or concave. Thus, when

it looks concave, the flute player, when climbing out of the window, stands safely

on the vaulting. However, if the appearance looks convex, and the flute player

does not pay attention, he might land far down when leaving the window. In

addition, the element on the floor above the two lizards, may be observed as shell-

shaped ceiling or shell-shaped basin. All above behaviors are phenomena related to

the cognition of vision by the brain.The reader can see in Fig.0-2 six locations,

designated 1 to 6, selected as possible states that a person, system, can occupy. In

principle, the possible occupations depend on the original location of the person,

namely, the initial state vector, and on the probabilities of moving from one state to

the other, i.e. the single-step probability matrix. The above concepts are elaborated

in Chapter 2.1-3. As demonstrated later, a person trying to walk along Escher's

Convex Concave structure, will end up walking up and down along the staircase

connecting states 2 and 3. This result solely depends on Eq.(2-42) which may be

looked upon as the policy-making matrix of the person. The matrix depends on his

mood, but for the sake of simplicity it has been assumed to remain unchanged.

Such a Markov process, i.e. ending walking endlessly between states 2 and 3, is

known as periodic chain. However, more interesting is the fact that the final

situation the person has been trapped in, is independent of the initial state. This is

known as without memory or ergodic process. In conclusion, the aforementioned

examples indicate that the origin of Markov chains goes back to very ancient days

and many wonderful examples can be found in the Book of Books to demonstrate

this process. In addition, some interesting relationships may also arise between the

subject matter and art, which are demonstrated in 2.1-3 of Chapter 2. However,

from Chapter 3 on, applications of Markov chains in Chemical Engineering are

demonstrated.

Page 12: Applications of Markov Chains in Chemical Engineering

Chapter 1

WHY WRITE THIS BOOK?

Markov chains or processes are named after the Russian mathematician

A.A.Markov (1852-1922) who introduced the concept of chain dependence and did

basic pioneering work on this class of processes [1]. A Markov process is a

mathematical probabilistic model that is very useful in the study of complex

systems. The essence of the model is that if the initial state of a system is known,

i.e. its present state, and the probabilities to move forward to other states are also

given, then it is possible to predict the future state of the system ignoring its past

history. In other words, past history is immaterial for predicting the future; this is

the key-element in Markov chains. Distinction is made between Markov processes

discrete in time and space, processes discrete in space and continuous in time and

processes continuous in space and time. This book is mainly concerned with

processes discrete in time and space.

Surprisingly, despite the widespread use of Markov chains in many areas of

science and technology such as: Polymers, Biology, Physics, Astronomy,

Astrophysics, Chemistry, Operations Research, Economics, Communications,

Computer Networks etc., their applications in Chemical Engineering has been

relatively meager.

A possible reason for this phenomenon might be that books containing

material on this subject have been written in such a way that the simplicity of

discrete Markov chains has been shadowed by the tedious mathematical derivations.

This caused one to abandon the book, thus loosing a potential tool for handling his

problems. In a humorous way, this situation might be demonstrated as follows.

Suppose that a Chemical Engineer wishes to study Markov processes and has been

suggested several books on this subject. Since the mathematics is rather complex

Page 13: Applications of Markov Chains in Chemical Engineering

or looks complicated, the probability of moving to the next book is decreasing and

diminishes towards the last books because the Chemical Engineer remembers

always the difficulties he has encountered in studying the previous books. In other

words, his long-range memory of the past has a significant and accumulative effect

on the probability of moving to the next book. However, had he known Markov

chains, he should have made efforts to forget the past or to remember only the

effect of the last book which might be better than the previous ones. In this way,

his chances of becoming famiUar with Markov chains would have been significantly

increased. M.C.Escher demonstrated the above situation very accurately in his

woodcut Still Life and Street [10, p.271], which is reproduced in Fig.1-1 .

Fig.1-1. Abandoned books on Markov chains according to Escher

(M.C.Escher "Still Life and Street" © 1998 Cordon Art B.V. - Baam - Holland. All rights

reserved)

Page 14: Applications of Markov Chains in Chemical Engineering

8

The reader can observe on the right- and left-hand sides of the desk a total of

twelve books on Markov processes among which are, probably, refs.[2-8, 15-18,

84]. Some support to the fact that the books are abandoned is the prominent fact

that the inmiediate continuation of the desk is the street... and that the books are

leaning on the buildings.

There are many advantages of using the discrete Markov-chain model in

Chemical Engineering, as follows.

a) Physical models can be presented in a unified description via state vector

and a one-step transition probability matrix. Consequently, a process is

demonstrated solely by the probability of a system to occupy a state or not to

occupy it. William Shakespeare profoundly stated this in the following way: " to

be (in a state) or not to be (in a state), that is the question". It is shown later

that this presentation coincides with the finite difference equations of the process

obtained from the differential equations. In some cases the process is also of

probabilistic nature, such as a chemical reaction, where the Markov-chain model

presentation seems natural.

b) Markov-chain equations provide a solution to complicated problems. The

increase in the complexity of the problem increases the size of the one-step

transition probability matrix on the one hand, however, it barely increases the

difficulty in solving it, on the other.

c) In some cases, the governing equations of the process are non-linear

differential equations for which an analytical solution is extremely difficult or

impossible. In order to solve the equations, simplifications, e.g. linearization of

expressions and assumptions must be carried out. For these situations the discrete

Markov-chain model may be advantageous.

d) The application of an exact solution is sometimes more complicated in

comparison to Markov-chain finite difference equations. For example, an analytical

solution with one unknown where the equation has no explicit solution, or an

equation with two unknowns where there is no analytical expression of one

unknown versus the other. Both cases may be encountered in problems with

chemical reactions where the solutions involve iterative means.

e) It is extremely easy to obtain all distributions of the state vector versus time

from the Markov-chain solution. However, it is not always easy or convenient to

Page 15: Applications of Markov Chains in Chemical Engineering

9

obtain these distributions from the analytical solution.

f) Elements of consecutive state vectors yield the transient response of the

system, undergoing some process, to a pulse input. Thus, RTD of the fluid

elements or particles are obtained, which gives an insight into the mixing properties

of a single-or multiple- reactor systems. Such a solution, given by Eq.(2-24), is

the product of the state vector by the one-step transition probability matrix.

g) One can model various processes in Chemical Engineering via combination

of flows, recycle streams, plug-flow and perfectly-mixed reactors. The processes

may be also associated with heat and mass transfer as well as chemical reactions.

The author believes that Markov Chains have not yet acquired their

appropriate status in the Chemical Engineering textbooks and applications although

the method has proven very effective and simple for solving complex processes.

Surprisingly, correspondence of the author with eminent Professors in Chemical

Engineering revealed they hardly have heard about Markov chains. Thus, the

major objective of the proposed book is to try to change this situation. Additional

objectives are:

a) Present a comprehensive collection of various applications of Markov

chains in Chemical Engineering, viz., reactions, reactors, reactions and reactors as

well as other processes. This is materiaUzed as from Chapter 3.

b) Provide the university Professor with a textbook for a possible course on

"Applications of Markov chains in Chemical Engineering". Alternatively, the book

can be used as reference book in other courses such as Reactor Design where

examples presented in the present book may be very useful.

c) Provide the practical engineer with numerous models and their solutions

in terms of the state vector and the one-step transition probability matrix, which

might be useful in his work. In addition, to convince the engineer about the

simplicity of applying Markov chains in solving complicated problems.

d) Stimulate application of Markov chains so as to become a common tool in

Chemical Engineering.

e) Last, but not least, to demonstrate the application of Markov chains in art

and biblical problems.

The organization of the book is as follows. The fundamentals of Markov chains

will be presented in Chapter 2 in an easy and understandable form where complex

Page 16: Applications of Markov Chains in Chemical Engineering

10

mathematical derivations are abandoned and numerous examples are presented including their solution. The chapter contains processes discrete in time and space, processes discrete in space and continuous in time and processes continuous in space and time. In Chapter 3, modeling of chemical reactions is presented as well as demonstrations of their transient behavior. In Chapter 4, modeling of chemical reactors is presented and their dynamic behavior with respect to a pulse input. The latter are important parameters for describing the RTD behavior of a system where graphical presentations follow the modeling. Chapter 5 presents modeling of a few processes encountered in Chemical Engineering where effects of heat and mass transfer as well as chemical reaction are also accounted for. A general presentation of the model and its solution by Markov chains is also provided.

Page 17: Applications of Markov Chains in Chemical Engineering

11

Chapter 2

FUNDAMENTALS OF MARKOV CHAINS

Markov chains have extensively been dealt with in refs.[2-8, 15-18, 84],

mainly by mathematicians. Based on the material of these articles and books, a

coherent and a short "distillate" is presented in the following. The detailed

mathematics is avoided and numerous examples are presented, demonstrating the

potential of the Markov-chain method. Distinction has been made with respect to

processes discrete in time and space, processes discrete in space and continuous in

time as well as processes continuous in space and time.

Demonstration of the fundamentals has been performed also on the basis of

examples generated from unusual sources, art and the Bible. Surprisingly, biblical

stories and paintings can be nicely analyzed by applying Markov chains discrete in

time and space.

For each example, a solution was obtained by applying Eq.(2-23) and the

EXCEL software. The solution is presented graphically, which demonstrates the

dynamical behavior of the system in occupying the various states under

consideration. Such an information was missing in the above refs.[2-8, 15-18,

84]. The latter contained only the one-step transition matrix, termed also as policy­

making matrix.

2.1 IMARKOV CHAINS DISCRETE IN TIME AND SPACE 2.1-1 The conditional probability

The conception of conditional probability plays a fundamental role in Markov

chains and is presented firstly.

Page 18: Applications of Markov Chains in Chemical Engineering

12

In observing the outcomes of random experiments, one is often interested how the outcome of one event S^ is influenced by that of a previous event Sj. For example, in one extreme case the relation between Sk and Sj may be such that Sk always occurs if Sj does, while in the other extreme case, Sk never occurs if Sj does. The first extreme may be demonstrated by the amazing Uthograph Waterfall

by Escher [10, p.323] depicted in Fig.2-0.

Fig.2-0. Conditional probability demonstrated by Esther's Waterfall (M.CEscher "V^aterfall" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

Page 19: Applications of Markov Chains in Chemical Engineering

13

If we follow the various parts of the construction in the figure one by one,

we are unable to discover any mistake in it. Yet the combination is impossible as

one may reveal. The basis for this phenomeon is a particular triangle, "the

impossible triangle", 3-4-5 in Fig.2-0 with sum of angles of 270^, first introduced

by Oscar Reutesverd in 1934. Two sides of such a triangle can exist in reality,

however, the overall element can never be constructed but can easily be perceived

by the brain. Therein lies the ingenuity of the artist who can give free play to the

imagination, relying on optical illusions to make the fanciful look real. As

observed, Fig.2-0 is based on three such triangles, i.e., 1-4-5, 3-4-5 and 2-3-4.

Despite this fact, the conception of conditional probability may be clearly

demonstrated in the following way. Assume that Si, S2, ..., S5 are five events

occurring along the trajectory of the moving water, i.e. that water pass through

points 1, 2,..., 5 in Fig.2-0 along their perpetual motion uphill. According to the

above conception, the passing of water at point 2 is completely dependent on their

previous passing at point 1. In other words, given that water pass at point 1,

ensures also their passing at point 2 with 100% probability. Similar are the

relationships between events S3 and S2 and the other consecutive events. Finally,

it should be noted that Fig.2-0 has also been widely used in Chapter 2.1-5 to

demonstrate the application of the equations developed there in classifying the

various states of Markov chains.

To characterize the relation between events Sk and Sj, the conditional

probability o/Sk occurring under the condition that Sj is known to have occurred,

is introduced. This quantity is defined [5, p.25] by the Bayes' rule which reads:

^ prob{SkS;} (2-1)

For the Escher's example, Eq.(2-1) reads, prob{S2 I Si} = prob{S2Si}/prob{Si}.

probjSk I Sj} is the probability of observing an event Sk under the condition that

event Sj has already been observed or occurred; prob{Sj} is the probability of

observing event Sj. SkSj is the intersection of events Sk and Sj, i.e., that both Sk

and Sj may be observed but not simultaneously. prob{SkSj} is the probability for

the intersection of events Sk and Sj or the probability of observing, not

Page 20: Applications of Markov Chains in Chemical Engineering

14

simultaneously, both S^ and Sj. Simultaneous observation of events or,

alternatively, simultaneous occupation of states which is impossible according to

the aforementioned, is nicely demonstrated in Fig.2-1, an oil on canvas painting

[11, p.92]. The Empire of Lights (1955) of Magritte, the greatest surrealist

philosopher.

Fig.2-1. The coexistence of two states, Day and Night, according to

Magritte ("The Empire of Lights", 1954, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

Page 21: Applications of Markov Chains in Chemical Engineering

15

The painting shows a house at night surrounded by trees. The only

bewildering element about this peacefully idyllic scene is the surprising fact that it

has been placed under the light blue clouds of a dayhght sky. This is Magritte's

amazing skill in combining seemingly disparate elements by simultaneously

showing two states that are mutually exclusive in time. Thus, in mathematical

terms, Magritte's painting depicted in Fig.2-1 contradicts the fact that Day and

Night can not coexist, i.e. prob{SkSj} = prob{Day Night} = 0.

Possible explanations for this contradiction by Magritte are the following

ones. The first is that Magritte was not familiar with Markov chains or

probabilistic rules. The second one is based on Genesis 1, which Magritte was,

probably, familiar with. In verse 1-2 is said: "When God began to create the

heaven and earth-the earth being unformed and void...". Thus, recalling the

philosophical character of Magritte, it may be assumed that in The Empire of Lights

Magritte has described the last second before the Creation, i.e. when "...the earth

being unformed..." and Day and Night could live in harmony together. Additional

support is provided by Rashi, the greatest Bible commentator. In Genesis 1 verse

4 it has been said: "God saw that light was good...". According to Rashi: "God

saw that it is good, and that it is not appropriate that darkness and light should be

mixed; he fixed the light for the day and the darkness for the night". In other

words before God's action. Day and Night were mixed and the prob{Day Night}

= 1. Following the above example, Eq.(2-1) gives:

and the question presented by the equation is: what is the probability that Day will

come, knowing that Night has already occurred? Although the answer is trivial,

i.e. prob{Day/Night} = 1, it will be demonstrated on the basis of the above

equation as follows. For a time interval of 24 hours probjDay Night} = 1, i.e. it is

certain that both events will occur within that time interval. In addition,

probfNight} = 1, thus it follows from Eq.(2-la) that, indeed, prob{Day/Night} =

1. On the other hand, because prob{Day Day} = 0 it follows that:

Page 22: Applications of Markov Chains in Chemical Engineering

16

proMDay/Day>=P^<,Xyr'°T°° <^-"'>

Let us now return to Eq.(2-1) for further derivations. If the two events Sk

and Sj are independent of one another, then the probability of observing Sk and Sj

not simultaneously is given by the product of the probabihty of observing Sk and

the probability of observing SJ; that is:

prob{SkSj} = prob{Sk}prob{Sj} (2-2)

It follows then from Eq.(2-1) that prob{Sk I Sj} = prob{Sk}. If prob{SkSj} = 0,

Eq.(2-2) yields that prob{Sk} = 0- In general, the events Si, S2,..., Sz are said to

be mutually independent if:

prob{SiS2 ... Sz} = prob{Si}prob{S2} ... prob{Sz} (2-2a)

The following examples demonstrate the above concepts, a) Assume a single

coin is tossed one time where the two sides of it are designated as event Sk and

event Sj. For a single toss prob{Sk} = prob{Sj} = 1/2. However, prob{SkSj} =

0, since in a single toss either event Sk or event Sj may occur. In other words the

two events are dependent according to prob{Sk} + prob{Sj} = 1. b) Assume now

that the coin is tossed twice. Again prob{Sk} = prob{Sj} = 1/2. However, the

two tosses are independent of each other because in the second toss one can obtain

either Sk or Sj. Therefore, according to Eq.(2-2) one obtains that prob{SkSj} =

1/4. If the number of tosses is three, probjSkSkSk} = prob{SkSjSk} = ... =

(l/2)(l/2)(l/2) = 1/8. c) Let Sk be the event that a card picked from a full deck (52

cards) at random is a spade (13 cards in a deck), and Sj the event that it is a queen

(4 cards in a deck). Considering the above information, we may write that

prob{Sk} = 13/52; prob{Sj} =4/52; probfSkSj} = 1/52

where the last equality designates that there is only one card on which both a spade

and queen are marked. It follows from the above probabilities that Eq.(2-2) is

satisfied, namely, the events Sk and Sj are independent. In other words, if it is

known that a spade is withdrawn from a deck, no information is obtained regarding

to the next withdrawn.

Page 23: Applications of Markov Chains in Chemical Engineering

17

In calculating the probability of observing an event Sk, the conditional

probability is applied in the following way. Suppose Si, S2, S3,... is a. full set of

mutually exclusive events independent of each other. By a full set is meant that it

includes all possible events and that the events Si, S2, S3, ... may always be

observed, but not simultaneously. If Sk is known to be dependent on Si, S2, S3,

..., then we can find probjSk} by using the total probability formula [5, p.27]

where Z is the total number of events; it reads:

z

prob{Sk} = ^ prob{Sj} prob{Sk I Sj} (2-3) j=i

Alternative conceptions to event and observe, more suitable in Chemical

Engineering, are, respectively, state and occupy. Thus, the prob{Sk} designates

the probability of occupying state Sk at step n+1. The prob{Sk I Sj} designates the

probability of occupying state Sk at step n+1 under the condition that state Sj has

been occupied at step n. The prob{Sj} is the probability that state Sj has been

occupied at step n where Z is the total number of states.

The application of Eq.(2-3) has been demonstrated on the basis of the

painting The Lost Jockey [12, p. 18] of Magritte (1898-1967) in Fig.2-2 which was

slightly modified. The surreal element in the picture has been achieved by the trees

which appear like sketched leaves, or nerve tracts. As seen the rider, defined as

system, leaves point O towards the trees designates as states. Eight such states are

observed in the figure. Si, S2,..., S7 and Sk. From each state the system can also

occupy other states, i.e., riding to other directions from each tree. The question is

what is the probability of the rider to arrive at Sk noting that, at first, he must pass

through one of the trees-states Si, S2, ..., S7? The solution is as follows. The

trees Si, S2v., S7 are equiprobable, since, by hypothesis, the rider initially makes

a completely random choice of one of these when leaving O. Therefore:

prob{Sj} = 1/7, j = l , . . . ,7

Once having arrived at Si, the rider can proceed to Sk only by making the proper

choice of one of the five equiprobable roads demonstrated by the five arrows in

Page 24: Applications of Markov Chains in Chemical Engineering

18

Fig.5-2. Hence, the conditional probability of arriving at Sk starting from S\ is

1/5. The latter may be designated by:

Fig.2-2. Application of Eq.(2-3) to Magritte's "Lost Jockey"

("The Lost Jockey", 1948, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

prob{Sk I Si} = 1/5. Similarly, for the other states S],.. . , S7:

prob{Sk I S2} = 1/3, prob{Sk I S3} = 1/4, prob{Sk I S4} = 1/4,

prob{Sk I S5} = 1/3, prob{Sk I S6} = 1/4, prob{Sk I S7} = 1/3

Similarly, for the other states Si,..., S7:

prob{Sk I S2} = 1/3, prob{Sk I S3} = 1/4, prob{Sk I S4} = 1/4,

prob{Sk I S5} = 1/3, prob{Sk I S6} = 1/4, prob{Sk I S7} = 1/3

Thus, it follows from Eq.(2-3) that the probabihty of arriving at point Sk is:

prob{Sk} = (l/7)(l/5 + 1/3 + 1/4 + 1/4 + 1/3 + 1/4 + 1/3) = 34/190 = 27.9%

Page 25: Applications of Markov Chains in Chemical Engineering

19

2.1-2 What are Markov chains? Introduction

A Markov chain is a probabilistic model applying to systems that exhibit a

special type of dependence, that is, where the state of the system on the n+1

observation depends only on the state of the system on the nth observation. In

other words, once this type of a system is in a given state, future changes in the

system depend only on this state and not on the manner the system arrived at this

particular state. This emphasizes the fact that the past history is immaterial and is

completely ignored for predicting the future.

The basic concepts of Markov chains are: system, the state space, i.e., the set

of all possible states a system can occupy and the state transition, namely, the

transfer of the system from one state to the other. Alternative synonyms are event

as well as observation of an event. It should be emphasized that the concepts

system and state Bit of a wide meaning and must be specified for each case under

consideration. This will be elaborated in the numerous examples demonstrated in

the following.

A state must be real, something that can be occupied by the system. Fig.2-3,

probably the most famous of Magritte's pictures, showing a painting in front of a

window, can nicely demonstrate the above. The painting is representing exactly

that portion of the landscape covered by the painting. Assume the tree to be the

state. Thus, the tree in the picture is an unreal state, hiding the tree behind it

outside the room, which is the real state. The latter can be occupied by a system,

for example, the Lost Jockey in Fig.2-2.

Another example of the above concepts is presented by a drunkard, the

system, living in a small town with many bars, the state space. As time goes by,

the system undergoes a transition from one state to another according to the mood

of the system at the moment. The drunkard is also staying in the bar for some time

to drink beer; in other words, the system is occupying the state for some time. If

the system transitions are governed by some probabilistic parameters, then we have

a stochastic process.

Page 26: Applications of Markov Chains in Chemical Engineering

20

Fig.2-3. The real and unreal state of Magritte ("The human condition", 1933, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

Another example is that of particles suspended in a fluid, and moving under the rapid, successive, random impacts of neighboring particles. This physical phenomenon is known as Brownian motion, after the Botanist Robert Brown who first noticed it in 1927. For this case the particle is the system, the position of the particle at a given time is its state, the movement of the particle from one position to the other is its transition and its staying in a certain position is the occupying of the state; all states comprise the state space. The difference between the above

Page 27: Applications of Markov Chains in Chemical Engineering

21

examples is that the first one may be considered as discrete with respect to the states

(bars) and the second one is continuous with respect to the states (position of the

particle in the fluid).

To summarize we may say the following about the applications of Markov

chains. Markov chains provide a solution for the dynamical behavior of a system

in occupying various states it can occupy, i.e., the variation of the probability of the

system versus time (number of steps) in occupying the different states. Thus,

possible applications of Markov chains in Chemical Engineering, where the

transient behavior is of interest, might be in the study of chemical reactions, RTD

of reactors and complex processes employing reactors.

Markov chains aim, mainly, at answering the following questions:

1) What is the unconditional probability that at step n the system is occupying some

state where the first occupation of this state occurred at n = 0? The answer is given

by Eqs.(2"23)-(2-25).

2) What is the probabiUty of going from state j to state k in n steps? The answer is

given by Eqs.(2-30)-(2-32).

3) Is there a steady state behavior for a Markov chain?

4) If a Markov chain terminates when it reaches a state k, defined later as absorbing

state or dead state, then what is the expected mean time to reach k (hence, terminate

the chain) given that the chain has started in some particular state j?

A few more examples

In order to examine the characteristics of Markov chains and the application

of the basic conceptions system, state, occupation of state as well as to elaborate the

idea of the irrelevance of the past history on predicting the future, on the one hand,

and the relevance of the present, on the other, the following examples are

considered.

Example 2.1 is the following irreversible first order consecutive reaction

kj ^2 3 4

Ai -> A2 ~> A3 -»A4--> A5 (2-4)

where a molecule is considered as system . The type Aj of a molecule is regarded

as the state of the system where the reaction from state Ai to state Aj is the

Page 28: Applications of Markov Chains in Chemical Engineering

22

transition between the states. A molecule is occupying state i if it is in state Aj.

The major characteristic of the above reaction is that the transition to the next state

depends solely on the state a molecule occupies, and on the transition probability of

moving to the next one. How the system arrived at the occupied state, i.e. the past

history, is immaterial. For example, state A4 is govemed by the following equation

^ = k3C3-k4C4 (2-5)

where the finite difference equation between steps n and n+1 (time t and t+At)

reads

C4(n+1) = C3(n)[k3At] + C4(n)[l - k4At] (2-6)

The quantities [k3At] and [1 -k4At] may be looked upon, respectively, as the

probabilities to transit from state A3 to state A4 and the probability to remain in state

A4. Eq.(2-6) indicates that the condition of state A4 at step n+1 depends solely on

the conditions of this state prevailing at step n where the past history of the reaction

prior to step n is irrelevant.

Example 2.2 is also a Markov chain. It deals with a pulse input of some

dye introduced into a perfectly-mixed continuous flow reactor. Here the system is

a fluid element containing some of the dye-pulse. The state of the system is the

concentration of the dye-pulse in the reactor, which is a continuous function of

time. The change of system's concentration with time is the state transition given

by

C(tO/Co = exp(-tVt^) (2-7)

CQ is the initial concentration of the pulse in the reactor, C is the concentration at

each instant t' and t ^ is the mean residence time of the fluid inside the reactor. It

may be concluded that once the state of the system is known at some step, the

prediction of the state at the next step is independent of the past history.

Page 29: Applications of Markov Chains in Chemical Engineering

23

Example 2.3 where the outcome of each step is independent of the past

history, is that of tossing repeatedly a fair coin designated as the system. The

possible states the system can occupy are heads or tails. In this case, the

information that the first three tosses were tails on observing a head on the fourth

toss is irrelevant. The probability of the latter is always 1/2, independent of the

past history. Moreover, thtfUture is also independent on the present, and from this

point of view the above chain of tosses is a non-Markov one.

Example 2.4 demonstrates a non-Markov process where the past history

must be taken into account for prediction of the future. We consider the state of

Israel as the system which has undergone many wars during the last fifty years.

This situation is demostrated schematically as follows:

Independence —> Sinai -^ Six day -> Attrition -» Yom kipur -» Lebanon war 1947 war 1956 war 1967 war 1968 war 1973 war 1982

-» The Gulf -> Intifadah uprising war 1991 1987-91/92

It is assumed that the system may occupy the following three states: war, no-war

and peace. In 1995 the system was in a state of war with Lebanon and in a state of

nO'War with Syria. It may be concluded that the prediction of the future state of the

system, if possible at all, depends not only on the present state, but also on the

outcome of preceding wars. In other words such a situation is not without memory

to the past and is affected by it.

Example 2.5 concerns the tossing of a die, the system, numbered 1 to 6,

the states. The probability of obtaining a 6 upward at the 6th toss, conditioned that

previous tosses differed from 6, is 1/6. This is because the probability of obtaining

any number at any toss is 1/6, since the outcome of any toss is independent of the

outcome of a previous toss. Similarly, the probability of obtaining a 6 at the 3rd

toss, conditioned that previous tosses differed from 6, is again 1/6. Thus, the

future is independent neither on the past and on the present, and the tossing chain is

a non-Markov one.

Page 30: Applications of Markov Chains in Chemical Engineering

24

Mathematical formulation

The formulation for the discrete processes may be presented as follows. Let

the possible states that a system can occupy be a finite or countably infinite number.

The states are denoted by Si, S2, S3, S4, S5, S6, ..., Si, ... where S stands for

state. The subscript i designates the number of the state and if we write Si =, it

means that after the equality sign must come a short description about the meaning

of the state.

A discrete random variable X(t) is defined, which describes the states of the

system with respect to time. The quantity t designates generally time where in a

discrete process it designates the number of steps from time zero, t is finite or

countably infinite. X(t) designates the fact that the system has occupied some state

at step t. X(t) can be assigned any of the values corresponding to the states Si, S2,

S3, S4,... However, at a certain occupation (observation) of a state by the system,

only one value can be assigned. When the following equality is applied , i.e.,

X(t) = Si (2-8)

this indicates that state i was occupied by the system on step t, or that the random

variable X(t) has been realized by acquiring the value Si. Thus, Si may be looked

upon as the realization of the random variable.

In example 2.5, the number of states is six where the observations are: Si =

1, S2 = 2, S3 = 3, S4 = 4, S5 = 5, S6 = 6. The figures corresponding to the states

are those appearing on the die. If the die was tossed 5 times, a possible outcome

while considering the upward observation in each toss as a result, may be presented

as:

[X(l), X(2), X(3), X(4), X(5)] = [Si, Si, S3, S2, S6]

Thus, for example, on the fourth toss state S2 was obtained, where 2 was observed

upward. In general, after n observations or steps of the system one has a sample

[X(l), ..., X(n)],

In the above examples, the observations are obtained sequentially. An

important question that can be asked is: Does our knowledge of the past history of

the system affect our statements for the probability of the future events? For

example, does knowledge of the outcome on the first k-1 observations affect our

Page 31: Applications of Markov Chains in Chemical Engineering

25

statements of the probability of observing some particular state, say Si, on the Alh

observation ? In example 2.4 above, the answer was yes, in examples 2.1 and 2.2

the answer was no, whereas examples 2.3 and 2.5 are non-Markov chains, because

neither the past and the present are relevant for predicting the future.

Therefore, the general question may be presented in the following form: What

is the probability of occupying state Si by the system on step k, knowing the

particular states that were occupied at each of the k-1 steps? This probability may

be expressed as:

prob{X(k) = Si I X(l) = Si, X(2) = S2, ..., X(k-l) = Sk-i} (2-9)

where Sn, for n = 1, 2,..., k-1, is a symbol for the state that was observed on the

nth observation or for the state that was occupied on the nth step. As shown later,

the answer to this question will be given by Eqs.(2-23)-(2-26). Eq.(2-9) contains

the conception of conditional probability previously elaborated, designated as

prob[Sk I Sj] which reads ''the probability of observing Sjc given that Sj was

observed". The Escher's Waterfall in Fig.2-0 is a nice demonstration of Eq.(2-9),

i.e.,

prob{X(5) = S5 I X(l) = Si, X(2) = S2, X(3) = S3, X(4) = S4}

In other words the probability of the system, a water element, to occupy S5, i.e.

point 5, is conditioned of previous occupation by the system of Si to S4.

If the outcomes of the observations of the states in a system are independent

of one another, it can be shown [6, p. 15] that the conditional probability in Eq.(2-

9) is equal to the unconditional probability prob{X(k) = Si}; that is:

prob{X(k) = Si I X(l) = Si,..., X(k-l) = Sk-i} = prob{X(k) = Si) (2-10)

Example 2.6 where observations are independent of one another, is that of

tossing repeatedly a fair coin, the system. The possible states are Si = head and S2

= tail. The probability of observing a head on the fourth toss of the coin, given the

information that the first three tosses were tails, is still simply the probability of

observing a head on the fourth toss, that is, 1/2. Thus, Eq.(2-10) becomes for this

example:

Page 32: Applications of Markov Chains in Chemical Engineering

26

prob{X(4) = Si I X(l) = S2, X(2) = S2, X(3) = 82}= prob{X(4) = Si} = 1/2

In general, however, physical systems show dependence, and the state that

occurs on the ^ h observation is conditioned by the particular states through which

the system has passed before reaching the kth state. For a probabilistic system this

fact may be stated mathematically by saying that the probability of being in a

particular state on the kth observation does depend on some or all of the k-1 states

which were observed. This has been demonstrated before in example 2.4.

A Markov chain is a probabilistic model that applies to processes that exhibit

a special type of dependence, that is, where the state of the system on the itth

observation depends only on the state of the system on the (k-l)si observation. In

other words, the processes are conditionally independent of their past provided that

their present values are known. Thus, for a Markov chain:

. prob{X(k) = Si I X(l) = Si, ..., X(k-l) = Sk-i} =

prob{X(k) = Si I X(k-l) = Sk-i} (2-11)

or alternatively:

prob{X(k+n) = Si I X(l) = Si, ..., X(k) = Sk} =

prob{X(k+n) = Si I X(k) = Sk} (2-1 la)

that is, the state of the system on the (k+n)ih observation depends only on the state

of the system on the kxh observation. We, therefore, have a sequence of discrete

random variables X(l), X(2),... having the property that given the value of X(k)

for any time instant k, then for any later time instant k+n the probability distribution

of X(k+n) is completely determined and the values of X(k-l), X(k-2),... at times

earlier than k are irrelevant to its determination. In other words, if the present state

of the system is known, we can determine the probability of any future state

without reference to the past, or on the manner in which the system arrived at this

particular present state. The theory of Markov chains is most highly developed for

homogeneous chains and we shall mostly be concerned with these. A Markov

chain is said to be time-homogeneous or to posses a stationary transition

mechanism when the probability in Eq.(2-1 la) depends on the time interval n but

not on the time k.

Page 33: Applications of Markov Chains in Chemical Engineering

27

Example 2.7 is concerned with the application of the above formulation

for two jars, one red and one black. The red jar contains 10 red balls and 10 black

balls. The black jar contains 3 red balls and 9 black balls. A ball is considered as

system and there are two states viz. Si = red ball and S2 = black ball. The process

begins with the red jar where a ball is drawn, its color is noted, and it is then

replaced. If the ball drawn was red, the second drawn is from the red jar; if the ball

drawn was black, the second draw is from the black jar. This process is repeated

with the jar chosen for a draw determined by the color of the ball on the previous

draw. It is also assumed that when drawing from an urn, each ball in that urn has

the same probability of being drawn. The probability that on the fifth drawing one

obtains a red ball, given that the outcomes of the previous drawings were (black,

black, red, black) = (S2, S2, Si, S2), is simply the probability of a red ball on the

fifth draw, given that the fourth draw produced a black ball and that the first draw

was from the red jar. That is:

prob{X(5) = Si I X(l) = S2, X(2) = S2, X(3) = Si, X(4) = S2} =

prob{X(5) = Si I X(4) = S2 } = 3/12 = 1/4

Note that:

prob{X(5) = Si I X(l) = Si, X(2) = Si, X(3) = Si, X(4) = S2} =

prob{X(5) = Si I X(4) = S2} = 3/12 = 1/4

while:

prob{X(5) = Si I X(l) = Si, X(2) = S2, X(3) = Si, X(4) = Si} =

Prob{X(5) = Si I X(4) = Si} = 10/20 = 1/2

2.1-3 Construction elements of Markov chains The basic elements of Markov-chain theory are: the state space, the one-step

transition probability matrix or the policy-making matrix and the initial state vector

termed also the initial probability function In order to develop in the following a

portion of the theory of Markov chains, some definitions are made and basic

probability concepts are mentioned.

Page 34: Applications of Markov Chains in Chemical Engineering

28

The state space

Definition. The state space 55 of a Markov chain is the set of all states a

system can occupy. It is designated by:

SS =[Si ,S2 ,S3 , ..., Sz] (2-12)

In Si, S designates state where the subscript stands for the number of the state.

The states are exclusive of one another, that is, no two states can occur or be

occupied simultaneously. This point is clearly elaborated in example 2.9 in the

following and its opposite in Fig.2-1 with the associated explanations. Markov

chains are applicable only to systems where the number of states Z is finite or

countably infinite. In the latter case, an infinite number of states can be arranged in

a simple sequence Si, S2, S3, .... For the preceding example 2.1, the state space

is SS = [Si, S2, S3, S4, S5] = [Ai, A2, A3, A4, A5]. For example 2.3, SS = [Si,

S2] = [heads, tails]. For example 2.5, SS = [Si, S2, S3, S4, S5, S6] = [1, 2, 3, 4,

5, 6] and for example 2.7 in the following, SS = [Si, S2] = [red ball, black ball].

Some properties of the state space are [6, p. 18]:

1 > prob{Si} > 0 i = 1, 2,..., Z (2-12a)

where prob{Si} reads the probability of occupying state Sj. An alternative

expression for Eq.(2-12a) for all Sj in the state space, in terms of the conditional

probability defined in Eq.(2-1), reads:

1 > prob{Sj I Sj} > 0 (2-12b)

Note that Si must be occupied before occupying each of the others. It should be

noted that Eqs.(2-12a) and (2-12b) satisfy:

probJ ^ S i l = l or probJ ^ Sj I sA = 1 (2-12c) I alii in SS J [ a l l j i n S S J

An additional property of the state space is:

Page 35: Applications of Markov Chains in Chemical Engineering

29

prob" i=l

X i = S P ^^ i ^ P H S J ' i = S P ^ ^J ' i ^"^^ L j = l j=l

The summation on the left-hand side designates a state (or event) comprised of Z

fundamental states. The prob{sunmiation} means the probability of occupying at

least one of the states [Si, S2, S3,..., Sz].

Definition. A Markov chain is said to be a finite Markov chain if the state

space is finite.

The one-step transition probability matrix

Deflnition. The one-step transition probability function pjk for a Markov

chain is a function that gives the probability of going from state j to state k in one

step (one time interval) for each j and k. It will be denoted by:

Pjk = prob{Sk I Sj} = prob{k I j} for all j and k (2-13)

Note that the concept of conditional probability is imbedded in the definition of pjk-

Considering Eq.(2-1 la), we may write also:

Pjk = prob{X(m+n) = Si I X(m) = Sj} (2-13a)

pjk is time-homogeneous or stationary transition probability function if it satisfies:

Pjk = function(time interval between j and k) (2-14)

Considering Eq.(2-13a), Eq.(2-14) is expressed by:

Pjk = function(n)

Pjk 5 function(m) as well as pjk 9 function(m+n) (2-14a)

Thus, for a time-homogeneous chain, the probability of a transition in a unit time or

in a single step from one given state to another, depends only on the two states and

not on the time.

In general, the one-step transition probability function is given by:

Page 36: Applications of Markov Chains in Chemical Engineering

30

Pjk(n, n+1) = prob{X(n+l) = Sk I X(n) = Sj} (2-15)

which gives the probabiUty of occupying state k at time n+1 given that the system

occupied state j at time n. This function is time-dependent, while the function given

by Eq.(2-13) is independent of time, or homogeneous in time. Since the system

must move to some state from any state j , Eq.(2-18) below is satisfied for all j

The one-step transition probabilities can be arranged in a matrix form as

follows:

P = (Pjk) =

Pll P12

P21 P22

PlZ

P2Z

PZl PZ2 Pzz

(2-16)

where pjk denotes the probability of a transition from state] (row suffix) to state k

(column suffix) in one step. The matrix is time-homogeneous or stationary if the

pjk's satisfy Eq.(2-14a). A finite Markov chain is one whose state space consists

of a finite number of states, i.e., the matrix P will be a ZxZ square matrix. In

general the state space may be finite or countably infinite. If the state space is

countably infinite then the matrix P has an infinite number of columns and rows.

Definition. The square matrix P is a stochastic matrix if it satisfies the

following conditions:

1) Its elements obay:

0 < p j k < l (2-17)

otherwise the transition matrix loses meaning.

2) For every row j :

z

SPjk = 1 (2-18)

k=l

Page 37: Applications of Markov Chains in Chemical Engineering

31

where Z is the number of states which can be finite or countable infinite. However, z

one may notice that /]Pjk ^ 1, a fact that apparently violates the standard theory of

Markov chains, and encountered, for example, in non-linear chemical reactions.

Other characteristics of the square matrix P are:

3) The elements pjj on the diagonal designate probabilities of remaining in same

state j .

4) The elements pjk above the diagonal designate probabilities of entering state k by

the system, from state j .

5) The elements pjk under the diagonal designate probabilities of leaving state j by

the system, to state k.

6) The sum of the products of the elements pjk (over a certain column k) by the

elements of the state vector (defined below) has the significance of the conditional

probability defined by Eq.(2-3) and is also identical with Eq.(2-23). The latter

equation is of utmost significance, giving the probability of occupying a certain

state at step (n+1) knowing that this state was influenced by other states at step n.

The transition matrix P is, thus, a complete description of the Markov

process. Any homogeneous Markov chain has a stochastic matrix of transition

probabilities and any stochastic matrix defines a homogeneous Markov chain.

In the non-homogeneous case the transition probability:

prob{X(r) = k I X(n) = j} (r > n)

will depend on both n and r. In this case we write:

Pjk(n,r) = prob{X(r) = k I X(n) = j} (2-19)

In particular the one-step transition probabilities pjk(n,n+l) will depend on the time

n and we will have a sequence of the following stochastic matrices corresponding

to Eq.(2-16) for n = 0, 1, 2, 3,...:

Page 38: Applications of Markov Chains in Chemical Engineering

32

P(n) = (pjk(n,n+l)) =

pll(n,n+l) pi2(n,n+l)

P2i(n,n+1) p22(n,n+l)

(2-20)

The initial and the n-step state vectors

Definition. The initial state vector, termed also as the initial probability

function, is a function that gives the probabihty that the system is initially (at time

zero or n = 0) in state i, for each i. The initial state vector will be denoted by:

Si(0) = prob{X(0) = Si} i = 1, 2,..., Z (2-21)

designating the probability of the system to occupy state i at time zero. The above

quantities may be arrayed in a row vector form of the initial state vector, i.e.:

S(0) = [si(0), S2(0), S3(0), ..., Sz(0)] (2-22)

designating the initial occupation probability of the states [Si, S2, S3, ..., Sz] by

the system. Similarly:

S(n) = [si(n), S2(n), S3(n), ..., Sz(n)] (2-22a)

is the state vector of the system at time n (step n). Si(n) is the occupation

probability by the system of state i at time n, where:

Si(n) = prob{X(n) = Si} i = 1, 2,..., Z (2-21a)

The relationship between the one-step probability matrix and the

state vector

This relationship takes advantage of the definition of the product of a row

vector by a matrix [7, p. 19]. The product of S(n) defined in Eq.(2-22a), by the

square matrix P defined in Eq.(2-16), yields the new row vector S(n+1). The

Sk(n+1) component of this vector, i.e., the (unconditional) probability of occupying

Sk at the (n + 1) step, reads:

Page 39: Applications of Markov Chains in Chemical Engineering

33

Sk(n+1) =X^j(n)pjk (2-23) j=i

It should be noted [15, p.384] that intuitively it is felt that the influence of the initial

state Sj(0) should gradually wear off so that for large n the distribution in Eq.(2-23)

should be nearly independent of the initial state vector S(0), i.e., the state

occupation is without memory to its initial history.

Let us now consider the significance of Eq.(2-23) with respect to Eq.(2-3)

given below, i.e.:

prob{Sk} = ^ prob{Sj} prob{Sk I Sj} j=i

where prob{Sj} is the probability that state Sj has been occupied at step n. Noting

Eq.(2-21a), it may be concluded that prob{Sj} is identical to Sj(n) in Eq.(2-23),

where Sj(n) is the occupation probability by the system of state j at time n. prob{Sk

I Sj} is the probability of occupying state Sk at step n+1 under the condition that

state Sj has been occupied at step n. Thus, it is identical to pjk in Eq.(2-23) which

designates the one-step transition probability from state j to k. Finally, probjSk} ^

Eq.(2-3) designates the probability of occupying state Sk at step n+l, which is

identiced to Sk(n+1). Eq.(2-23), which is a recurrence relation, may be expressed

in matrix notation as:

S(n+1) = S(n)P (2-24)

and on iteration we obtain:

S(n+1) = S(0)P'^+l (n = 0, 1, 2,. . .) (2-25)

where P and S(0) are given by Eqs.(2-16) and (2-22). Alternatively, if S(n)

denotes a column vector, then:

S(n+1) = P'^+1S(0) (2-25a)

since the choise of S(n) as a row vector is arbitrary.

Page 40: Applications of Markov Chains in Chemical Engineering

34

Eqs.(2-23)-(2-25) are the fundamental expressions of Markov chains because

of the following reasons:

a) The equations give an answer to question 1 (in the introduction of section

2.1-2), i.e., what is the unconditional probability that at time n+1 (n+1 steps after

the first occupation) the system occupies state kl As can be seen, a Markov chain

is completely described when the state space, initial state vector and the one-step

transition probability matrix are given. For a physical system that is to be

represented by a Markov chain, this means that first, the set of possible states of the

system, 55, must be determined or defined. Second, the initial (at time zero)

probabilities of occupying each of these states, Si(0), must be calculated or

estimated. Finally, the probability of going from state j to state k in one time

interval (one step), pjk, must be determined or estimated for all possible j and k .

Thus, the probabilities of future state of a system, namely, at step n+1, can be

predicted from its present state at step n, and the transition probabilities in one step;

the past has no influence at all in the predictions.

b) On tha basis of Chapters 3 and 4, it may be concluded that Eqs.(2-23)-(2-

25), or the state vector S(n) given by Eq.(2-22a) and the matrix P yielding the

above equations, is an elegant way of writing the Euler integration algorithm for the

differential equations which describe the mechanism of the process.

The discrete Chapman-Kolmogorov equation

In deriving this equation, the following question is considered: W/zar is the

probability of transition of a system from state Sj to state Sjc in exactly n steps i.e.:

Pjk(n) = prob{X(n+t) = Sk I X(t) = Sj} (2-26)

In other words, Pjk(n), the n-step transition probability function, is the conditional

probability of occupying Sk at the nth step, given that the system initially occupied

Sj. pjk(n), termed also higher transition probability, extends the one-step transition

probability pjk(l) = Pjk and gives an answer to question 2 in 2.1-2. Note also that

the function given by Eq.(2-26) is independent of t, since we are concerned in

homogeneous transition probabilities.

In answering the above question, we refer again to The Lost Jockey depicted

in Fig.2-2 defined as system. We designate now point O, the initial state of the

Page 41: Applications of Markov Chains in Chemical Engineering

35

system, by Sj. The other states are the trees Si to Sz where the final state is S^. It

may be seen that the system can move from state Sj to Sk by a number of different

paths. For example, if a system has Z possible states, then in two steps it may go

fromSj to Skby:

Sj -^ Si -^ \

Sj ^ S 2 ^ S k

(2-27) S j - > S i - > S k ^ ^

Sj - > S z ^ S k

where Z = 7 in Fig.2-2, excluding Sj and Sk. In order to compute the probability

of the transition S; -^ Sj -> Sj , assuming the states are independent of one

another, one applies the concept of independence given by Eqs.(2-2) and (2-2a). Thus, for a Markov chain, the transition S: -^ Sj in one step is independent of the

transition Sj -^ Sj , yielding:

prob{Sj -> Si and Sj -> S^} = prob{Sj ^ Sj}prob{Si -> Sj } = pjiPi (2-28)

Noting that prob{Sj ^ Sj ^ S^} = prob{Sj -4 Sj and Sj -^ S^}, we may now

have expressions for computing the probabilities of the transitions to the states

listed in Eq.(2-27). Since the transitions to and from the Z states in Eq.(2-27) are

mutually exclusive (that is, no pair of them can be occupied simultaneously), the

probability of the transition from state j to state k in two steps, i.e. pjk(2), is equal

to the sum of the probabilities over the Z different paths; it is given by:

Pjk(2) = Pjipik + Pj2P2k + ... + PjzPzk (2-29)

It should be noted that the above result follows also from the concept of conditional

probability given by Eq.(2-3).

Assume now that the Jockey becomes tired and wants to rest along the paths i to k. Thus, the latter trajectories are performed in the two steps Sj -^ Sj ~> \ ,

i.e., he is resting at state Si for one time interval and then riding towards the state

Page 42: Applications of Markov Chains in Chemical Engineering

36

Sk. The corresponding probability for this step is pik(2), i = 1, 2,..., Z, where the

total probability for Si to Sk, i.e., Pik(3), is given recursively by:

Pik(3) = PilPlk(2) + Pi2P2k(2) + ... + PizPzk(2) (2-29a)

The general case is where the Jockey is moving from Sj to Si in n steps and from Si

to state Sk in m steps. Based on the above considerations, one can show that

Pjk(n + m) = 2 ^ pji(n)pik(m) (2-30) 1=1

which is the discrete Chapman-Kolmogorov equation. In order to have Eq.(2-30)

true for all n > 0 we define pjk(O) by pjj(O) = 1 and pjk(O) = 0 for j 9 k as is natural

[15, p.383]. pji(n) and pik(ni) are the n and m-step transition probabilities,

respectively. The latter quantities are arrayed in matrix form denoted by P(n) in the

same way as pjk form the matrix P in Eq.(2-16), i.e.:

P(n) = (pjk(n)) =

Pll(n) pi2(n) Pi3(n) ... piz(n)

P2l(n) P22(n) P23(n) ... P2z(n)

P3l(n) P32(n) P33(n) P3z(n)

Pzl(n) Pz2(n) Pz3(n) ... Pzz(n)

(2-31)

The calculation of the components Pjk(n) is as follows. In general:

P(n) = Pn (2-31a)

where P' is the one-step transition probability matrix multiplied by itself n times.

In a matrix form notation, we may write Eq.(2-30) as:

Page 43: Applications of Markov Chains in Chemical Engineering

P(n + m) = P(n)P(m) = P(m)P(n)

37

(2-32)

where also:

P(n+ 1) =P(n)P = PP(n) (2-32a)

The above equations require the multiplication of a matrix by a matrix yielding a

new matrix. According to [7, p. 19], we define the product:

Q = (qjk) = AB

to have the components:

z

r=l

j , k = l , 2 , . . . , Z

where A and B are both a ZxZ square matrices given by:

(2-33)

(2-33a)

A = (ajk) =

an ai2

a2l a22

azi a22

aiz

a2z

azz

B = (bjk) =

bii bi2

b21 b22

bzi bzi

biz

b2z

bzz

(2-34)

Eq.(2-33a) states that to obtain the component qjk of Q in Eq.(2-33), we have to

multiply the elements of theyth row of A by the corresponding components of the

kxh column of B and add all products. This operation is called row-into-column

multiplication of the matrices A and B.

As an example, we consider the case 81-^82 corresponding to the following

2x2 matrix:

Page 44: Applications of Markov Chains in Chemical Engineering

38

P = Si

S2

Si

Pll

1 P21

S2

P12

P22

(2-35)

From Eq.(2-33), noting that A = B = P where Z = 2, follows from P2 that:

Pll(2) = PIipi 1 + P12P21 for the paths

Si - » S i - ^ S , s,-^s. Pl2(2) = P11P12 + P12P22 for the paths

S| —> Sj —> S2 Sj — S2 — S2

where Sj-^Sj and 82-^82 indicate "resting" steps at states Si and S2.

P2l(2) = P21P11 + P22P21 for the paths

0 2 —^ >Ji —^ ^\ ^2 —^ * 2 —^ ^\

P22(2) = P21P12 + P22P22 for the paths

0 2 —^ »J| —^ > 2 ^ 2 —^ 2 —^ 2

Similarly, from P^ = PP^ it is obtained that:

Pll(3) = p i ip i ip i i +P11P12P21 +P11P12P21 +P12P22P21 for the paths

Si -^Si-^Si-^Si Si->Si->S2->Si

S i -^S i^S2-^S i Si-^S2->S2-^Si

Pl2(3) = PI ipi 1P12 + Pi 1P12P22 + P12P21P12 + P12P22P22 for the paths

Si-^Si-»Si-^S2 Si ->Si^S2-^S2

vS|—^^2—^^\—^^^2 1—^ 2—^ 2— * 2

P2l(3) = P21P11P11 +P22P21P11 +P21P12P21 +P22P22P21 for the paths

S2->Si-»Si->Si S2-^S2-^Si-^S|

02—^^\—^^2—^ 1 2— * 2—^^ 2—^ 1

P22(3) = P21P11P12 + P22P21P12 + P21pl2p22 + P22P22P22 for the paths

Page 45: Applications of Markov Chains in Chemical Engineering

39

02—^^i—^^\—^^^2 2—^*^2—^^1—^^^2

09—^*^l—^*^2—^*^2 2— " 2— * 2—^ 2

2.1-4 Examples In the following examples, the application of Eqs.(2-23)-(2-25) is

demonstrated to describe the dynamics of the occupation of the states by the

system. The basic conceptions system and state are defined in each example

selected from the Bible, art as well as real life problem.

Biblical examples Example 2.8 combines bible and art and refers to the oil on canvas painting

Still Life with Open Bible (1885) by van Gogh [13, p. 15], one of the greatest

expressionists. It should also be noted that the Bible was a symbol of van

Gogh's parents' home. To demonstrate Markov chains on this painting, the

original painting was slightly modified by placing two identical books on the Bible

located to the left-hand side of the candles as depicted in Fig.2-4. As a matter of

fact, the two books are copies of the original one on the right-hand side to the

Bible.

Page 46: Applications of Markov Chains in Chemical Engineering

40

Fig.2-4. The modified *StiIl Life with Open Bible*

("still Life with Open Bible", 1885, V. van Gogh )

The following Markov chains is generated. It is clear that if one wants to

study the Bible, the Bible has to rest on the top of the pile. For three books,

designated as system, which are placed one on the top of the other, three states are

possible according to the order of the books: Si = Bible, book, book; S2 = book,

Bible, book; S3 = book, book, Bible where the book on the left-hand side is placed

on the top of the pile. A one-step transition from one state to the other is conducted

by taking a book from the bottom of the pile and placing it on the top of it. Thus,

the three states may be expressed by a 3x3 one-step transition matrix given by:

P =

Si

S2

S3

Si

1

0

1

S2 0

0

0

S3 0

1

0

(2-36)

Page 47: Applications of Markov Chains in Chemical Engineering

41

The matrix was applied for the following cases:

1) Assume the system is initially at state S]. In this case pn = 1, namely,

the state is a trapping or a dead state which is impossible to escape from. Thus,

one can study safely the Bible because the Book of books is always on the top of

the other books.

2) The system is initially at state S2, i.e., S2(0) = 1 where si(0) = S3(0) = 0.

The question is how many steps are needed to move to Si where the number of

States Z = 3? The initial state vector reads:

S(0) = [ 0, 1, 0 ]

Applying Eqs.(2-23), (2-24), i.e., multiplying the state vector by the matrix (2-

36), yields:

si(n), S2(n), S3(n)

S(1) = [0, 0, 1 ]

S(2) = [ l , 0, 0 ]

S(3) = [ l , 0, 0 ]

S(4) = [ l , 0, 0 ]

The above calculations indicate that at the second step, as expected, the system will

be at Si. Once it reaches this state, it will remain there forever.

3) If the system is initially at S3, the initial state vector reads:

S(0) = [0, 0, 1 ]

Similar calculations yield:

si(n), S2(n), S3(n)

S(l) = [ l , 0, 0 ]

S(2) = [ l , 0, 0 ]

S(3) = [ l , 0, 0 ]

i.e. a steady state at Si will be reached after one step. It should be noted that Si is

always reached independent of S(0), namely, this state is without memory to the

initial state.

Page 48: Applications of Markov Chains in Chemical Engineering

42

Example 2.9 is related to the creation of the two states already mentioned in

Chapter 0, vis., Si = Day and S2 = Night in Genesis 1 verse 4-5. The system

may be a man or a population who may occupy the above states. The two states are

expressed by the following 2x2 matrix corresponding to 24 hours:

P =

Si

S2

Si

0

1

S2

1

0

(2-37)

The significance of the above probabilities is the following. The probability of

remaining in Si is zero because after day comes always night, i.e., p n = 0. The

transition probability from Si to S2 is, of course, pi2 = 1. Similarly, p2i = 1, i.e.,

the transition probability from S2 to Si, as well as probability of remaining in S2,

P22 = 0. It should be noted that above probabilities are independent of time. If the

initial state vector reads:

S(0) = [ 1, 0 ]

application of Eqs.(2-23), (2-24) yields:

si(n), S2(n)

S(l) = [ 0, 1 ]

S(2) = [ 1, 0 ]

S(3) = [ 0, 1 ]

S(4) = [ 1, 0 ]

The behavior of the vector si(n) indicates that if initially there was Si, the

probability of remaining at this state after one step is si(l) = 0. After two steps (24

hours), si(2) = 1, i.e., there will be again a state of Day, as expected. According

to section 2.1-5 Table 2-2, the above chain is defined as a periodic chain.

Page 49: Applications of Markov Chains in Chemical Engineering

Si

0

0

S2

1

1

43

Example 2.10 is a Markov chain representation of king Solomon's famous

trial [1 Kings 3] discussed in Chapter 0. The child was selected here as system.

The two states are: Si = Living child and S2 = Divided child. These states are

presented by the following 2x2 matrix where the probability of the system to

remain in state Si is, unfortunately, pn =0 . This is because king Solomon said:

Divide the living child in two. The transition probability from Si to S2 is,

therefore, 100%, i.e., pi2 = 1. Similarly, p2i = 0, which is the transition

probability from S2 to Si. Finally, the probability of remaining in state S2, P22 = 1

and the matrix reads:

Si

P = I I (2-38)

S2

It is interesting to note that the matrix is characterized by the so-called dead state,

i.e., once the system reaches this state, it will remain there for ever. The

application of Eq.(2-24), assuming that initially si(0) = 1, i.e., the system is at Si

= Living child and that:

S(0) = [ 1, 0 ]

yields:

si(n) S2(n)

S(l) = [ 0, 1 ]

S(2) = [ 0, 1 ]

The behavior of the vector si(n) indicates that if initially there was a Living child,

the probability of remaining at this state after one step is, unfortunately, si(l) = 0.

Thus, the Markovian dead state will be also a real description of the actual situation.

Example 2.11 considers the order of the days of the week in the Creation

mentioned in Chapter 0. The states are defined as Si = Sunday, S2 = Monday,...,

and S7 = Saturday. The probability matrix reads:

Page 50: Applications of Markov Chains in Chemical Engineering

44

P =

Si S2 S3 S4 S5 S6 S7

Si

S2

S3

S4

S5

S6

S7

0

0

0

0

0

0

1

1

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

1

0

(2-39)

The system may be a man or the universe, which may occupy the states at some

time according to matrix (2-39).

Assuming that:

S(0) = [ 1, 0, 0, 0, 0, 0, 0 ]

i.e., that the system initially occupying Si, yields the following behavior:

si(n) S2(n) S3(n) S4(n) S5(n) S6(n) S7(n)

S(1) = [ 0 , 1, 0, 0, 0, 0, 0 ]

(2-39a)

S(2) = [ 0,

S(3) = [ 0,

S(4) = [ 0,

S(5) = [ 0,

S(6)= [ 0,

S(7) = [ 1,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0 ]

0 ]

0 ]

0 ]

1 ]

0 ]

The above behavior reveals that all states are periodic of a period of seven days.

Example 2.12 generates A Markov chain based on the biblical story about

the Division of the Promised Land among the twelve tribes. In Book of Books

[Joshua 13, verses 1 and 7] we read: "Now Joshua was old and advanced in years;

and the Lord said to him: You are old and advanced in years, and there remains yet

very much land to be possessed... Now therefore divide this land..." In [25,

p.52], a map depicted below, shows the results of this division, i.e. the boundaries

of the inheritances of the tribes in the 12th century BC. corresponding to the above

verses.

Page 51: Applications of Markov Chains in Chemical Engineering

45

The Big Sea / Sj-Asher

1 Acre L

f\ 1 ( 1 / ^^S3=Zebulun ^

1 1 S = Issachar

/ S Jaffa i

1 I S ^ Ephraim /S^= Dan

1 / S = Benjamin

/

1 /Oaza S=Judah

1/ 1 S.y Simeon

8^= Naphtali 1

^ 1 Kinereth 1

. j Sea 1

.=/Manasseh 1 > 1 1

ISg=Gad

1 Jordan 1 V River 1

•*SS = Reuben 1

Deadl SeaJ

A visitor, designated as system, wishes to visit the tribes. His transition

between the states assumes that the probabilities of remaining in a state or moving

to the other states is the same and that the following one-step transition matrix

holds:

Page 52: Applications of Markov Chains in Chemical Engineering

46

P =

Si

S2

S3

S4

S5

S6

S7

Sg

S9

siol S l l

Si S2 S3 S4 S5 S6 S7 Sg S9 Sio S u S12

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1/4 0 0 0 0

1/3 1/3 1/3

1/3 1/3 1/3

1/4 1/4 1/4 0 0 0

0

0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

sd 0 0

0

0

0

6 0

0

0

0

0

1/3 1/3 1/3 0 0 0 0 0 0

1/3 1/3 1/3 0 0 0 0 0

0 0 0 1/4 1/4 1/4 1/4 0 0

0 0 0 1/4 1/4 0 1/4 1/4 0

0 0 1/5 1/5 0 1/5 1/5 0 1/5

0 0 0 1/6 1/6 1/6 1/6 1/6 1/6

0 0 0 0 0 1/4 1/4 1/4 1/4

0 0 0 0 1/4 1/4 1/4 1/4 0

0 0 0 0 0 0 1 / 2 0 1 / 2

(2-40)

The application of Eq.(2-24) yields Fig.2-5(a,b) showing the probability

distribution of visiting the twelve tribes, i.e., Si(n), against the number of steps n.

A Step is equivalent to some time interval the visitor stays in a tribe after which he is

leaving to the next one. Two cases were considered: a) The visitor begins at state

Si = Asher, where the initial state vector S(0) is given at the top of Fig.2-5(a). b)

The visitor begins at state S12 = Simeon, where the initial state vector S(0) is given

atthetopofFig.2-5(b).

Page 53: Applications of Markov Chains in Chemical Engineering

47

0.8

(0~

1—I—I—I— \—I—I— \—I—I—I 1—I—T"

S(0) = [1,0,0,0,0,0,0.0,0,0,0,0]

I I I

(a)

s^(n), Sg(n)

.s (n)

0.8 h

0.6 U

0.4

0.2

1 — I — I — \ — I — I — I — I — \ — I — I — I — I — I — I — I — I — I — r

S(0) = [0,0,0,0.0,0.0,0,0.0,0,1]

-s (n)

3^(n) to s^(n)

(b)H

\ y^o(") Sii(");V") ^ " ^f") %(")

f]/ \^\ I I I I I I I I 1 1 I I I \ i* I 10 n

15 20

Fig.2-5(a, b). Probability of visiting the twelve tribes

As observed, in both cases the visiting probabihty distribution of the tribes,

which may be used to decide the policy of visiting of the tribes, reaches a steady

state after several steps. In other words, he will start at the tribe of the highest

probability and then move to the tribe of a lower probability, and so on. According

Page 54: Applications of Markov Chains in Chemical Engineering

48

to the results, in case (a) he starts at Si and moves according to the following order

of states where he reaches the last tribe after 12 steps:

Si -^82^83-^84-^85 ->86->87->89->88-^89->8io->8ii-^8i2

It should be noted that the results indicate that S2(l) = S3(l) = 0.333, S5(6) = S6(6)

-= 0.044. Bearing this in mind this fact, the visitor has decided to move according

to 82->S3 and 85 ->86. In case (b), he starts at 812 and his transition was found

to be according to the following trajectory:

812 -^8io-^89->8i 1-^88 -^87-^86

where it has been observed that S9(2) = sii(2) = 0.125. However, the interesting

result is that the visitor terminates his visits after 7 steps and will never reach states

81 to 85 because the values of the probabilities Si(n) = 0, i = 1,..., 5.

Artistic examples Example 2.13 is a Markov-chain model for Magritte's painting The Castle

in the Pyrenees' [14, p.l l6] depicted in Fig.2-6. In 1958-1959 particularly,

Magritte was obsessed by the volume and weight of enormous rocks, but he altered

the laws of gravity and disregarded the weight of matter; for instance, he had a rock

sink or rise. 8imilarly, in Fig.2-6, he visioned a castle on a rock floating above the

sea. Considering the floating effect, two states may be visualized, i.e., 81 = the

rock is floating at some height above sea level, as seen in Fig.2-6; 82 = the rock is

floating at very small distance above sea level. The rock was chosen as system.

Thus, one may assume the following one-step probability matrix given by Eq.(2-

41). The reason for assuming pn and p22 to be unity, i.e., the system remains in

its state, is that since the rock is in a floating state, once it is "located" somewhere,

it will remain there.

Page 55: Applications of Markov Chains in Chemical Engineering

49

Fig.2-6. Magritte's gravityless world ("The Castle in the Pyrenees", 1959, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

Page 56: Applications of Markov Chains in Chemical Engineering

50

Si

p =

S2

Si S2

1 0 1

0 1 1

If we assume the following matrix:

Si S2

Si

P =

S2

1 0

1 0

(2-41)

(2-41a)

where p2i = 1» this suggests that if the rock was placed very near the sea level at

S2, it will move up to Si, a dead state, and remains there.

Example 2.14 demonstrates the situation depicted in Fig.0-2 by Escher as

a Markov process in the following way. States Si ( i = 1, 2, ..., 6) are various

locations which the system, a moving man, is occupying along its trajectory. The

one-step transition of the system is according to the following matrix:

P =

Si S2 S3 S4 S5 S6

Si

S2

S3

S4

S5

S6

0

0

0

0

0

0

0

0

1

0

1

0

0

1

0

0

0

0

1

0

0

0

0

1

0

0

0

1

0

0

0

0

0

0

0

0

(2-42)

Some explanations are needed regarding to the underlying assumptions in the

matrix. The probabilities pii on the diagonal are all zero, indicating that the system,

never remain in these states. P45 has been assumed 1, namely, that climbing along

the staircase from state 4 to 5 is possible. This is applicable if the reader covers

with his palm the right half of the staircase. He then sees that the staircase is in the

Page 57: Applications of Markov Chains in Chemical Engineering

51

upward direction. However, when he unveils the staircase, the latter seems to turn

over. Therefore, it has been assumed that P55 = 0, i.e., it is impossible to remain

at state 5. Since gravitational forces are effective, the man will fall to state 2.

Praying that he remains alive, it is assumed that P52 = 1. Assume the following

initial state vector, namely, the starting position is at state Si:

S(0) = [ 1, 0, 0, 0, 0, 0 ]

and applying Eq. (2-24) yield that:

si(n) S2(n) S3(n) S4(n) S5(n) S6(n)

S(1) = [ 0 ,

S(2) = [ 0,

S(3)= [ 0,

S(4) = [ 0,

S(5)= [ 0,

S(6) = [ 0,

S(7)=[ 0,

S(8) = [ 0,

0,

0,

1,

0,

1,

0,

1,

0,

0,

0,

0,

1,

0,

1,

0,

1,

1,

0,

0,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

0 ]

0 ]

0 ]

0 ]

0 ]

0 ]

0 ]

0 ]

(2-42a)

Inspection of the above state vectors behavior reveals that the system, at

steady state from n = 4, will end up walking up and down along the staircase

connecting states 2 and 3. The latter behavior was also independent of its initial

state vector, i.e., this is an ergodic Markov chain which is without memory to the

initial step.

Example 2.15, demonstrating the common situation of "dead state" (pii =

1) in Markov chains, is based on Escher's lithograph 'Reptiles' [10, p.284]

depicted in Fig.2-7. It demonstrates the life cycle of a little alligator. Amid all

kinds of objects, a drawing book lies open. The drawing on view, is a mosaic of

reptilian figures in three contrasting shades. Evidently one of them has tired of

lying flat and rigid among his fellows, probably in a "dead state", so he puts one

plastic-looking leg over the edge of the book, wrenches himself free and launches

into real life. He climbs up the back of a book on zoology and works his laborious

Page 58: Applications of Markov Chains in Chemical Engineering

52

way up a slippery slope of a set square to the highest point of his existence. At

states 5 and 7 he might slip and fall on the book joining again the "dead state"

situation. If this does not happen at this stage, after a quick snort, tired but

fulfilled, he goes downhill again, via an ashtray, to the level surface to that flat

drawing paper, and meekly rejoins his previous friends, taking up once more his

function as an element of surface division, i.e, the "dead state" situation.

Fig.2-7. Escher's reptiles demonstrating life cycle and "dead state"

(M.C.Escher "Reptiles" © 1998 Cordon Art B.V. - Baam - Holland. All rights reserved)

The above description can be framed by a Markov process in the following

way. A reptile was defined as ^y r m and the follo'wmg states shown in the figure

were selected, i.e. Si = reptile at position i, i = 1, 2,..., 11. On the basis of above

states, the following matrix may be established. Some assumptions made were:

Page 59: Applications of Markov Chains in Chemical Engineering

53

from S5, the reptile can move to states 85 and S7 with equal probabilities, i.e., 1/2.

Similarly, from S7 it can move to Sg and S9 with equal probabilities. Other

transitions of the reptile are governed by the one-step probability matrix given by

Eq.(2-43) which is the policy-making matrix of the reptile.

Si S2 S3 S4 S5 S6 S7 Sg S9 Sio Sii

P =

Si

S2

S3

S4

S5

S6

s? Sg

S9

Sio

Sii

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0.5 0.5 0

1

0

0

0

0

0

0

0

0

0

0

0

0

0.5

1

0

0

0

0

0

0

0

0

0

0.5

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

1

1

(2-43)

To demonstrate the behavior of a reptile along his life cycle we assume that

S(0) = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0 , 0]

i.e., the system (reptile) in Fig.2-7 initially at state Si,yields the following

dynamical behavior:

si(n) S2(n) S3(n) S4(n) S5(n) S6(n) S7(n) S8(n) S9(n) sio(n) sii(n)

S(l) = [0,

S(2) = [0,

S(3) = [0,

S(4) = [0,

S(5) = [0,

S(6) = [0,

S(7) = [0,

S(8) = [0,

1,

0,

0,

0,

0.

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

0,

0,

0.5,

0.5,

0.5,

0.5,

0,

0,

0,

0,

0.5,

0,

0,

0,

0,

0,

0,

0,

0,

0.25,

0.25,

0.25,

0,

0,

0,

0,

0,

0.25,

0,

0,

0 ,

0 ,

0 ,

0 ,

0 ,

0 ,

0.25 ,

0 , 0

0]

0]

0]

0]

0]

0]

0]

.25]

Page 60: Applications of Markov Chains in Chemical Engineering

54

S(9) = [0, 0, 0, 0, 0, 0.5, 0, 0 .25,0, 0 , 0.25]

As seen, the system attains a steady state after seven steps where the reptile

has a probability of 50% to occupy state 85 and 25% probability to occupy Sg and

Sii . Note, however, that in all these states the reptile is in a "dead state", pn = 1,

as also demonstrated Escher's Fig.2-7.

If the reptile is initially at S9, i.e.

S(0) = [0, 0, 0, 0, 0, 0, 0, 0, 1, 0 , 0]

it is obtained that:

si(n) S2(n) S3(n) S4(n) S5(n) S6(n) S7(n) sgCn) S9(n) sio(n) sii(n)

S(l) = [0, 0, 0, 0, 0, 0, 0, 0, 0, 1 , 0]

S(2) = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0 , 1]

S(3) = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0 , 1]

i.e., the reptile is in a steady "dead state" at Sn already after two steps.

Example 2.16 models the movement of fish based on Escher's painting

'Fish' [10, p.311] depicted in Fig.2-8. The selected states Si to S12, are various

locations of the fish as shown in Fig.2-8. The system is a fish. The underlying

assumptions on the transition of the system are: a) The probability of remaining in

some state pa = 0. b) The probability of occupying the state of an adjacent fish

moving in counter current flow is also zero, c) A fish can not jump above another

fish, d) There are equal probabilities to occupy two adjacent fish states. Bearing in

mind the above assumptions yields the matrix below.

Page 61: Applications of Markov Chains in Chemical Engineering

55

Fig.2-8. Movement of fish according to Escher (M.CEscher "Fish" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

P =

Si S2 S3 S4 S5 S6 S7 Sg S9 Sio Sii S12

Si

S2

S3

S4

S5

Se S7

Sg

S9

Sio Sii

S12

0 0

0

0

0

0.5

0.5 0

0

0

0

0

0

0

0

0.5

0

0.5

0

0

0

0

0

0

0.5

0.5

0

0

0

0

0

0

0

0

0

0

0.5

0.5

0

0

0

0

0

0

0

0

0

0

0 • 0

0.5

0

0

0.5

0

0

0

0

0

0

0 0 0

0 0 0

0 0 0

0 0 0

0.5 0.5 0

0 0 0

0 0 0.5

0.5 0.5 0

0 0 0

0 0 0.5

0 0 0

0 0 0

0

0

0

0

0

0

0.5

0

0

0

0

0

0

0

0

0

0

0.5

0.5 0

0

0

0.5

0

0

0

0

0.5

0

0

0

0

0

0

0

0.5

0

0

0

0

0

0

0

0

0.5

0

0.5

0

(2-44)

Page 62: Applications of Markov Chains in Chemical Engineering

56

Results of the dynamical behavior of the system are presented in Fig.2-9 for

the initial state vector S(0) = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], i.e., the fish is

initially occupying state 1 in Fig.2-8. As seen, there are two groups of states

attaining an identical steady state after ten steps where one group lags one step

behind the other. In addition, the occupation probability distribution of the states at

steady state, which equals 0.167, is periodic. One group lags by one step behind

the other where the system has to decide at each step what state to occupy among

six possibilities of an equal probability. It has also been observed that the steady

state is independent of the initial state vectors S(0) which classifies this case as an

ergodic Markov chain.

0.8

0.6

c C O '

0.4

0.2

s^ (n) = 82(0) = s^(n) = Sg(n) = Sg(n) = s^ (n)

SgCn) = s^(n) = Sg{n) = s^(n) = 5^ (11) = s^2(n)

8 10 0 2 4 6

n Fig.2-9. Probability versus time of occupying the states by the fish

Examples 2.17-2.22 (and 2.41, 2.42) relate to what is normally called

random walk [7, p.26; 4, p.89]. In principle, we imagine a particle moving in a

straight line in unit steps. Each step is one unit to the right with probability p or

one unit to the left with probability q, where p + q = 1. The particle moves until it

reaches one of the two extreme points called boundary points. The possibilities for

its behavior at these points determine several different kinds of Markov chains

Page 63: Applications of Markov Chains in Chemical Engineering

57

demonstrated in the following. An artistic demonstration of the different cases is

based on Escher's painting Sun and Moon [10, p.295] depicted in Fig.2-10.

Fig.2-10. Random walk demonstrations according to Escher

(M.C.Escher "Sun and Moon" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

We select nine states Si, S2,..., S9, which are various locations in the bird's

state field that can be occupied by the system - the moving bird. The states are

designated by 1,2, ..., 9 in Fig.2-10. Although the movement is not in a straight

line, the transition from one state to the other defined by the transition probability

matrices below, ensures the random walk model. States Si and S9 are the

boundary states and S2, S3,... , Sg the interior states.

Page 64: Applications of Markov Chains in Chemical Engineering

58

Example 2.17 is a simple random walk between two absorbing barriers. It

is characterized by the behavior of the moving bird, the system, that when it

occupies states S\ and S9, it remains there from that time on (pn = P99 = 1). In

this case the transition matrix is given by:

Si S2 S3 S4 S5 S6 S7 Sg S9

Si

S2

S3

S4

p = S5

S6

S7

S8

S9

where p + q = 1.

1

q 0

0

0

0

0

0

0

0

0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

0

0

0

0

0

0

0

0

p 1

(2-45)

Fig.2-11 presents results for q = 0.5 and S(0) = [0, 0, 0, 0, 1, 0, 0, 0, 0],

i.e., the bird is initially occupying S5 in Fig.2-10. It is observed that as time goes

by, the probabilities of the bird to occupy states 1 and 9 is increasing as well as are

identical. After 90 steps a steady state is achieved where S(90) = [0.5, 0, 0, 0, 0,

0, 0, 0, 0, 0.5]. In other words only the boundary states may be occupied with

identical probabilities. The above behavior is evident recalling that S\ and S9 are

absorbing boundaries. It should be noted that the behavior of the other states is

similar to S5 reported in the figure, i.e., their occupation probability diminishes

versus time. For q = 0.2, the steady state state vector reads S(30) = [0.004, 0, 0,

0, 0, 0, 0, 0, 0.996], i.e., the probability of occupying S9 is much higher than that

of state 1.

Page 65: Applications of Markov Chains in Chemical Engineering

59

CO"

1

0.8

0.6

0.4

0.2

n

— 1

r—

r s,(n)

T ;": ;.

i ''/'\ V VIV V ';'.

i 1

s,(n) = s (n) _

'. ''• '"' '\ .'

- 1

H

H

H

10 20 n

30 40

Fig.2-11. Probability distribution of occupying various states for the

two absorbing barriers Si and S9

Example 2.18 is a simple random walk with reflecting barriers. Whenever

the bird reaches states S\ and S9, it returns to the point from which it came, i.e. pn

= P99 = 0. The transition matrix reads

Si S2 S3 S4 S5 85 S7 Sg S9

Si

S2

S3 S4

p = S5

S6

S7

Sg

S9

wherep + q = 1.

0

q 0

0

0

0

0

0

0

1

0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

1

0

0

0

0

0

0

0

p 0

(2-46)

Page 66: Applications of Markov Chains in Chemical Engineering

60

Fig.2-12 presents results for q = 0.5 and S(0) = [0, 0, 0, 0, 1, 0, 0, 0, 0].

The general trend observed is that after 20 steps, the probabihties of occupying the

various states attain the following steady states corresponding to the groups

designated clearly in Fig.2-12:

S(20) = [0.125, 0, 0.25, 0, 0.25, 0, 0.25, 0, 0.125]

S(21) = [0, 0.25, 0, 0.25, 0, 0.25, 0, 0.25, 0]

S(22) = [0.125, 0, 0.25, 0, 0.25, 0, 0.25, 0, 0.125]

S(23) = [0, 0.25, 0, 0.25, 0, 0.25, 0, 0.25, 0]

This behavior is plausible recalling that the boundaries are of reflecting barrier type.

In other words, the moving bird will never be at rest. A final remark is that the

limiting behavior is independent of S(0), characterizing an ergodic Markov chain.

1

0.8

0.6

0.4

0.2

S2(n) = s^(n) = Sg(n) = Sg(n)

Sg(n) = Sg(n) = s^(n)

s^(n) = Sg(n)

10 15

Fig.2-12. Probability distribution of occupying various states for

the two reflecting barriers Si and S9

Example 2.19 belongs also to the random walk model with reflecting

barrier. However, it is assumed that whenever the moving bird, system, hits the

Page 67: Applications of Markov Chains in Chemical Engineering

61

boundary states Si or S9, it goes directly to the central state S5. The corresponding

transition matrix is given by Eq.(2-47) where p + q = 1. Fig.2-13 presents results

for q = 0.5 and S(0) = [1,0,0,0,0,0,0,0,0], i.e., the system is initially at the Si

reflecting barrier shown in Fig.2-10.

P =

Si S2 S3 S4 S5 S6 S7 Sg S9

Si

S2

S3

S4

S5

S6

S7

S8

S9

0

q 0

0

0

0

0

0

0

0

0

q 0

0

0

0

0

0

0

p 0

q

0

0

0

0

0

0

0

p 0

q 0

0

0

0

1

0

0

p 0

q

0

0

1

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p

0

q 0

0

0

0

0

0

0

p 0

0

0

0

0

0

0

0

0

p 0

(2-47)

As seen, the elements of the state vector oscillate towards a steady state

distribution attained for n = 31, i.e., S(31) = [0.029, 0.059, 0.117, 0.177, 0.235,

0.177, 0.117, 0.059, 0.029] where the maximum probability corresponds to S5.

As in previous cases, the steady state distribution is independent of S(0) and if the

system has to occupy some state, it will be, probably, S5, of the highest

probability.

Page 68: Applications of Markov Chains in Chemical Engineering

62

Fig.2-13. Probability distribution of occupying the states for the two

reflecting barriers Si and S9 sending the bird directly to S5

Example 2.20 is a random walk with retaining barriers (partially

reflecting). It has been assumed that the occupation probability by the system of

the boundary state, or moving to the other boundary state is 0.5. Thus, the one-

step transition probability matrix reads:

P =

Si

S2

S3

S4

S5

S6 S7

Sg

S9

Si

0.5

q 0

0

0

0

0

0

1 0.5

S2 0

0

q 0

0

0

0

0

0

S3 0

p 0

q 0

0

0

0

0

S4 0

0

p 0

q 0

0

0

0

S5 0

0

0

p 0

q 0

0

0

S6 0

0

0

0

p 0

q 0

0

S7 0

0

0

0

0

p 0

q 0

S8 0

0

0 0

0

0

p 0

0

S9

0.51 0

0

0 0

0

0

p 0.5

(2-48)

Page 69: Applications of Markov Chains in Chemical Engineering

63

where p + q = 1. Fig.2-14 presents results for q = 0.5 and S(0) = [0, 0, 0, 0,1, 0,

0, 0, 0], i.e., the system is initially occupying S5. As expected, the following

steady state is obtained at n = 90, i.e., S(90) = [0.500, 0, 0, 0, 0, 0, 0, 0, 0.500].

Under these conditions, the bird, has a 50% probability of occupying the boundary

states Si and S9 in Fig.2-10. It should also be noted that the limiting behavior for

large n, is independent of the initial state, i.e. a situation without memory with

respect to the far past.

0.8

0.6

0.4

0.2

0 10 15 20 25 30 35 40

n

Fig.2-14. Occupation probability distribution of some states for the

partially reflecting barriers Si and S9

Example 2.21 assumes that when the bird reaches one of the boundary

states Si or S9, it moves directly to the other, like in SLping-pong game (pi9 = p9i

= 1). Thus, the transition matrix reads:

Page 70: Applications of Markov Chains in Chemical Engineering

64

P =

Si S2 S3 S4 S5 Se S7 Sg S9

Si

S2

S3

S4

S5

S6

S7

Sg

S9

0

q 0

0

0

0

0

0

1

0

0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p 0

q

0

0

0

0

0

0

0

p 0

q 0

0

0

0

0

0

0

p

0

q 0

0

0

0

0

0

0

p 0

0

1

0

0

0

0

0

0

p 0

(2-49)

where p + q = 1. Fig.2-15 presents the variation against time of si(n), S3(n) and

S9(n) for q = 0.5 and S(0) = [0,0,1,0,0,0,0,0,0], i.e., the system is initially

occupying S3 as shown in Fig.2-10.

0)"

1

0.8

0.6

0.4

0.2

i:: ;; S •: II :• i i ;; j l - i n :• n ; : !:• i : : M - - M : : M : -

i n a , : : i , : ; i ; : | . : ,h;!h:l i : :

- i - v - v . ' - I - ' -

40 0 5 10 15 20 25 30 35 n

Fig.2-15. Ting-pong' type probability distribution of the boundary

states Si and S9

Page 71: Applications of Markov Chains in Chemical Engineering

65

As seen, S3(n) is oscillating and approaches zero at steady state. A similar behavior

was observed also for the other Si(n)'s excluding si(n) and S9(n) which correspond

to the boundary states S\ and S9. These oscillating quantities attain the following

limiting behavior of the state vectors for S(0) = [0,0,1,0,0,0,0,0,0]:

S(86) = [0.75, 0, 0, 0, 0, 0, 0, 0, 0.25]

S(87) = [0.25, 0, 0, 0, 0, 0, 0, 0, 0.75]

S(88) = [0.75, 0, 0, 0, 0, 0, 0, 0, 0.25]

S(89) = [0.25, 0, 0, 0, 0, 0, 0, 0, 0.75]

This behavior is plausible recalling the 'ping-pong' type behavior of the

boundaries. It should be noted that the limiting values of Si and S9 depend on

S(0).

For example:

a) S(0) = [1, 0, 0, 0, 0, 0, 0,0, 0] yields

S(l) = [0, 0, 0, 0, 0, 0, 0, 0, 1]

S(2) = [1, 0, 0, 0, 0, 0, 0, 0, 0]

where for S(0) = [0, 0, 0, 0, 0, 0, 0, 0, 1] the values of S(2) replace these of

S( l ) .

b) S(0) = [0, 1, 0, 0, 0, 0, 0, 0, 0] yields

S(79) = [0.875, 0, 0, 0, 0, 0, 0, 0, 0.125]

S(80) = [0.125, 0, 0, 0, 0, 0, 0, 0, 0.875]

where for S(0) = [0, 0, 0, 0, 0, 0, 0, 1, 0] the values of S(80) replace these of S(79).

c) S(0) = [0, 0, 0, 1, 0, 0, 0, 0, 0] yields

S(89) = [0.625, 0, 0, 0, 0, 0, 0, 0, 0.375]

S(90) = [0.375, 0, 0, 0, 0, 0, 0, 0, 0.625]

Page 72: Applications of Markov Chains in Chemical Engineering

66

where for S(0) = [0, 0, 0, 0, 0, 1, 0, 0, 0] the values of S(90) replace these of S(89).

d) S(0) = [0, 0, 0, 0, 1, 0, 0, 0, 0] yields

S(90) = [0.500, 0, 0, 0, 0, 0, 0, 0, 0.500]

S(91) = [0.500, 0, 0, 0, 0, 0, 0, 0, 0.500]

Example 2.22 is a modified version of the random walk. If the system

(bird) occupies one of the seven interior states S2 to S7, it has equal probability of

moving to the right, moving to the left, or occupying its present state. This

probability is 1/8. If the system occupies the boundaries Si and S9, it can not

remain there, but has equal probability of moving to any of the other seven states.

The one-step transition probability matrix, taking into account the above

considerations, is given by:

P =

Si S2

S3 S4

S5

S6

S7

Sg

S9

Si 0

1/3

0

0

0

0

0

0 1/8

S2 1/8

1/3

1/3 0

0

0

0

0 1/8

S3 1/8

1/3

1/3 1/3

0

0

0

0 1/8

S4 1/8

0

1/3 1/3

1/3

0

0

0 1/8

S5 1/8

0

0 1/3

1/3

1/3

0

0 1/8

S6 1/8

0

0 0

1/3

1/3 1/3

0 1/8

S7 Sg

1/8 1/8

0 0

0 0

0 0

0 0

1/3 0

1/3 1/3

1/3 1/3

1/8 1/8

S9 1/8

0

0

0 0

0

0

1/3

0

(2-50)

Fig.2-16 presents results for S(0) = [0,0,0,0,1,0,0,0,0] indicating that the

probability distribution of approaching a steady state for n = 13 reads S(13) =

[0.03, 0.078, 0.134, 0.168, 0.179, 0.168, 0.134, 0.078, 0.03]. It should be

noted that, generally, si(n) = S9(n), S2(n) = sgCn), S3(n) = S7(n), S4(n) = S6(n), i.e.,

the curves are symmetrical. In addition, it was found that the limiting distribution

is independent of S(0), i.e., the resulting Markov process generates an ergodic

chain.

Page 73: Applications of Markov Chains in Chemical Engineering

67

0 ) "

15 20 25 30

n Fig.2-16. Probability distribution of occupying various states

Example 2.23 is the last one on artistic examples. In Fig.2-1, Magritte has

demonstrated an impossible situation on the coexistence of Day and Night, two

states which can not coexist. It is also interesting to show how Escher

demonstrated the above situation in his woodcut Day and Night [10, p.273] and to

model this behavior. The situation is depicted in Fig.2-17. One sees gray

rectangular fields develop upwards into silhouettes of white and black birds. The

black ones are flying towards the left and the white ones towards the right, in two

opposing formations. To the left of the picture, the white birds flow together and

merge to form a daylight sky and landscape. To the right, the black birds melt

together into night. The day and night landscapes are mirror images of each other,

united by means of the gray fields out of which, once again, the birds merge. The

difference between the two demonstrations of day and night is therefore the

following. In Magritte's picture, 'half of the picture, i.e., clouds and sky, are at

daylight. The other 'half, house surrounded by trees, are at night. In Escher's

woodcut, however, the right 'half is at night, where the other one, on the left

(mirror image of the right), is at day light.

Page 74: Applications of Markov Chains in Chemical Engineering

68

Fig.2-17. The "coexistence" of Day and Night according to Escher

(M.C.Escher "Day and Night" © 1998 Cordon Art B.V. - Baam - Holland. All rights reserved)

In Fig.2-17 sixteen states S\ to Si6, i.e., sixteen possible locations of birds

in the sky along the flying route, are shown; the states are designated by 1, 2,...,

16. The system is a bird. The underlying assumptions for the system are: a) A

bird occupying states Sn to Si6 moves only to the right, b) A bird occupying S5

to Si moves only to the left, c) A bird occupying 85 to S10 can move to the left

and to the right, d) A bird occupying Si and S16 remains there, i.e., p n and

Pi6,16 = 1- Other assumptions can be concluded from the 16x16 one-step

transition probability matrix given by Eq.(2-51).

Page 75: Applications of Markov Chains in Chemical Engineering

69

Si 1 S2 !

S3

S4

Ss S6

P = S7

S8

S9

Sio Sii

Sl2

Sl3 Si4

Sl5

Sl6

Si 1

1/2

1/2

0

0

0

0

0

0

0

0

0

0

0

0

0

S2 S3 S4 S5 85 S7 Sg S9 Sio 0

0

0

1/2

1/2 0

1/2

0

0

0

0

0

0

0

0

0

0

0

0

0

1/2

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

1 / 2 0 0 0 0 0

1/2 0

1/3

0

0

0

0

0

0

0

0

0

0

0

0 0 0 0 0

0 0 1/3 1/3 0

1/3 0 0 0 1/3 1/3

0

0

0

0

0

0

0

0

0

1/3 0 0 1/3 0

1/5 1/5 1/5 0 1/5

0 1/3 0 1/3 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

Sii 0

0

0

0

0

0

0

1/3

0

0

0

1/3

0

0

0

0

S12 Si3 Si4

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0

0

0

0

0

0

0

0

0

Sl5 S16 0 0

1/5 0

0 1/3 0

1/2 0 1/2

0 1/3 0

1 0 0

0 0 0

0 0 1/2

0 0 0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 0

0 0

0 0

1/3 0

0 0

1/2 1/2

0 1/2

0 1

(2-51)

Fig.2-18 presents results for S(0) = [0,0,0,0,0,1/3,1/3,0,1/3,0,0,0,0,0,0,0]

indicating equal initial probabilities, 1/3, of occupying 85, 87 and 89 (6, 7, 9 in

Fig.2-17). 8ince this initial condition enables the bird to fly to the left or to the

right, it is observed in Fig.2-18 that at steady state the bird has a probability of

47.2% to occupy 81 (left) and 52.8% to occupy Si6(right). This behavior is

explained by the fact that 81 and S16 are absorbing (dead) states, thus, the

occupation probability of other states must diminish versus time.

Page 76: Applications of Markov Chains in Chemical Engineering

70

(0

0.6

0.5

0.4 h

0.3

0.2

»AaiiUUamM»

15 20

Fig.2-18. Occupation probability of states by the bird

Table 2-1 shows limiting occupation probabilities (for n = ©o) for the case where

the initial occupation of the bird is of a certain state, i.e., Si(0) = 1, i = 1, 2, ....,

16.

Table 2-1. Limiting occupation probabilities of states

Si and Si6 for Si(0) = 1

i 1 1-5

6

7

8

9

10

1 11-16

Si(oo)

1

0.542

0.542

0.292

2/3

0.292

0

Sl6M

0 1 0.458

0.458

0.708

1/3

0.708

1 _

Page 77: Applications of Markov Chains in Chemical Engineering

71

Generally, the results comply with the assumptions spelled out above. For

example, if the bird is initially at states Si to S5, it will eventually occupy state Si.

If it is initially at states Si 1 to Si5, it will finally occupy state Si5. If it is initially at

states S6 to Sio, it will at the end occupy state Si or S16 depending on the

magnitude of the relative probability.

Real life examples Examples 2.24 and 2.25 were inspired by the assassination of the Prime

Minister of the state of Israel Itzhak Rabin (1922-1995) on Saturday, November

4th, 1995. During his life, Rabin was a soldier, the Chief of Staff, participating in

all Israeli wars and the greatest motivating force for peace in the Middle East.

Thus, the examples deal with Life and Death as well as with Peace, War and No

peace-No war situations.

Example 2.24 assumes two states a man, designated as system, can

occupy, viz.. Si = Life, S2 = Death. The two states may be expressed by the

following 2x2 matrix:

P = Si

S2

Si

q

0

S2

l-q

1

(2-51)

where q is a parameter depending, among others, upon the age of the man, his

profession (soldier, university professor, worker), health, etc. For q = 0, the

matrix is of the "dead-state" type, i.e., once the system occupies S2, it will remain

there for ever. In other words, if the man is initially alive, after one step he will

die. Fig.2-19 presents results for S(0) = [1, 0] and q = 0.5, i.e., the man is

initially alive and his probability to occupy this state is 50%. His probability to die

is also 50%. It is observed that after ten years the system will occupy state S2,

namely, "dead state". It should be noted that si(n) = 1/2" where S2 is occupied

only as n-^00. Of course si(lO) = 1/2 ^ is nearly zero for all practical purposes.

Page 78: Applications of Markov Chains in Chemical Engineering

72

c^ (/)"

0 2 4 6 8 10

n

Fig.2-19. Probability of remaining alive

Example 2.25 presents a three-state model for which S\ = War-No war,

S2 = War and S3 = Peace. The system is at some state. The matrix for this case

is the following one:

P =

Si

S2

S3

Si

q r

u

S2

P t

V

S3 1-q-p

l-r-t

1-u-v

(2-52)

Eq.(2-52) is a multi-parameter matrix which depends on time and other factors not

easy to evaluate. This is because we deal with complicated states. Fig.2-20

presents results for S(0) = [0, 1, 0], i.e., the system (some country) is initially at a

state of war. The following values were also assumed for the transition

probabilities: q = p = 0.1,r = 0.5, t = 0, u = 0.2 and v = 0.1. It is observed that

after five steps the system approaches a steady state for which S(5) = [0.206,

0.091, 0.704]. The state vector indicates that the chances for peace are quite high,

70.4%, promising a bright future.

Page 79: Applications of Markov Chains in Chemical Engineering

73

0.8

0.6

0.4

0.2

Fig.2-20. Dynamical probability of Si = War-No war, S2 = War and

S3 = Peace

A very interesting behavior may be obtained by varying the initial state vector S(0).

It is observed that the steady state behavior is independent of S(0), where such a

Markov chain, later discussed, is defined as ergodic.

Example 2.26 is a simulation of a tennis game. The system is a tennis

ball for which the following states are defined and schematically depicted in Fig.2-

21a. Si = the ball is down on the ground on the right-hand side, briefly

designated, DR (down right); S2 = the ball is up in the air on the right-hand side,

UR (up right); S3 = the ball is up in the air on the left-hand side, UL (up left); S4 =

the ball is down on the ground on the left-hand side, DL (down left).

Page 80: Applications of Markov Chains in Chemical Engineering

74

S3=UL=UpLeft

m

S4=DL= Down Left

® - tennis ball

S2 = UR = Up Right

o

Fig.2-21a. Scheme of the states in a tennis game

The corresponding matrix reads:

S i=DR S2 = UR S3 = UL 84 = DL

P =

Si

S2

S3

S4

1

P21

P31

0

0

0

P32

0

0

P23

0

0

0

P24

P34

1

(2-53)

where the one-step transition probabilities are:

P21 = UR -^ DR, P23 = UR -^ UL, p24 = UR -^ DL,

P31 = UL -» DR, P32 = UL -> UR, p34 = UL -^ DL.

pi 1 = P44 = 1 means that once the ball is on the ground, it will remain there until the

game starts again. P22 = P33 = 0 indicates that if the ball is in the air the game must

go on. It should also be noted that the various probabilities in the matrix depend on

the characteristics of the players which depend on time and their talent. In the

following demonstrations the probabilities remain unchanged. Assume the

following values:

P2i= 0.01, p23 = 0.99, p24 = 0,

P31=0, P32 = 0.8, p34 = 0.2

Page 81: Applications of Markov Chains in Chemical Engineering

75

The above data indicate that the tennis player on the right-hand side of the tennis

court is a better one. This is because p2i is significantly lower than P34, i.e., the

probability to hit the ball to his court is lower. On the other hand, both players will

always hit the ball up to the air to the other court, i.e., p3i = p24 = 0. Fig.2-21b

presents results for the dynamics of the tennis game corresponding to S(0) = [0, 1,

0, 0] indicating that the ball is initially at the right-hand side court. It is observed

that as time goes by, the probability for state S4 to be occupied is increasing, and

the player on the right-hand side is going to win the game. The steady state value

of the state vector reads S(68) = [0.048, 0,0, 0.952], indicating the above trend.

c 0)"

0.8

0.6

0.4

0.2

n

1 ] 1 1

— - "" r 1 ^ '

• . : : : : , ! " sjn), UR

: . 1 A'\*

• ; • ^' .' I i: J/; : ;:

_X \\ »[ 'fT

u / • 1 ' l 1 • 1 • 1 %

1 • 1 ) I * ' ,'•

'''{'-' • ' . - y •••• • • • r « *\

» %»

1

NFV

1 1

s <n), DL 4 1

H

J

s,(n), DR J

/ 1 '^ i\x ' v ' J - . .

10 15 20 n

25 30 35 40

Fig.2-21b. Dynamics of a tennis game (better player on the

right-hand side)

Assume now the following values for the one-step transition probabilities:

P21 = 0.01, P23 = 0.99, P24 = 0,

P31 = 0.12, p32 = 0.8, P34 = 0.08

Considering the values of p2i, P23 and P24, indicates that the performance of the

tennis player on the right-hand side is unchanged. However, the player on the left-

hand side improved his performance; there are good chances he can now hit the ball

from his court down to the ground on the right-hand side, i.e., p3i = 0.12 instead

of nil before. This situation is reflected in Fig.2-22 where the probability for

occupying Si is increased, which is opposite to the behavior in Fig.2-21b. At

Page 82: Applications of Markov Chains in Chemical Engineering

76

steady state S(68) = [0.619, 0, 0, 0.381], indicating the above trend. It is

interesting to note that if p3i is increased, i.e., p3i = 0.2 ( p32 = 0.8, P34 = 0),

S(68) = [1, 0, 0, 0], i.e., the left-hand player has excellent chances to win the

game at the end.

1

0.8 h

0.6 h-

0.4

0.2

p 1 p 1 t *

s s H

I ' l "J • 1 ^ 1 , J '

1 , J '

/::'"

• « ' 1 ' 1 '

1

'! ^ •1 • I',

1

^ X " "

' '• •"'• 1 ' •"• .'

UR

1

s/n). DR

s,(n). DL

'\lx V .,r,.**

• =d

H

10 20 n

30 40

Fig.2-22. Tennis game dynamics of the better player

(on left-hand side)

Example 2,27 is a simulation of an ideal pendulum, taken as system,

where drag force is ignored. The four states assumed for the system, demonstrated

in Fig.2-23a, are: Si = maximum height of the pendulum on left-hand side; S24eft

= minimum height of the pendulum while reaching this point from left-hand side;

S2,right = minimum height of the pendulum while reaching this point from right-

hand side; S3 = maximum height of the pendulum on the right-hand side. The

above states are schematically depicted in Fig.2-23a

Page 83: Applications of Markov Chains in Chemical Engineering

77

Fig.2-23a. Scheme of the states of an ideal pendulum

The governing one-step transition matrix is given by:

P =

Si

S2,left

S2,right

S3

Si

0

0

1

0

S2,left

1

0

0

0

S2,right

0

0

0

1

S3

0

1

0

0

(2-54)

If the following initial state vector is assumed S(0) = [1, 0, 0, 0], i.e., the

pendulum is initially at Si, the behavior depicted in Fig.2-23b is obtained. As

seen, S\ is occupied after four steps whereas states S2,right and S2,ieft» which are

practically the same point, are occupied after two steps, once from left-hand side

and once from right-hand side. As observed also, the behavior of the system is

un-damped oscillating. The more complicated case of a damped oscillations may be

obtained by varying the transition matrix versus the number of steps.

Page 84: Applications of Markov Chains in Chemical Engineering

78

I' V \ V 2 4 6

n

Fig*2-23b. The behavior of an ideal pendulum

Example 2.28, a student progress scheme at a university, is a slightly

modified version of the example appearing in [7, p.30]. In the faculty of

Engineering at Ben-Gurion university of the Negev in Israel, a student, the system,

is studying for four years. The following states are assumed: Si-first year; S2-

second year; Ss-third year and S4-fourth year. Additional states are: Ss-the student

has flunked out; S6- the student has graduated. Let r be the probability of flunking

out, p the probabiUty of repeating a year and q the probability of passing on to next

year where r + p + q = 1. The six states are governed by the following matrix:

P =

Si

S2

S3

S4

S5

S6

Si

P 0

0

0

0

0

S2

q

p 0

0

0

0

S3 0

q

p 0

0

0

S4 0

0

q

p 0

0

S5 r

r

r

q 1

0

S6

0 1 0

0

r

0

1

(2-55)

The first situation considered is a student having some probability of

repeating a year, p = 0.1, and good chances for passing to next year, i.e., q = 0.9;

Page 85: Applications of Markov Chains in Chemical Engineering

79

thus, his flunk out probabiUty r = 0. In addition, the student is a first year student,

thus S(0) = [1, 0, 0, 0, 0, 0]. Fig.2-24 demonstrates the progress of the student at

the university, where after eight steps (years) the probability of graduating is

100%, S6(8) = 1. The reason for not graduating after four years are his slight

chances, 10%, for repeating a year. If p = 0, the student will graduate exactly after

four years.

0.8 h

0.6

0.4

0.2 h

4 n

Fig.2-24. Student's progress dynamics at the university in the

absence of flunk out probability (r = 0)

Fig.2-25 demonstrates the progress of the student with good chances to pass

to next year, however, there are chances of 16% to flunk out. The prominent result

is that at his fourth year, he has only 50% chances for graduating, i.e., he has equal

probabilities to occupy S5 or 85. The probability of occupying the other states has

diminished.

Page 86: Applications of Markov Chains in Chemical Engineering

80

CO"

1 \ \

p = 0, q = 0.86, r = 0.16

8

Fig.2-25. Student's progress dynamics at the university in the

presence of flunk out probability (r = 0.16)

Examples 2.29-2.31 relate to weather forecast, whereas example 2.31

demonstrates a way to improve the forecast by considering previous days

information.

Example 2.29 [4, p.78] predicts the weather in Tel Aviv (Israel) by a two-

state Markov chain during the rainy period December, January and February. The

system is the city of Tel Aviv where the states it can occupy are: Si = D, Dry day

and S2 = W, Wet day. Using relative frequencies from data over 27 years it was

found that during this season the probability of wet day following a dry day is

0.250 (= pi2) and the probability of a dry day following a wet day is 0.338 (=

p2i). These data resulted in the following transition matrix:

P =

Si

S2

Si = D

0.750

0.338

S2 = W

0.250

0.662

(2-56)

Page 87: Applications of Markov Chains in Chemical Engineering

81

Results of the calculation of the state vectors S(n), are depicted in Fig.2-26

for different initial state vectors S(0) spelled out at the top of the figure. Thus for

example, given that January 1st is a dry day (right-hand side of Fig.2-26), yields a

probability of 0.580 that January 6th will be a dry. If January 1st is a wet day

(middle of Fig.2-26), then the probability that January 6th is a dry day is 0.586.

However, after ten days, the equilibrium conditions has for all practical purposes

been reached. Thus, for example, if we call December 31st day 0 and January 10th

day 10, then whatever distribution S[0] we take for day 0, we find that S(10) =

[0.575, 0.425]. Such a Markov chain is defined as ergodic and is without memory

to the initial state.

1

0.8 ^ 0.6 ^-0 .4

0.2 0

S(0) = 1 1

I s n)

1 1

[0.5. 0.5] 1 1

1 1

S(0) = [0,1] — \ — \ — r

J I \ L

S(0) = \ ' ' X?iln)

/ 1 1

= [1.0] 1 1

1 1

-\

—=i

0 2 4 6 8 10 0 2 4 6 8 10 0 2 4 6 8 10

n n n Fig.2-26. Weather in Tel Aviv for different initial weather conditions

Example 2.30 is related to the weather in the land of Oz [7, p.29] where

they never have two nice days one after the other. If they have a nice day, they are

just as likely to have snow as rain the next day. If they have snow (or rain), they

have an even chance of having the same the next day. If there is a change from

snow or rain, only half of the time is this a change to a nice day. The system is the

land of Oz whereas the states it can occupy are: S] = Rain; S2 = Nice; S3 = Snow.

The transition matrix reads:

P =

Si

S2

S3

Si

1/2

1/2

1/4

S2 1/4

0

1/4

S3 1/4

1/2

1/2

(2-57)

Three cases, differing by the initial state vector S(0) given above each graph, were

studied. The results of the computations are depicted in Fig.2-27, indicating that

Page 88: Applications of Markov Chains in Chemical Engineering

82

after two steps (days) the weather will always be snowy; it will continue like this

independent of the weather two days before, i.e., on S(0). The limiting behavior is

similar to that obtained in example 2.29, indicating an ergodic Markov chain.

S(0) = [1.0,0] S(0) = [0,1,0]

vsjn) / ^a^^H

S(0) = [0.0,1]

<t(") 1 L

k

r~

\

1

v /

. s/n)

1,

1 J

=%wj i 1 ;

3 0

Fig.2-27. Weather forecast

1 2 n

Example 2.31. In examples 2.29 and 2.30 the Markov property clearly

held, i.e., the new step solely depends on the previous step. Thus, the forecast of

the weather could only be regarded as an approximation since the knowledge of the

weather of the last two days, for example, might lead us to different predictions

than knowing the weather only on the previous day. One way of improving this

approximation is to take as states the weather of two successive days.

This approach is known as expanding a Markov chain [7, p.30, 140], which

may be summarized as follows. Consider a Markov chain with states Si, S2, ...»

Sz. The states correspond to the land of Oz defined as system. We form a new

Markov chain, called the expanded process where a state is a pair of states (Si, Sj)

in the original chain, for which pij > 0. We denote these states by Sij. Assume

now that in the original chain the transition from Si to Sj and from Sj to S^ occurs

on two successive steps. We shall interpret this as a single step in the expanded

process from the state Sij to the Sjk- With this convention, transition from state Sij

to state Ski in the expanded process is possible only if j = k. Transition

probabilities are given by

P(ij)(jl) = Rl; P(ij)(kl) = 0 for j 9i k

Consider now example 2.30 applying the expanded process approach.

Making the following designations, S = Snowy day, N = Nice day and R = Rainy

day, yields the following states: Si = NR, S2 = NS, S3 = RN, S4 = RR, S5 = RS,

S6 = SN, S7 = SR, Sg = SS. Note that NN is not a state, since PNN = 0 in the

Page 89: Applications of Markov Chains in Chemical Engineering

83

original process in example 2.30. The transition matrix for the expanded process

is:

RR RN RS NR NS SR SN SS

P =

RR

RN

RS

NR

NS

SR SN

SS

1/2 0

0

1/2

0

1/2

0

0

1/4 0

0

1/4

0

1/4

0

0

1/4

0

0

1/4

0

1/4

0

0

0

1/2

0

0

0

0

1/2

0

0

1/2

0

0

0

0

1/2

0

0

0

1/4

0

1/4

0

0

1/4

0

0

1/4

0

1/4

0

0

1/4

0

0

1/2

0

1/2

0

0

1/2

(2-58)

The results of the computations are depicted in Fig.2-27a for two initial state

vectors given at the top of the graphs. The one on the left-hand side gives 50%

chances for the two previous days to be either Rainy-Nice or Snowy-Nice. The

one on the right-hand side gives 100% chances for the two previous days to be

Snowy-Rainy. The prominent observation is that the probability distribution of the

states becomes unchanged from the 7th day on and is independent of S(0). It is

given by S(7) = [0.2, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.2], indicating that the weather

on the 7th day has 20% chances to be either Rainy or Snowy. If it was Snowy on

the 7th it will remain like this; if it was Rainy, it will continue Rainy. Recalling the

previous example, where only the information on the previous day was taken into

account, it was obtained that the weather in the winter season will be 100% Snowy

from the 3rd day on.

S(0) = [0,0.5,0,0,0,0,0.5,0] S(0) = [0,0,0,0,0,1,0,0] T — \ 1 — I 1 — r

S2(n) = s (n) = ...=s^(n)

h s (n)

1—I—I—\ 1 — r

8 (11) = s^(n)=... = s (n)

(n) = s^(n)H

Fig.2-27a. Weather forecast by the expanded Markov process

Page 90: Applications of Markov Chains in Chemical Engineering

84

Examples 2.32-2.33 below treat the behavior of a drunkard. In the first

example the new step solely depends on the previous step. In example 2.33, the

behavior depends on the last two steps.

Example 2.32. A drunkard, the system, is living in a small town with

four bars, states Si = i, i = 1, ..., 4. As time goes by, the drunkard jumps from

one bar to the other according to the following one-step transition matrix. As seen,

the probabilities of moving from one bar to the other are equal; also, the drunkard

eventually leaves the bar to the next one, i.e. pii = 0.

P =

Si

S2

S3

S3

Si S2 S3 S4

0 1/3 1/3 1/3

1/3 0 1/3 1/3

1/3 1/3 0 1/3

1/3 1/3 1/3 0

(2-59)

Fig.2-28 shows the behavior of the drunkard corresponding to two initial

state vectors S(0) shown above the graphs. It is observed that after six steps the

behavior of the drunkard reaches a steady state where his chances to visit any of the

bars on the next step are 25%. The steady state is independent of S(0), thus the

chain is an ergodic one.

1

0.8

^ 0.6

0)" 0.4

0.2

0 (

S(0) = [1,0,0,0] . 1 1 1 1 1

J}f s (n) = S3(n) = s (n) _

AA^-D 1 2 3 4 5

n 5 (

S(0) = [0.5,0,0,0.5] 1 1 i 1 1 1

- s {n) = s (n) J

/ I 1 1 1 1 1 D 1 2 3 4 5 6

n

Fig.2-28. The drunkard behavior

Example 2.33 applies the expanded process approach (elaborated in

example 2.31) for the drunkard, the system, in example 2.32. The states are

designated in the present example by Sy indicating that before moving to state Sjk

at step n+1, the drunkard spent in state Si some time at step n-1, and at step n he

Page 91: Applications of Markov Chains in Chemical Engineering

85

visited state Sj. In this way, the effect of the last two steps, rather than one step, on his next visits can be studied. Considering the assumptions made in example 2.32, yields the following matrix where q = 1/3:

Si2 Si3 Si4 S21 S23 S24S31 S32 S34 S41 S42 S43

O O O q q q O O O O O o l

P =

S12

Sl3

Si4

S2I

S23

S24

S31

S32

S34

S41

S42

S43

0

0

q 0 0

q 0 0

q

0

0

0

0

q 0 0

q 0 0

q 0

0

0

0

q 0 0

q 0 0

q 0

0

0

0

0

0

0

0

q 0 0

q 0

0

0

0

0

0

0

q 0 0

q 0

0

0

0

0

0

0

q 0 0

q 0

q

0

0

q

0

0

0

0

0

0

q

q 0

0

q

0

0

0

0

0

0

q

q 0 0

q

0

0

0

0

0

0

q

0

q 0 0

q 0 0

q 0

0

0

0

q 0

0

q

0

0

q

0

0

0

(2-60)

The results of the computation for two initial state vectors given at the top of the graphs, each comprising of twelve states, are depicted in Fig.2-29. They indicate that after 7 steps, the drunkard is always at the same situation, i.e., his chances to occupy the next state, depending on his past deeds of two steps before, are 8.3%. This situation is independent of his initial step, which is plausible recalling that we deal with a drunkard.

S(0) = [0.5,0 0,0.5]

0.2

0

" 1 — I — I — I — i — I — r s,(n) = s (n)

kl

s^(n) = . . . = Sg(n)

\ s (n) =... = s (n) \ / 9 12

l^' \ L_J I L 1 2 3 4 5 6 7 8

n

S(0) = [1,0 0] I 1 1 1 1 1 1

- \ s (n) =... = s (n)

\ ' J *i" '7" '""i "f^

1 ]

"r 2 3 4 5 6 7 8

n

Fig.2-29. The drunkard*s behavior according to the expanded Markov process

Page 92: Applications of Markov Chains in Chemical Engineering

86

Example 2.34 deals with actuarial considerations needed for premium

calculations. The problem may be presented by assuming the following states that

a system, the customer of an insurance company, may occupy: Si = Healthy

customer, S2 = Handicapped customer, and S3 = Dead customer. The following

matrix clarifies the interaction between the states

P =

Si

S2

S3

Si S2 S3

Healthy Handicapped Dead

p q 1-p-q 0 r 1-r

0 0 1

(2-61)

For example, the matrix indicates that a healthy man can remain healthy, can

become handicapped or die, i.e., p n = p, pi2 = q and pi3 = 1- p - q. If he is

handicapped, he can never become healthy again, thus P2i = 0, and if he is dead,

he is in the so-called dead state. It should be noted that the parameters p, q and r

depend strongly on age but are taken here as constants.

Fig.2-30 demonstrates the situation of a young man and an old man after five

steps (years); both are initially healthy, i.e., S(0) = [1, 0, 0]. It has been assumed

that the young man has 95% chances to remain healthy (p = pn = 0.95) whereas

the old man has 50% chances to remain in Si (p = p n = 0.5); other quantities

shown on the graphs, r and q, are identical. It is observed in the figure that the

probability of the young man to remain healthy after five years are quite good, si(5)

= 0.774; for the old man si(5) = 0.0313, quite small to remain healthy. An

interesting observation is related to the effect of r, i.e. to remain handicapped, on

the state vector S(n); for example:

Young man: S(5) = [ 0.774, 0.0613, 0.210] r = 0

S(5) = [ 0.774, 0.0905, 0.136] r = 1

Old man: S(5) = [ 0.0313, 0.0380, 0.930] r = 0

S(5) = [ 0.0313, 0.0013, 0.968] r = 1

It may be observed that si(5) is not affected by r where the other quantities are

influenced by varying r.

Page 93: Applications of Markov Chains in Chemical Engineering

87

1

0.8

0.6 c^ vT 0.4

0.2

0

Y(

-

- p = 0.95 q = 0.02

- r = 0.97

-

3ung man i 1

-^ji!!L

-

-

sjn) s (n) -

2 3 n

Old man

Fig.2-30. The future dynamics of a young man and an old man

Example 2.35 is the Ehrenfest diffusion model [6, p.21] for a simple

random walk with reflecting barrier presented in example 2.18. The model

assumes two containers A and B containing Z molecules. The containers are

separated by a permeable membrane so that the molecules may move freely back

and forth between the containers. It is assumed that at each instant of time t, one

of the Z molecules chosen at random, is moving from one container to the other.

The system are molecules in container A and the state Sj of the system is the

number of molecules in container A which equals j - 1. Thus, the following states

are assumed: Si = 0, S2 = 1, S3 = 2, S4 = 3, ..., Sz+i = Z molecules. In the

Ehrenfest model, if A has j molecules, i.e., it is in state Sj+i, it can on the next step

move to Sj or to Sj+2 with probabilities

Pj+lj = j/Z; Pj+i,j+2 = (Z - j)/Z; j = 1,..., Z (2-62)

pj+ij+i = 0; j = 0 ,1 , . . . , z

The transition probability matrix is then given by:

Page 94: Applications of Markov Chains in Chemical Engineering

88

P =

Si

S2

S3

S4

Si S2 S3 S4 85

0 1 0 0 0

1/Z 0 (Z-l)/Z 0 0

0 2/Z 0 (Z-2)/Z 0

0 0 3/Z 0 (Z-3)/Z

• Sz

. 0

. 0

. 0

. 0

Sz+1 0

0

0

0

Sz

Sz+1

0

0

0

0

0

0

0

0

0

0

... 0

... 1

1/Z

0

(2-63)

Fig.2-31 presents results for total number of molecules Z = 3 and S(0) = [1,

0,0,0]. The initial state vector indicates that the system (molecules in container A)

is initially at Si, i.e., does not contain any molecule. In other words the three

molecules are in vessel B. It is observed in the figure that the system attains the

following two sets of constant values:

S(9) = [1/4, 0, 3/4, 0], S(10) = [0, 3/4, 0, 1/4]

S(l l ) = [1/4, 0, 3/4, 0], S(12) = [0, 3/4, 0, 1/4]

The results indicate that states S2 and S3 have the same occupation

probability which oscillate against time. Note that state S2 corresponds to one

molecule in container A where in state S3 two molecules should occupy container

A. Thus, if at step n = 10 there is one molecule in A, two molecules will occupy

container B. If at step n = 11 there are two molecules in A, then one molecule will

occupy B and this process repeats its self ad infinitum.

Page 95: Applications of Markov Chains in Chemical Engineering

89

CO"

0.8

0.6

0.4

0.2 h"

Fig.2-31. The approach towards equilibrium for a total number of

molecules in the containers Z = 3

Fig.2-32 presents results for an even number of molecules in the containers,

i.e., Z = 8 and an identical initial state vector as before, S(0) = [1, 0, 0, 0]. It is

observed that the system attains the following behavior after about 20 steps, which

is similar to that before, i.e.:

S(25) = [0, 0.063, 0, 0.438, 0, 0.437, 0, 0.062, 0]

S(26) = [0.008, 0, 0.219, 0, 0.547, 0, 0.218, 0, 0.008]

S(27) = [0, 0.063, 0, 0.438, 0, 0.437, 0, 0.062, 0]

S(28) = [0.008, 0, 0.219, 0, 0.547, 0, 0.218, 0, 0.008]

The results indicate that S4(25) = S6(25) and that states S4, S5 and 85 have the

highest occupation probability which oscillate against time; other state probabilities

are lower. Note that S4 corresponds to three molecules in container A, 85 to five

molecules where in 85 four molecules should occupy container A. Thus, if at step

n = 25 there are three or five molecules in A, because the states are of equal

probability, the mean value is four. Thus, since the total number of molecules is m

= 8, four molecules will occupy container B. If at step n = 26 there are four

molecules in A corresponding to 85 with the highest probability, then also four

molecules will occupy container B. Therefore, at steady state the eight molecules

will be equally distributed between the two containers.

Page 96: Applications of Markov Chains in Chemical Engineering

90

CO"

Fig.2-32. The approach towards equilibrium for a total number of

molecules in the containers Z = 8

The general conclusion drawn is that in both cases, i.e., with odd and even

number of molecules in the containers, the tendency of the system, molecules in

container A, is to shift towards an equilibrium state of half molecules in each

container. This trend is also expected on physical grounds.

Example 2.36 is the Bernoulli-Laplace model of diffusion [15, p.378],

similar to the one suggested by Ehrenfest. It is a probabilistic analog to the flow of

two incompressible liquids between two containers A and B. This time we have a

total of 2Z particles among which Z are black and Z white. Since these particles are

supposed to represent incompressible liquids, the densities must not change, and so

the number Z of particles in each container remains constant. The system are

particles in container A of a certain color and the state Si of the system, is the

number of these particles in container A where

Si = i - l ( i = l , 2 , ...,Z+1) (2-64)

If we say that the system is in state Sk = k - 1, i.e., the container contains k - 1

black particles, this implies that it contains Z - (k -1) white particles. The transition

probabilities for the system are:

Page 97: Applications of Markov Chains in Chemical Engineering

91

pj+l j = 0/Z)2; pj+i,j+i = 2j(Z - j)/Z2; pj+i.j+2 = ((Z - j)/Z)2 ; j = 1,..., Z

Pj+l,k+l = 0 whenever i j - k I >1; j = 0, 1 ,..., Z (2-65)

The transition probability matrix is then given by:

P =

Si

S2

S3

S4

Sz

Sz+1

Si 0

P21 0

0

S2 1

P22

P32 0

Ss 0

P23

P33 P43

S4 0

0

P34

P44

Ss .. 0 ..

0 ..

0 ..

P45 ••

. Sz

0

0

0 0

Sz+1 0

0

0 0

0

0

0

0

0

0

0

0

0

0 Pzz Pz,z+1

1 0

(2-66)

where

P21 = (1/Z)2; P22 = 2(Z - 1)/Z2; p23 = ((Z - 1)/Z)2

P32 = (2/Z)2; p33 = 4(Z - 2)/Z2; P34 = ((Z - 2)/Z)2

P43 = (3/Z)2; P44 = 6(Z - 3)/Z2; P45 = ((Z - 3)/Z)2

Pzz = 2(Z-l)/Z2; p2,2+i = l/Z2

Fig.2-33 presents results for 2Z = 8, i.e., 4 black and 4 white particles. Two

initial state vectors S(0) were considered, given at the top of the figure. On the left-

hand side, it is assumed that initially container A contains 2 black particles. On the

right-hand side, there are 50% chances that container A will contain one black

particle and 50% to contain four black particles. The results indicate that after ten

steps, the system attains a steady state, independent of the initial conditions, where

S(10) = [0.014, 0.229, 0.514, 0.229, 0.014]. Such a process is known as

Page 98: Applications of Markov Chains in Chemical Engineering

92

ergodic Markov chain later discussed. As seen, S3(10) = 0.514 corresponding to

two black particles in each container at steady state, is the highest probability with

respect to the other states (Si = 0, S2 = 1, S3 = 2, S4 = 3 and S5 = 4). In other

words, the state of the highest probability will exist at steady state; this is also the

expected physical result. An interesting behavior observed in Fig.2-33 on the left-

hand side is the following. In S(0), it was assumed that S3 = 2 has 100%

probability. As seen, the probability of this state, i.e., S3(n), remains always the

highest along the path towards equilibrium until it reaches its ultimate value. On the

other hand, on the right-hand side, the state S3 = 2 had initially a probability of

0%. As seen, along its approach towards equilibrium, S3(n) is continuously

increasing and attains the value of S3(10) = 0.514, which remains constant from

thereon.

1

0.8

^ 0.6

^' 0.4

0.2

0

S(0) = [0,0,1,0,0] T T

\,>_>fe.''^'___3 . s (n) = s (n)

s (n) = s (n)

S(0) = [0,0.5.0,0,0.5]

4 6 n

10

Fig.2-33. The dynamics of approach towards equilibrium for a total

number of particles in the containers 2Z = 8 (4 black and 4 white)

Example 2.37 considers the random placement of balls [15, p.379] where

a sequence of independent trials, each consisting in placing a ball at random in one

of Z given cells, is performed. The system are balls and the state of the system is

the number of cells occupied by the balls. The state Si of the system is given by:

Si = i - 1 (i = 1, 2,..., Z+1) (2-67)

Thus, if we say that the system is in state Sk, this implies that k-1 cells are occupied

and Z - (k -1) cells are still free. For the placing process of the balls in the cells, the

following transition probabilities apply:

Page 99: Applications of Markov Chains in Chemical Engineering

Pj+lo+l=j/Z; pj+i,j+2 = (Z-j)/Z; j = 0, 1,...,Z

yielding the following matrix:

Si

S2

S3 S4

Si 0

0

0

0

S2 S3 1 0

P22 P23

0 P33 0 0

S4 0

0

P34 P44

S5 .. 0

0 ..

0 ..

P45 ••

. s 0

0

0 0

Sz+1 0

0

0 0

Sz

Sz+1

0 0

0 0

0

0

0

0

0

0

Pzz Pz,z+1

0 1

where

93

(2-68)

(2-69)

P22 = 1/Z; P23 = (Z - 1)/Z; P33 = 2/Z; p34 = (Z - 2)/Z

P44 = 3/Z; P45 = (Z - 3)/Z; Pzz = (Z - 1)/Z; pz,z+i = (Z - 1)/Z

Fig.2-34 presents results for three cells, i.e., Z = 3. Thus, the four states

are: Si = 0, S2 = 1, S3 = 2 and S4 = 3 cells occupied. S2 = 1 indicates that one cell

has already been occupied by the ball. It may be observed that in both cases

depicted in the figure, differing by the initial state vectors S(0), the ultimate

situation is identical, i.e., all four cells are occupied, as expected. This occurs after

15 steps. The maximum in the Si(n) curves, i = 2, 3 is interesting and clear. For

example, S2(n) corresponding to S2 = 1 is attaining a maximum after one step, n =

1, because after Si = 0, S2 must come.

Page 100: Applications of Markov Chains in Chemical Engineering

94

S(0) = [1.0,0,0] S(0) =

_s,(n) /

[0.25,0.25,0.25,0.25]

^ wS (n)

0 5 10 15 n n

Fig.2-34. The dynamics of cell occupation by balls for Z = 3

Example 2.38 is concerned with cell genetics [15, p.379] where a Markov

chain occurs in a biological problem which may be described roughly as follows.

Each cell of a certain organism contains N particles, some of which are of type A,

others of type B. The system is a cell and the state of the cell is the number of

particles of type A it contains; there are N + 1 states. A cell is said to be in state i if

it contains exactly i -1 particles of type A, where

Si = i - l ( i = l , 2 , . . . , N + l ) (2-70)

The cell is undergoing the following process. Daughter cell are formed by

cell division, but prior to its division each particle replicates itself. The daughter

cell inherits N particles chosen at random from 2i particles of type A and 2N - 2i

particles of type B present in the parental cell. The probability that a daughter cell

occupies state k is given by the following hypergeometric distribution

(2j)! (2N-2J)! (Nir Pj+i.k+1 j.,(2j _ k)! (N - ]f)!(N - 2j + Ic)! (2N)!

(2-71)

j , k = 0, 1,2, ...,N

Note that pi i and PN+I,N+I = 1; Pj+i,k+i = 0 if the expressions in the parenthesis in

the denominator < 0.

Page 101: Applications of Markov Chains in Chemical Engineering

95

The behavior predicted by the model is demonstrated for N = 4, yielding the

following states from Eq.(2-70): S] = 0, S2 = 1, S3 = 2, S4 = 3 and S5 = 4

particles of type A. The corresponding matrix obtained from Eq.(2-71) reads:

Si

S2

= S3

S4

S5

Si

1

0.2143

0.0143

0

0

S2 0

0.5714

0.2286

0

0

S3 0

0.2143

0.5143

0.2143

0

S4 0

0

0.2286

0.5714

0

S5 0

0

0.0143

0.2143

1

(2-72)

Results for the formation dynamics of particles of type A from their cells is

demonstrated in Fig.2-35. Three initial state vectors, given at the top of each

figure, were considered. The calculation indicate that a steady state is reached after

thirty steps (generations), always at Si and S5, which are dead or absorbing states.

For the case on the left-hand side, si(30) = 0.75 and S5(30) = 0.25 where for that

on the right-hand side si(30) = 0.25 and S5(30) = 0.75. The state with the highest

probability is always the one nearest to the state of the highest probability in the

initial state vector. The case in the middle is symmetrical, hence, si(50) = S5(50) =

0.50. It is interesting to mention that the results after sufficiently many

generations, n -^00, comply with the following theoretical predictions. The entire

population will be (and remain) in one of the pure states Si and SN+I ; the

probability of these two contingencies at steady state are

Sj(oo) = 1 - i/N, SN+I(«>) = i/N where i is the number of particles of A at the initial

state. For example, i = S3(0) = 2 and N = 4, for the case on the middle in Fig.2.35. Thus, Si(oo) = S5(oo) = 0.5.

S(0) = [0,1,0.0,0] S(0) = [0,0,1,0,0]

: 1

s,n 4"

_\s,(n) =

\/r

\

s 1

1 1

-

(n) = s(n) 5 —

S(0) = 1

\ \~\

\

\ /C^^<^

[0.0.0,1.0] 1

.^(")

-*f- *-

1

_s_^(n)J

JnT

20 0 10 n

15 20 0 10 n

15 20

Fig.2-35. The formation dynamics of A-type particles for N = 4

Page 102: Applications of Markov Chains in Chemical Engineering

96

Example 2.39 is taken from population genetics [15, p.380]. Consider the

successive generation of a population which is kept constant in size by the selection

of N individuals in each generation. A particular gene, assuming the forms A and

a, has 2N representatives. If in the nth generation A occurs i times, then a occurs

2N - i times. The system is the population (such as plants in a com field). The

state of the system is the the number of times that the A-gene occurs after some

generations. A population is said to be in state i if A occurs 2i times, i.e.,

Si = i - 1 ( i = l , 2 , . . . , 2N+ 1) (2-73)

where the number of states is 2N + 1. Assuming random mating, the composition

of the following generation is determined by 2N Bernoulli trials in which the

probability of the occurrence of the A-gene is i/2N. We have, therefore, a Markov

chain with

(2N)! M Yfi M" -k)!V2N>' V 2 N ;

2N-k

Pj+iMi - i,,(2N

j , k = 0 , 1, ...,2N

(2-74)

Note that pn and P2N+i,2N+i = 1; Pj+i,k+i = 0 if the expressions (2N - k) in the

denominator < 0. The above indicates that at states Si and S2N+1» called

homozygous, all genes are of the same type, and no exit from these states is

possible.

The behavior predicted by the model is demonstrated for N = 2, yielding the

following states from Eq.(2-73): Si = 0, S2 = 1, S3 = 2, S4 = 3 and S5 = 4 A-gene

occurrence. The corresponding matrix reads:

P =

Si

S2

S3

S4

S5

Si

1

0.3164

0.0625

0.0039

0

S2 0

0.4219

0.2500

0.0469

0

S3 0

0.2109

0.3750

0.2109

0

S4

0

0.0469

0.2500

0.4219

0

S5 0

0.0039

0.0625

0.3164

1

(2-75)

Page 103: Applications of Markov Chains in Chemical Engineering

97

Fig 2.35 demonstrates the dynamical behavior of the A-gene occurrence for

two initial state vectors S(0). The behavior is, in general, similar to example 2.38

where the process is terminated at one of the dead states for which pii = 1. In other

words, generally Si and S2N+1 designate the homozygous states at one of which

the ultimate population will be fixed, depending on the magnitude of the

corresponding probabilities. It should be emphasized that the ultimate results are

dependent only on the initial step where the steady state probabilities are given by

Sj(oo) = 1 - i/(2N), S2N+i(° ) = i/(2N). I is the number of A-genes at the initial

step. For example, for the case on the right-hand side in, I = si(0) = 1 and N = 2. Thus, si(oo) = 0.75 and S5(oo) = 0.25.

1

0.8

^ 0 . 6

" "0.4

0.2

0

S(0) = [0,1,0.0.0] I 1 ' 1

^s^(n) ^ ^ ^

i • y"""^^

\l -Tit 3 - : • '

I

— 1 —

^(n) H

"H

s,(n) 1

10 15

S(0) = [0,1/3,1/3,1/3,0]

^s (n) = Sg(n)

n • n Fig.2-35a. The dynamics of A-gene occurrence for N = 2

15

Example 2.40 is a breeding problem [15, p.380] where in the so-called

brother-sister mating, two individuals are mated. Among their direct descendants,

two individuals of opposite sex are selected at random. These are again mated, and

the process continues indefinitely. With three genotypes AA, Aa and aa for each

parent, we have to distinguish six combinations of parents, designated as states,

which we label as follows: Si = AA*AA, S2 = AA*Aa, S3 = Aa*Aa, S4 = Aa*aa,

S5 = aa*aa, S6 = AA*aa. The system is two individuals of opposite sex which are

mating. Based on above reference, the matrix of transition probabilities reads:

Page 104: Applications of Markov Chains in Chemical Engineering

98

P =

Si S2 S3 S4 S5 S6

Si

S2

S3

S4

S5

S6

1 0

1/4 1/2

1/16 1/4

0 0

0 0

0 0

0

1/4

1/4

1/4

0

1

0

0

1/4

1/2

0

0

0 0

0 0

1/16 1/8

1/4 0

1 0

0 0

(2-76)

Fig.2-36 demonstrates the results of brother-sister mating against time. As

expected from the matrix given by Eq.(2-76), containing two dead states S\ and S5

(pil and P55 = 1), the system will eventually occupy one of these states, depending

on the magnitude of the relative probabilities si(n) and S5(n). The latter depends on

S(0) given in Fig.2-36 at the top of the graphs.

S(0) = [0,1,0,0,0,0] S(0)

U h^^(")

W^T R u i^ r i 7^-i i/^ 1

= [0,0,0,0,0,1]

^("L-T-^"^"^

• " ^

'

s^(n) = i

. . • • • L ~ ' . " * '

's^"jJ —

-

•.... —• . H

n ' n Fig.2-36. Breeding dynamics

10 15

Examples 2.41-2.42 are examples of random walk type on real life

problems, in addition to examples 2.17-2.22.

Example 2.41 demonstrates a game [6, p.21] related to a simple random

walk between two absorbing barriers, considered also in example 2.17. Assume

that Jacob and Moses have five shekels (Israeli currency) divided between them.

On one side there is a symbol of a lectf and on the other side appears the number

one. The shekels are assumed fair ones, so that the probability of a leaf on a toss

equals the probability of a one on a toss, equals 1/2. Jacob tosses a coin first and

records the outcome, lector one. Then Moses tosses a coin. If Moses matches

Page 105: Applications of Markov Chains in Chemical Engineering

99

Jacob (obtains the same outcome as Jacob), then Moses wins the shekel, otherwise

Jacob wins. Note that Moses or Jacob win with probability 1/2. Let the states Si =

i - 1 (i = 1, 2, ..., 6) represent the number of shekels that Moses, the system, has

won. The game ends when Moses has 0 or 5 shekels, i.e., states Si and S6 are

then dead or an absorbing states. Given that Moses is in state k (has k -1 shekels),

he goes to state k + 1 (wins) with probability 1/2 or goes to state k - 1 (loses) with

probability 1/2. The above rules of the game may be summarized in the following

matrix:

P =

Si S2 S3 S4 S5 S6

Si

S2

S3

S4

S5

S6

1

1/2

0

0

0

0

0

0

1/2

0

0

0

0

1/2

0

1/2

0

0

0

0

1/2

0

1/2

0

0

0

0

1/2

0

0

0

0

0

0

1/2

1

(2-77)

where, for example, P45 = 1/2 is the probability that Moses wins a fourth coin

given that he already has three coins.

Fig.2-36 demonstrates the dynamics of the game for two cases. On the left-

hand side is observed that initially Moses is in S2, he has one shekel. As time goes

by, his chances to win are decreasing where after 20 steps his situation is given by

S(20) = [0.8, 0, 0, 0, 0, 0.2]. This means that there are 80% chances he will be

left without money. It is also observed that after one step, there are 50% chances

he will be left without money at all (be in Si) or win one shekel (be in S3) and from

thereon his chances to loose are increasing. So if he is smart, on the one hand, and

knows Markov chains, on the other, he should better stop the game after one step.

On the right-hand side, Moses begins with three shekels, he is in S4, and as time

goes by, his chances to win are increasing where S(25) = [0.4, 0, 0, 0, 0, 0.6].

Page 106: Applications of Markov Chains in Chemical Engineering

100

S(0) = [0,1,0,0,0,0]

L s

/ \

%

- ;

S(0) = [0,0,0,1,0,0] 1

,(n)

> s(") . -

'^if-i^^'\\\S^

1

^ — -

. \ l X .

sJn) J

s,(n) _

X ^ - > 15 0 10

n n Fig.2-36. Dynamics of the matching shekels game

15

Example 2.42 deals with a political issue of establishing a coalition in

Israel in the eighties. Generally, the treatment of the problem is based on

the

random walk model incorporating the reflecting-absorbing barrier effects, i.e., p n

= t and P99 =1, respectively, assuming that the maximum size of the coalition

comprise nine parties.

In Fig.2-37, the Israeli caricaturist Moshik [19] demonstrates the efforts

made by Mr. Menahem Begin, the Prime Minister these days, to establish a

coalition. He is seen trying to attract to the coalition additional parties in order to

establish a stable government. As observed in the figure, so far five parties have

joined the coalition.

The system is the coalition headed by Mr. Begin, where a state is the number

of parties in the coalition, i.e.. Si = 1, 2,..., 9. The underlying assumptions of the

"political game" are: a) The probability to increase the coalition from one state to the

next one is p, independent of Si. b) There is a probability, r, that the coalition will

remain unchanged in its size, c) Similarly, there is a probability, q, that the size of

the coalition will decrease due to unsuccessful negotiations. Note that q + p + r =

1.

Page 107: Applications of Markov Chains in Chemical Engineering

101

Fig.2-37. Establishment of a coalition in Israel

The following matrix summarizes the above considerations:

P =

Si 1 S2

S3 S4

S5

S6 S7

S8

S9

Si t

q 0 0

0

0

0

0

0

S2 i-t

r

q 0

0

0

0

0

0

S3

0

p r

q 0

0

0

0

0

S4

0

0

p r

q 0

0

0

0

S5 0

0

0

p r

q 0

0

0

S6 0

0

0

0

p r

q 0

0

S7

0

0

0

0

0

p r

q 0

Sg 0

0

0

0

0

0

p r

0

S9

0

0

0

0 0

0

0

p 1

(2-78)

Page 108: Applications of Markov Chains in Chemical Engineering

102

Fig.2-38, containing the input data and S(0) above the graphs, demonstrates

results of the calculations with respect to the following points. Note that S(0)

corresponds in all cases to a coalition already with five parties, i.e. S5, as seen in

Fig.2-37. In general, depending on time, a stable coalition consisting of nine

parties will be established because S9 = 9 will, eventually, acquire the highest

probability. In case c, the approach towards a stable coalition is very fast because

the 'negotiation-success factor' p = 0.8 is a relatively high value, in comparison to

p = 0.25 in cases a and b. It is also observed in case c that at each step, after

intensive rounds of talks, the highest probability corresponds to the state of a

higher number of parties, i.e., a larger coalition.

In cases a and b, S5 remains of the highest probability until n = 15; from

thereon the probability of S9 becomes the highest. However, this behavior

depends on the reflecting barrier effect governed by the factor t in the matrix given

by Eq.(2-78). If t = 0 (case a), i.e., an ideal reflector, S9(n) > si(n). If t = 1 (case

b). Si becomes an absorbing or dead state like S9, S9(n) = si(n) and the chances to

establish a stable coalition or to fail are the same. It should be noted that for high

p's, the effect of t is negligible, as observed in case c.

S(0) = [0,0,0.0,1.0,0,0,0.0] S(0) = [0.0.0.0.1,0.0.0.0] \ 1

q = 0.25, p = 0.25, r=: 0.5 J t=1

.(n) s,(n) = Sg(n)

^ - ^ ' X ^ 1 - - - - 1 10 n

15 20

(a) (b)

Page 109: Applications of Markov Chains in Chemical Engineering

103

1

0.8

0.6

S(0) = [0,0,0,0,1,0,0,0,0]

s.(n) ' s(n) '

u p ® q = 0.1,p = 0.8, r = 0.1 ttfW^^^ , , t=o-i

V ^ ' - ' " ' .,(n) , _ ^ 10 n

(c)

15 20

Fig.2-38. The dynamics of establishing the coalition

Examples 2.43-2.45 treat a few models of periodic chains, also referred

to as recurrent events. Generally speaking, a system undergoes some process as a

result of which it occupies the states sequentially. The latter repeats its self ad

infinitum or attains some steady state.

Example 2.43, recurrent events 1 [15, p.381], obeys the following

transition probabilities matrix:

P =

Si

S2

S3

S4

S5

Si

Pll 1

0

0

0

S2

P12

0

1

0

0

S3

P13

0

0

1

0

S4 . . . P14 . . .

0 . . .

0 . . .

0 . . .

1 . . .

(2-79)

To visualize the process which generates from above matrix, suppose that

initially the system occupies Si. If the first step leads to Sk-i, the system is

bound to pass successively through states Sk-2, Sk-3, ..., and at the k th step it

Page 110: Applications of Markov Chains in Chemical Engineering

104

returns to Si, whence the process starts from scratch passing, in principle, the

above steps again and again.

Practical examples conforming with the above model are the following ones.

A drunkard, the system, treated also in examples 2.32 and 2.33, is acting now

according to different rules dictated by the transition matrix, Eq.(2-79). The states

Si to S4 are four bars in the small town the drunkard is occupying. Another

example is concerned with a dancer, the system, acting as follows. There are four

nice dancers, states S\ to S4, standing on a circle at equal distances. The dancer is

moving from one nice dancer to the other, performing with her a dance, and

moving to the next one. His occupation of states are according to Eq.(2-79). The

interesting question is what happens versus time with the system! Fig.2-39a,b

demonstrate the behavior of the system as a function of the Pij's in the matrix,

Eq.(2-79), and S(0). The characteristic behaviors observed in the figure are: a) In

each state the system oscillates until a steady state is achieved, b) The magnitude

of the steady state is independent of S(0), i.e., the chain is ergodic. c) The

magnitude of the steady state depends on the policy-making matrix, Eq.(2-79),

i.e., on the py's. Regarding to the behavior of the system, the drunkard or the

dancer, it is observed in Fig.2-39a,b that at steady state, it will remain in Si, the

state of the highest probability. The ultimate values of the state vectors are:

Fig.2-39a, S(24) = [ 0.333, 0.300, 0.234, 0.133]

Fig.2-39b, S(10) = [ 0.500, 0.300, 0.150, 0.050]

S(0) = [1,0,0.0] S(0) = [0,0,0,1]

20 0 5 10 15

0.2, P13 = 0.3, P14 = 0.4

Page 111: Applications of Markov Chains in Chemical Engineering

105

1

0.8

0.6

S(0) = [1,0,0.0] S(0) = [0,0,0,1] « • " I ^

/^s/") s (n) 83(0) s (n)

b) p u = 0.4, P12 = 0.3, P13 = 0.2, P14 = 0.1 Fig.2-39 a,b. The dynamical behavior of the system

Example 2.44, recurrent events 2 [15, p.382], obeys the following transition-probability matrix:

P =

Si

S2

S3

S4

S5

Si S2 S3 S4 S5

qi Pi 0 0 0

qi 0 P2 0 0

q3 0 0 P3 0

q4 0 0 0 p4 (2-80)

The matrix indicates that the system moves from one state to the other, and upon reaching the new state it has always some probability of returning to the initial state.

Fig.2-40 demonstrates results of the calculations for the case of four states, i.e.. Si,..., S4. The corresponding matrix reads:

P =

Si

S2

S3 S4

Si

qi

q2

qa 1

S2

pi 0

0

0

S3 0

P2

0

0

S4 0

0

P3 0

(2-80a)

Page 112: Applications of Markov Chains in Chemical Engineering

106

The input data are reported on the figure. On the left-hand side the values of the

qi's are relatively high in comparison to the right-hand side, thus, the probability of

remaining in or returning to Si is high, as a result of which the approach towards a

steady state is after one step. By increasing qi, the occupation of the states by the

system is of a recurrent type, where eventually a steady state is achieved of the

state-probability distribution. In both cases, the Markov chain is ergodic.

0.8 P

0.6

0.4

0.2

0

S(0) = [1,0,0,0]

_V. s(n) q=0.8, p=0.2;i = 1,2

[_ 2 ") ^ (n) s^n)

-_.-"Iy-.-i.--1

S(0) = [1,0,0.0]

s(n)

.s (n) ^ q=0.2,p. = 0.8;i = 1,2

5 10 15 20 0 5 n

Fig.2-40. The dynamical behavior of the system

10 15 20 n

Example 2.45, recurrent event 5, demonstrates 3. periodic chain [4, p. 102]

by considering the behavior of a Saudi sheilc. Fig.2-41 shows the Saudi sheik,

defined as system, opening the door for an amazing beauty symbolizing the West at

the entrance of his harem. The Israeli caricaturist Moshik [19] demonstrates in the

figure the approaching process of the West towards Saudi Arabia at the beginning

of the eighties. The following model is developed for the caricature, which is

applicable for cases 1 and 2 below. The states are seven Saudi beauties, Si =

Saudi beauty , i = 1, ..., 7 and Sg = Western beauty. The caricature may be

understood in several ways, dictating the construction of the 8x8 one-step transition

probability matrix.

Page 113: Applications of Markov Chains in Chemical Engineering

107

P =

Si S2 S3 S4 S5 Se 87 Sg Si

S2

S3

S4

S5

S6 s? Sg

0 0

0

0

0

0

1/2

q

0 0

0

0

0

0

1/2

q

1

1

0

0

0

0

0

0

0

0

1/3

0

0

0

0

0

0

0

1/3

0

0

0

0

0

0

0

1/3

0

0

0

0

0

0 0

0 0

0 0

1/2 1/2

1/2 1/2

1/2 1/2

0 0

0 p

(2-81)

Fig.2-41. The harem of the Saudi sheik

The matrix demonstrates several interesting characteristics, for example: from Si the sheik never goes to S2; similarly, from S2 he never goes to Si; however, from these states he always moves to S3. From S3 the sheik has to decide where to go, because P34 = P35 = P35, As seen, by analyzing the various pij's, which are

Page 114: Applications of Markov Chains in Chemical Engineering

108

assumed to remain constant, a complete understanding of the sheik's behavior can be obtained.

Case 1. It has been assumed that in the matrix p = 0 and q = 1/2, indicating that Sg has no preference over the other states. Sg has been located on the third floor of the harem, right to S7 as seen in Fig.-41. It has also been assumed that the sheik is initially occupying Si, i.e., S(0) = [1, 0, 0, 0, 0, 0, 0, 0]. The occupation dynamics of the sheik is obtained by applying Eq.(2-24), i.e., multiplying the state vector by the matrix, Eq.(2-81), yielding Fig.2-42.

c^ ^

0.8

0.6

0.4

0.2

0 2 4 6 8 10 12 14

n Fig.2-42. States occupation dynamics of the Saudi sheik

The prominent observation in Fig.2-42 is the periodic behavior of the system

(sheik), where each state is reoccupied after four steps. Hov ever, in general, the s\\<^\]fi is occupying at each step another state, thus acting very very hard, sooner or later might affect his health. The only way to change the dangerous results predicted by the model, is by modifying the matrix, Eq.(2-81); the latter depends on the habits of the sheik. Thus, he should be advised by his doctor accordingly.

Additional interesting observations are: a) Whenever the sheik is moving to 87, he should consider to occupy instead Sg because the occupation probability of

Page 115: Applications of Markov Chains in Chemical Engineering

109

these states is 50%. He has the same problem with Si and S2. b) The occupation

problem of S4, S5 and 85 is more complex since he has to decide among three

beauties, c) No problem with S3; the occupation probability of this state, whenever

reached, is 100%.

Case 2. It has been assumed here that in matrix, Eq.(2-81), p = 1, q = 0

and, as in previous case, S(0) = [1, 0, 0, 0, 0, 0, 0, 0]. The result in Fig.2-43

indicate clearly a significant preference for Sg over the other states. After 43 steps

it is obtained that S(43) = [0, 0, 0, 0, 0, 0, 0, 1], i.e., the sheik will be 'absorbed'

at Sg. Noting the amazing beauty in Fig.2-41, his behavior is not surprising at all.

if)'

zyz-r.

S3(n)

J s^(n) = 85(11) = Sg(n)

A s (n)

,;!i\/!\AA.>:/^v --?<N ^''T** ^**t "^ .l-»i t*«ifc

10 15

n

20 25 30

Fig.2-43. States occupation dynamics of tlie Saudi slieili

Case 3. In cases 1 and 2 the Saudi sheik was the system and the beauties

were the states. Now we look at the caricature in Fig.2-41 from a different point of

view. The system is the Western beauty where the states S j , i = 1, ..., 8, are the

eight rooms occupied by the Saudi beauties. Note that number 8 in the figure,

representing the western beauty in the above cases, designates now room number 8

located on the third floor of the harem, right to S7. The figure shows also that

initially the West is invited by the sheik to join the harem, hoping it will occupy

Page 116: Applications of Markov Chains in Chemical Engineering

no only one of the rooms, i.e. number 8 which is the only available one. The behavior

of the system may be deduced from the following single-step transition matrix:

P =

Si S2 S3 S4 S5 S6 S7 Sg

Si

S2

S3

S4

S5

S6 S7

S8

1/5

1/4

1/5

1/8

1/6

0

0

0

1/5

1/4

0

1/8

1/6

0

0

0

1/5

0

1/5

1/8

0

1/4

1/6

0

1/5

1/4

1/5

1/8

1/6

1/4

1/6

1/4

1/5

1/4

0

1/8

1/6

0

1/6

1/4

0

0

1/5

1/8

0

1/4

1/6

0

0 0

0 0

1/5 0

1/8 1/8

1/6 1/6

1/4 0

1/6 1/6

1/4 1/4

(2-82)

Two cases were explored for the dynamical behavior of the West. In the first

case, the West occupies initially Si; this is expressed by the initial state vector S(0)

given on the top of Fig.2-44, left-hand side. On the right-hand side, the state

vector corresponds to the case where the West has equal probabilities to occupy

states S2, S4 and S6, i.e., Si(0) = 1/3, i = 2, 4, 6. The results depicted in the

figure are very interesting, indicating that after ten steps the system has a certain

probability to be found in every state, i.e., the domineering process of Saudi Arabia

by the West is very effective. Moreover, this process after some time, becomes

independent of the initial step, i.e., an ergodic Markov chain which is without

memory to the past. The probability distribution at steady state after ten steps is

given by S(10) = [0.119, 0.095, 0.119, 0.190, 0.143, 0.095, 0.143, 0.095]

noting that the probabilities are not too different from each other.

Page 117: Applications of Markov Chains in Chemical Engineering

I l l

S(0) = [1,0,0,0,0.0,0,0] S(0) = [0,1/3,0,1/3,0,1/3,0,0]

s,(n)=s^(n) s^(n)=s^(n)

sjn)= sJn) = Sg(n)

0 5 ^ 10 15 0 5 _ 10 15

Fig.2-44. The domineering dynamics of Saudi Arabia by the West

Example 2.46 is of fundamental importance and completes our real life

examples. It considers the imbedded Markov chain of a single-server queuing

process [4, pp.8, 89]. Queuing is encountered in every comer of our life such as

queues at servers, telephone trunk Unes, traffic in public transportation - bus, trains

as well as airports, queues at surgery rooms in hospitals and governmental offices,

department stores, supermarkets and in a variety of industrial and service systems.

One of the simplest models of queuing is the following one. Let customers

arrive at a service point in a Poisson process [see 2.2-3] of rate X [customers

arriving per unit time]. Suppose that customers can be served only one at a time

and that customers arriving to find the server busy queue up in the order of arrival

until their turn for service comes. Such a queuing policy is called First In First Out

(FIFO). Further, suppose that the length of time taken to serve a customer is a

random variable with the exponential p.d.f. = (probability density function) given

by

f(t) = pe"' ^ ( t>0) (2-83)

f(t) [1/time] is the probability density that the length of time it takes to complete the

service is exactly t, f(t)dt is the probability that the length of time to complete the

service is between t and t + dt where

prob{x > t} = f f(t)dt = f pe'P^dt: Jo Jo

1 - e -PT (2-83a)

Page 118: Applications of Markov Chains in Chemical Engineering

112

is the probability to complete the service between 0 - x. For x -> ©o the probability

equals unity, i.e. the probability to complete the service at infinite time is unity.

The constant 1/(3 [time/customer] is the expected mean service time per customer

where p is the mean number of customers being serviced per unit time. The

exponential distribution is employed to describe the service probability distribution

function in many queuing systems. It should also be noted that for any other

distribution the problem becomes intractable. It can also be shown that if the

service of a customer is in progress at time t, and the p.d.f. of the service time is

given by Eq.(2-83), then the probability that the service time is completed in the

time (t, t + At)is:

PAt + 0(At2) (2-84)

where O(At^) denotes a function tending to zero at the same rate as At .

The above queuing process, and more precisely between consecutive times of

two customers who completed to receive their service, can be described also by the

so-called discrete imbedded Markov chain of a single server. In the queuing

literature this process is noted by M/M/1. The first M denotes a Poisson arrival

time, the second M is an identical independent exponential probability distribution

service time and the 1 represents a single server. We define the system as the

queue and the state of the system is the number of customers waiting in the queue.

The state space is the possible number of states, i.e., S] = 0, S2 = 1, S3 = 2,... If

the state space is finite. Si = 0, S2 = 1, S3 = 2, ..., Si = i - 1 customers. It should

be noted that the state of the system is evaluated at the moment the nth customer

has completed to receive her/his service. By observing the state of the system at

these points (completion of service time) the queuing process of the system is

described by the Markov property of absence of memory, i.e., ignoring the past

history of the customers which have already been served. Accordingly, let Xn

denote the number of customers in the queue immediately after the nth customer has

completed his service. Thus, Xn includes the customer, if any, whose service is

just commencing. Then we can write down the following equations:

Page 119: Applications of Markov Chains in Chemical Engineering

113

Xn-1+Yn+i (Xn>l) (2-85a)

Xn+1 = I Yn+1 (Xn = 0) (2-85b)

where Yn+i is the number of customers arriving during the service time of the

(n+1 )ih customer. Eq.(2-85a) expresses the fact that if the nth customer does not

leave an empty queue behind him (Xn > 1), then during the service time of the

(n+1 )ih customer, Yn+i new customers arrive and his own departure diminishes

the queue size by 1. If the nth customer leaves an empty queue, then the (n+7)th

customer arrives and depart after the completion of his service during which Yn+i

new customers arrive. We can distinguish by a simple argument between two

types of behavior of the system. The rate of arrival of customers is X and within a

long time to the average number of customers arriving is Xto- As indicated before,

the mean service time per customer is 1/(3. If the service of the customers were to

go on continuously, the average number of customers served during time to would

be pto. Hence, if ^ > P, we can expect the queue of unserved customers to

increase indefinitely; in a practical application when this occurs, customers would

be deterred from joining the queue and X will therefore decrease. If, however, X, <

P the server needs to work only for a total time of about Xto/P in order to serve the

customers: that is the server will be idle, i.e., the process will be in state Si = 0, for

about a proportion 1- X/p of the time.

In the following presentation we assume that customers arrive in a Poisson

process of rate X and that their service times are independently distributed. The

distribution function B(t) (0 < t < <») is the probability that the service time ts

satisfies

B(t) = prob{ts<t} (2-86)

where

dB(t) = B'(t)dt (2-87)

Page 120: Applications of Markov Chains in Chemical Engineering

114

needed later is the probability that the service time lies between t and t + dt. B'(t) is

the p.d.f. defined in Eq.(2-83).

A useful quantity based on the above concepts, needed for evaluating the

probabilities in the one-step transition matrix, is the following one

bi = prob{Yn = i} (i = 0,1,2,...) (2-88)

i.e., bi is the probability that the number of customers arriving during the service of

the nth customer is equal to i. The above quantity may be calculated in the

following way. For the Poisson process

Pi(t) = prob{N(t) = i} = t-^KUy/il (i = 0, 1, 2,...) (2-89)

where Pi(t) is the probability that the number N(t) of events occurred (customers

arrived) is equal to i, given that the service time is t. From the above definitions,

Eqs.(2-86) and (2-89), it follows that the product Pi(t)dB(t) designates the

probability that i additional customers will be added to the queue (see Eq.2-85a) if

the service time is between t and t + dt. Since the service time ts varies as 0 < tg <

oo, the probability that the number of customers arriving during the service time is

equal to i, is given by [4, pp.88]

Jo i(t)dB(t) (i = 0,1,2,. . .) (2-90)

From Eqs.(2-85a) and (2-85b) the elements of the transition matrix of the process

Xn are given by the following matrix:

P =

Si

S2

S3

S4

Ss

Si

bo bo 0

0

0

S2

bi

bi

bo 0

0

S3

b2

b2

bl

bo 0

S4

b3

b3

b2

bl

bo

Ss . . . b4 . . .

b4 . . .

b3 . . .

b2 . . .

bl . . .

(2-91)

Page 121: Applications of Markov Chains in Chemical Engineering

115

It should be emphasized that the transition matrix, Eq.(2-91), applies to the time

interval between two consecutive service completion where the process between the

two completions is of a Markov-chain type discrete in time. The transition matrix

is of a random walk type, since apart from the first row, the elements on any one

diagonal are the same. The matrix indicates also that there is no restriction on the

size of the queue which leads to a denumerable infinite chain. If, however, the size

of the queue is limited, say N - 1 customers (including the one being served), in

such a way that arriving customers who find the queue full are turned away, then

the resulting Markov chain is finite with N states. Immediately after a service

completion there can be at most N -1 customers in the queue, so that the imbedded

Markov chain has the state space SS = [0, 1, 2, ..., N - 1 customers] and the

transition matrix:

P =

Si

S2

S3

S4

S5

SN-1

SN

SI

bo

bo 0

0

0

0

0

S2

bi

bi

bo 0

0

0

0

S3

b2

b2

bl

bo 0

0

0

S4

b3

b3

b2

bl

bo

0

0

Ss .

b4 .

b4 .

b3 .

b2 •

bl .

0 . .

0 . .

. SN-1

• • bN.2

• • bN-2

• • bN-B

• • bN-4

• • bN-S

• bl

. bo

SN

dN-i

dN-i

dN-2

dN-3

dN-4

d2

di

(2-92)

where dN-i = 1 - (bo + bi + ... + bN-2)-

An example demonstrating the above concepts was calculated for N = 4

states; thus, SS = [0, 1,2, 3 customers]. The corresponding matrix is the

following one where the probabilities were selected arbitrarily.

P =

Si

S2

S3

S4

Si S2 S3 S4

0.5 0.2 0.1 0.2

0.5 0.2 0.1 0.2

0 0.5 0.2 0.3

0 0 0.5 0.5

(2-93)

Page 122: Applications of Markov Chains in Chemical Engineering

116

Fig.45 shows results for two initial state vectors S(0). On the left hand-side

the system is initially at Si, i.e., there are no customer in the queue. On the right

hand-side the system is initially at S4, i.e., there are three customers in the queue.

The propagation of the probability distribution indicates that the system (queue)

reaches a steady state independent of S(0), i.e., an ergodic Markov chain. The

calculation indicates that the steady state is achieved after nine steps and the

appropriate state vector reads S(9) = [0.211, 0.212, 0.254, 0.322], i.e., S4 with

three customers is the state of the highest probability. Certainly, if the possible

number of states N (with maximum number of customers N -1) is changed, a new

steady state would have been achieved, which depends also on the matrix, Eq.(2-

93).

S(0) = [1,0,0,0] S(0) = [0,0,0,1] 1 1

\

\ _ \ \

L ^'^ '--- -.

r / " h/ spy

1 1 -]

1 2 3 4 5 0 1 2 3 4 n n

Fig.2-45. The dynamics of a queuing process

2.1-5 Classification of states and their behavior The examples presented in section 2.1-4 indicate that the states of a Markov

chain fall into distinct types according to their limiting behavior. In addition, it is

also possible to identify from the transition matrix some characteristic behavior of

the states. Suppose that the system is initially at a given state. If the ultimate

occupation to this state is certain, the state is called recurrent, in this case the time of

first return will be called the recurrence time. If the ultimate return to the state has

probability less than unity, the state is called transient. On the basis of the

examples presented above, the following classification of states may be suggested

[4, p.91; 6, p.28; 15, p.387].

Page 123: Applications of Markov Chains in Chemical Engineering

117

Ephemeral state. A state j is called ephemeral if pij = 0 for every i.

Thus, an ephemeral state can never be reached from any other state. For example,

Si in the matrix given by Eq.(2-43) is such a state since pn = 0 for every i.

Accessible state. A state k is said to be accessible from state j if there

exists a positive integer n such that pjk(n) > 0 where pjk(n) is defined in Eq.(2-26).

For example, according to Eq.(2-49), S4 is accessible from S2 since it may be

obtained from Eqs.(2-32) to (2-34) that p24(2) = p^ > 0. However, in the same

matrix S2 is not accessible from Si since pi2(n) = 0. In Eq.(2-51), S2,..., S16

are not accessible from Si since it is an absorbing state.

Inter-communicating states. Two states j and k are said to

communicate if k is accessible from j in a finite number of transitions, i.e., if there

is an integer n such that pjk(n) > 0. We define Ujk to be the smallest integer n for

which this is true. If j is accessible from k and k is accessible from j , then j and k

are said to be inter-communicating.

As indicated before, in Eq.(2-49), state S4 is accessible from S2 since P24(2)

= p2 > 0. In fact, S2 is also accessible from S4 since it may be shown that P42(2)

= q^ > 0. Thus, S2 and S4 are inter-communicating. It may also be shown that

other states in Eq.(2-49) demonstrate the same behavior.

Irreducible chain. Perhaps the most important class of Markov chains is

the class of irreducible chains. An irreducible chain is one in which all pairs of

states communicate, i.e., pjk(n) > 0 for some integer n where j , k = 1,2,..., Z; the

latter is the number of states. In other words, every state can be reached from

every other state. The following matrix, corresponding to example 2.32

P =

Si

S2

S3

S3

Si 0

1/3

1/3

1/3

S2 1/3

0

1/3

1/3

S3 1/3

1/3

0

1/3

S4 1/3

1/3

1/3

0

satisfies the above condition since it follows from Eqs.(2-32) to (2-34) that pii(2),

P22(2), P33(2), P44(2) > 0 as well as the other pjk's. Pjj(n), defined in Eq.(2-26),

Page 124: Applications of Markov Chains in Chemical Engineering

118

is the probability of occupying Sj after n steps (or at time n) while initially

occupying also this state.

Absorbing state. State j is said to be an absorbing (dead or a trapping)

state if the occupation probability of the state by the system satisfies pjj = 1. In

other words, once a system occupies this state, it remains there forever. Examples

of such a state are: 2.8, 2.10, 2.13, 2.15, 2.17, 2.23, 2.24, 2.28, 2.34, 2.37,

2.38, 2.39, 2.40, 2.41, 2.42.

Prior to further definition of states, additional notation is established in the

following! 15, p.387]. Throughout the following chapter we will use fjj(n) as the

probability that in a process starting from Sj, the next occupation of Sj occurs

exactly at the nth step. In other words, we may say that conditional on Sj being

occupied initially, fjj(n) is the probability that Sj is avoided at steps (times) 1, 2,

..., n - 1 and re-occupied at step n. fjj(l) = pjj and for n = 2, 3,...,

fjj(n) = prob{X(r) ^ j , r = 1,..., n - 1; X(n) = j I X(0) = j} (2-94)

It is interesting to demonstrate the above concepts considering Fig.2-0 by Escher.

Assume that X(r) designates locations along the water trajectory after r steps.

Thus, X(l) = 1 is location of the system at point 1, the top of the waterfall, after

one step. Similarly, X(5) = 2 designates location at point 2 after 5 steps. On the

basis of Eq.(2-94), the following probabilities may be established, i.e.:

fll(5) = prob{X(r) 5 1, r = 1,..., 4; X(5) = 1 I X(0) = 1} = 1

fll(3) = prob{X(r) t 1, r = 1, 2; X(3) = 1 I X(0) = 1} = 0

On the basis of fjj(n) the following quantities, later applied, are defined:

Probability fy that, starting from Sj, the system will ever pass through Sj,

reads:

fjj = £fj j(n) (2-95) n=l

In other words, ^j designates the probability that Sj is eventually re-entered. The

application of Eq.(2-95) can be demonstrated on the basis of Fig.2-0 in the

following way. Instead of the perpetual motion of the water, the loop is opened at

Page 125: Applications of Markov Chains in Chemical Engineering

119

S3, point 3. A certain amount of water is then poured at Si which flows through

S2 and is leaving away at point 3, state S3. For a water element, the system,

Eq.(2-95) yields that fn = f i i( l) + fii(2) + ... = 0 + 0 + ... = 0, i.e., the

probability of the water element to re-enter at point 1 is nil. Similarly, fil = 33 =

0.

Mean recurrence time is given by

^^J=S^¥^) (2-96) n=l

where for the above example, it gives ILII = ^2 = |Lt3 = 0.

Probability fjk(n) that in a process starting from Sj, the first occupation of

Sk (or entry to Sk) occurs at the wth step, is defined in Eq.(2-97). In other words,

fjk(n) indicates that Sk is avoided at steps (times) 1,..., n - 1 and occupied exactly

at step n, given that state Sj is occupied initially. Thus, fjk(l) = Pjk and for n = 2,

3, ..., fjk(n) is give n by:

fjk(n) = prob{X(r) ; k, r = 1,..., n - 1; X(n) = k I X(0) = j} (2-97)

The application of Eq.(2-97) to Fig.2-0 yields that:

fl5(4) = prob{X(r) t 5, r = 1,..., 3; X(4) = 5 I X(0) = 1} = 1

fl5(7) = prob{X(r) ?t 5, r = 1, ..., 6; X(7) = 5 I X(0) = 1} = 0

whereas:

fl3(7) = prob{X(r) ; 3, r = 1, ..., 6; X(7) = 3 I X(0) = 1} = 1

The calculation of fjk(n) is detailed in [15, p.388]. We put fjk(O) = 0 and on the

basis of fjk(n) we define

Page 126: Applications of Markov Chains in Chemical Engineering

120

fjk=£fjk(n) (2-98) n=l

fjk is the probability that, starting from Sj, the system will ever pass through S^.

Thus, fjk ^ 1. When fjk = 1, the {fjk(n), n = 1, 2, ...} is a proper probability

distribution and we will refer to it as the first-passage distribution for Sk. In

particular, {fjj(n), n = 1, 2, ...} represents the distribution of the recurrence time

for Sj. It should be noted that the definition in Eq.(2-96) is meaningful only when

fjj = 1, that is, when a return to Sj is certain. In this case ^j < <» is the mean

recurrence time for Sj.

Considering again the case of an open loop between points 1 and 3, states S2

and S3, in Fig.2-0, and applying Eq.(2-98) for a fluid element initially at point 1,

yields:

fl2 = fl2(l) + fl2(2) + fi2(3) + ... = 1 + 0 + 0 + ... = 1

fl3 = fl3(l) + fl3(2) + fl3(3) + ... = 0 + 1 + 0 + ... = 1

i.e., the probability of the water element to pass S2, after one step and S3 after two

steps is 100%.

Transient or non-recurrent state. State j is said to be transient if the

conditional probability of occupying (or returning to) Sj, given that the system

initially occupies Sj, is less than one. Thus, the eventual return to the state is

uncertain. According to [15, pp.389], Sj is transient if, and only if:

Xpjj(n)<oo (2-99)

n=l

In this case:

Xpjk(n)<oo (2-100) n=l

for all j in the state space. On the basis of the quantity defined in Eq.(2-95), Sj is

transient if:

Page 127: Applications of Markov Chains in Chemical Engineering

121

fjj < 1 (2-101)

For the open loop between states Si and S3 corresponding to points 1 and 3

in Fig.2-0, where a certain amount of water is poured at Si, Eqs.(2-99) and (2-

100) yield

P22(l) + P22(2) + P22(3) + ... = 0 + 0 + 0 + ... < 00

P13(1) + P13(2) + P13(3) + ... = 0 + 1 + 0 + ... < 00

i.e., S2 and S3 are transient states. By applying Eqs.(2-95), it is obtained that

f22 (= 0) < 1 and f33 (= 0) < 1

results which are in agreement with Eq.(2-101) for characterizing transient states.

Recurrent or persistent state. State j is said to be recurrent if the

conditional probability of occupying Sj or returning to it, given that the system

initially occupied Sj or started in it, is one. Thus, the eventual return to the state is

certain. Taking into account Eq.(2-95), it follows that a state Sj is recurrent if:

fjj=l (2-102)

Considering Eq.(2-96), the mean recurrence time |lj = 00 2ind Sj is called null-

recurrent state or null state. If |ij is finite, Sj is defined as positive-recurrent.

The application of Eq.(2-102) can be demonstrated in the following way on

the basis of Fig.2-0, now for a closed water loop in its perpetual motion uphill.

Considering a fluid element, and applying Eq.(2-95), yields:

fll = f u d ) + fll(2) + ... + fii(5) + ... = 0 + 0 + ... + 1 + ... = 1

hi = f22(l) + f22(2) + ... + f22(5) + ... = 0 + 0 + ... + 1 + ... = 1

and similarly:

Page 128: Applications of Markov Chains in Chemical Engineering

122

f33 = UA = f55 = 1

Thus, on the basis of Eq.(2-102), the states Si (points i, i = 1, 2, ..., 5) are

recurrent states. From Eq.(2-96) it follows that:

m = ^2 = ... = ^5 = 5

i.e., each state is reoccupied after 5 steps, where the states are also positive-

recurrent considering the definition following Eq.(2-102).

Periodic state. Suppose that a chain starts in state Sj. Subsequent

occupations of Sj can only occur at steps (times) Iv, 2v, 3v, 4v ... where v is an

integer.

I f v > l (2-103)

and the chain is finite, Sj is periodic. The period of Sj is the greatest common

divisor of the set Iv, 2v, 3v, 4v ... for which Pjj(n) > 0 where n is an integral

multiple of v. In the absence of the latter, pjj(n) = 0.

If v = l (2-104)

and the chain is finite, Sj is called an aperiodic Markov chain. The above concepts

are demonstrated on the basis of Fig.2-0 by considering point 2, state S2.

Occupation of this state occurs at steps 5, 10, 15, 20, ..., hence, Iv = 5, 2v = 10,

3v = 15, 4v = 20,..., yielding that v = 5. From Eq.(2-103) S2 is periodic with a

period of 5. It should be noted that the same results applies also for the other states

Si, S3, S4 and S5.

An additional example is 2.9 for which the transition probability reads:

P =

Si

S2

0

1

1

0

Page 129: Applications of Markov Chains in Chemical Engineering

From Eq.(2-33a), it follows that:

123

P2 =

Si

S2

1

0

^2 0

1

= p4 = p6 = p8 = plO

Hence, Iv = 2, 2v = 4, 3v = 6, 4v = 8, 5v = 10, thus v = 2, i.e.. Si and S2 have a

period of 2.

Other examples showing periodic behavior are: Example 2.11 with a 7x7

matrix showing a period of v = 7 in Eq.(2-39a); Example 2.14 with a 6x6 matrix

and a period of v = 2 for states S2 and S3 in Eq.(2-42a) where the probability of

the other states vanishes; Example 2.16 with a 12x12 matrix and a period of v = 2

for all states demonstrated in Fig.2-9; Example 2.18 with a 9x9 matrix and a period

of V = 2 for all states depicted in Fig.2-12; Example 2.21 with a 9x9 matrix and a

period of v = 2 shown in Fig.2-15; Example 2.27 with a 4x4 matrix and a period

of V = 2 for all states demonstrated in Fig.2-23; Example 2.35 with a 4x4 matrix

and a period of v = 2 for all states depicted in Figs.2-31, 32; Example 2.45 case 1

with a 8x8 matrix and a period of v = 4 as demonstrated in Fig.2-42.

The following may be shown:

a) A Markov chain is aperiodic if for a state Sj there exist pjj = Pjj(l). Thus, if any

of the diagonal elements in P is non zero, the chain is aperiodic.

b) A Markov chain is aperiodic if there exists an integer n such that pjk(n) > 0 for

allj andk.

The following examples demonstrate the above behaviors:

If

P =

Si

S2

S3

Si

0

0

1

S2 S3

1 0

1/2 1/2

0 0

the chain is aperiodic, since P22 = 1/2 > 0.

Page 130: Applications of Markov Chains in Chemical Engineering

124

If

P =

Si

S2

S3

Si S2 S3

0 1/2 1/2

1/2 0 1/2

1/2 1/2 0

the chain is aperiodic, since

P2 = Si

S2

S3

Si

1/2

1/4

1/4

S2 1/4

1/2

1/4

S3 1/4

1/4

1/2

and Pjk(2) > 0 for all j and k. Other examples exhibiting aperiodic behavior may be

found among examples 2.8-2.46.

Ergodic state. A finite Markov chain is ergodic if there exist probabiUties

TCk such that [6, p.41; 4, p.lOl]:

lim Pjk(n) = % for all j and k (2-105)

n—>oo

These limiting probabilities TCk are the probabilities of being in a state after

equilibrium has been achieved. As can be seen from Eq.(2-105), the TCkare

independent of the initial state j , i.e., they are without memory to the past history.

Similarly, from Eq.(2-25), we may write that:

,n+l l imS(n-hl) = limS(0)P"^' = 7C n-»oo n^oo

where n is the stationary distribution of the limiting state vector. Thus:

TCP = 7C (2-105a)

Hence, if the system starts with the distribution n over states, the distribution over

states for all subsequent times is n. This is the defining property of a stationary

Page 131: Applications of Markov Chains in Chemical Engineering

125

distribution. Ergodic systems retain the property of having unique equilibrium

distribution and these are also unique stationary distributions.

The limiting probabilities TCk may be found by solving the following system

of equations:

z fork= 1,2, ...,Z (2-106)

subject to the conditions:

n^>0 for all k

z

X^k= (2-107)

1 k=l

As indicated before, the probability distribution {TCk} defined in Eqs.(2-106)

and (2-107) is called a stationary distribution. If a Markov chain is ergodic, it can

be shown [17, pp.247-255] that it possesses a unique stationary distribution; that

is, there exist 7Ck that satisfy Eqs.(2-105), (2-106) and (2-107). There are Markov

chains, however, that possess distributions that satisfy Eqs.(2-106) and (2-107),

i.e., they have stationary distributions, which are not ergodic. For example, if the

probability transition matrix is given by:

P =

Si

S2

Si

0

1

S2

1

0

then:

Pll (n)={ 1 if n is even

0 if n is odd

Page 132: Applications of Markov Chains in Chemical Engineering

126

so the chain is not ergodic. However, one may solve Eqs.(2-106) and (2-107) to

obtain the stationary probabilities Tii and 7t2 = 1/2. Recall that this P is the

transition matrix for an irreducible periodic Markov chain.

Some sufficient conditions for a finite Markov chain to be ergodic are based

on the following theorems, given without proof [17, pp.247-255]. The first one

states that: A finite irreducible aperiodic Markov chain is ergodic. Let:

P =

Si

S2

S3

Si

1/4

0

3/4

S2 1/4

2/3

1/4

S3 1/2

1/3

0

This chain is irreducible (all pairs of states communicate) since pjk(2) > 0 for all j ,

k. It is aperiodic since p n = 1/4 > 0. Hence by above theorem the chain is

ergodic. To find the limiting probabilities, solve Eqs.(2-106) for Z = 3 to obtain

the following equations:

TCi = (l/4)7Ci + (3/4)7C3; 7C2 = (l/4)7Ci + (2/3)7l2 + (1/4)7^3; 7C3 = (l/2)7Ci + (1/3)7C2

The solution of these equations is TC] = 2/7, 7i2 = 3/7, 71:3 = 2/7. Thus, the

asymptotic probability of being in state Si is 2/7, in S2 is 3/7 and in S3 is 2/7.

The following Table summarizes part of the classification of the above states:

Table 2-2. Classification of states (4, p.93)

Type of state

Periodic

Aperiodic

Recurrent

Transient

Positive-recurrent

Null-recurrent

1 Ergodic

Definition of state

(assuming it is initially occupied)

Return to state is possible only at times Iv, 2v, 3v,

..., where the period v > 1

Not periodic. Essentially it has a period of v = 1

Eventual return to state is certain

Eventual retum to state is uncertain

Recurrent, finite mean recurrence time

Recurrent, infinite mean recurrence time, |iij = 00

I Aperiodic, positive-recurrent, |ij < 00

Page 133: Applications of Markov Chains in Chemical Engineering

127

Doubly stochastic matrix. A transition probability matrix is said to be

doubly stochastic if each column sums to 1, that is, if

7,Pi], = 1 for each k (2-108)

On the basis of the following theorem, i.e., if the transition matrix P for a

finite irreducible aperiodic Markov chain with Z states is doubly stochastic^ then the

stationary probabilities are given by

Ttk = 1/Z fork= 12, ..., Z

it follows that the transition matrix

Si

S2

S3

S3

Si

1/8

1/2

3/8

0

S2 1/4

1/4

1/4

1/4

S3 1/8

1/8

1/4

1/2

S4

1/2 1 1/8

1/8

1/4

p =

is irreducible, aperiodic, and doubly stochastic. Hence Tlk = 1/4 for k = 1, 2, 3, 4.

Definition. Closed sets of states. A non empty set C of states is called a

closed set if each state in C communicates only with other states in C. In other

words, no state outside C can be reached from any state in C. A single state Sk

forming a closed set is an absorbing state. Once a closed set is entered it is never

vacated. If j belongs to a closed set C then pjk = 0 for all k outside C. Hence, if all

rows and columns of P corresponding to states outside C are deleted from P, we

are still left with a stochastic matrix obeying Eqs.(2-17) and (2-18). A Markov

chain is irreducible if there exists no closed set other than the set of all states.

According to [17, p.210], a closed communicating class C of states

essentially constitutes a Markov chain which can be extracted and studied

independently. If one writes the transition probability matrix P of a Markov chain

so that the states in C are written first, and P can be written as:

Page 134: Applications of Markov Chains in Chemical Engineering

128

P = (2-109)

where * denotes a possibly non-zero matrix entry and p^ is the sub matrix of P

giving the transition probabilities for the states in C, then:

pn =

(Pc) 0

(2-109a)

Thus, if a Markov chain consists of one or more closed sets, then these sets are

sub-Markov chains andean be studied independently. The following example

demonstrates the above ideas, i.e.:

P =

Si $2 S3 S4 S5 S6 S7 Sg S9

Si

S2

S3

S4

S5

S6 S7

Sg S9

0

0

0

1

0

0 0

0 0

0

Y 0

0

0

1 (0

0 0

0

5

0

0

0

0 0

1 0

a 0

0

0

0

0 0

0 V

0

e 0

0

1

0 0

0 0

0

0

0

0

0

0

I 0 0

0

0

0

0

0

0 X

0 0

0

0

1

0

0

0 0

0 0

p X

0

0

0

0 0

0

^

(2-110)

In order to find the closed sets, it suffices to know which pjk vanishes and

which are positive. In the fifth row in matrix (2-110) P55 = 1, thus S5 is

absorbing. S3 and Sg form a closed set since p38 = pgs = 1. From Si, passages

are possible into S4 and S9, and from there only to Si, S4, S9. Accordingly the

latter states form another closed set. The appearance of the matrix and the

determination of the closed sets can be simplified by renumbering the states in the

order S5 S3 Sg Si S4 S9 S2 S6 S7 so that the modified matrix reads:

Page 135: Applications of Markov Chains in Chemical Engineering

129

P =

S5 S3 Sg Si S4 S9 S2 S6 S7

S5

S3

Sg

Si

S4

S9

S2

S6 S7

1

0

0

0

0

0

e 0 0

0

0

1

0

0

1

6 0 0

0 1

0

0 0

0

0

0

0

0

0

0

0 0

0 0 0 0

0 0

0

a 1

V

0

0 0

0 0

0

p 0

H X

0

0

0 0

0

0

0

0

Y 1

(0

0 0

0

0

0

0

0

0

^

0

0

0

0

0

0

0

0 X

(2-110a)

The closed sets then contain only adjacent states and a glance at the new matrix

reveals the grouping of the states. In addition, the sub matrices whose states form

closed sets may be studied separately and on the basis of Eqs.(2-109) and (2-

109a), matrix (2-110a) reads:

(p)" =

S5

S3

Sg

Si

S4

S9

S2

S6 S7

S5 1

0

0

0

0

0

e 0 0

S3 0

0

1

0

0

1

5

0 0

Sg Si 0 0

1

0

" 0

0

0

0

0

0

0

0

0 0

0 0 0 0

S4 0

0

0

a 1

V

0

0 0

S9 0

0

0

p 0

1 X

0 0

S2 0

0

0

" 0

0

0

Y 1 CO

S6 0

0

0

0

0

0

0

0

^

S7 0

0

0

0

0

0

0

0 X

(2-110b)

Examples

To conclude section 2.1-5 on classification of states, we consider the

following matrix corresponding to five states which belongs to the random walk

model considered in examples 2.17-2.22, 2.41 and 2.42.

Page 136: Applications of Markov Chains in Chemical Engineering

130

P =

Si

S2

S3

S4

Ss

Si

P

q 0

0

0

S2 1-p

r

q 0

0

S3 0

p r

q 0

S4 0

0

p r

q

S5 0

0

0

p 1-q

By changing the parameters p, q and r, many behaviors are revealed depicted in

Fig.2-46. It should be noted that examination of the various behaviors has been

done with respect to the state vector S(n) rather than exploration of the quantities

Pij(n) referred to in part of the above definitions. Eqs.(2-23) and (2-25) were used

in the calculation of S(n) and an initial state vector S(0) = [0, 0, 1,0, 0] was

assumed, i.e., S3(0) = 1.

Case a in Fig.2-46, for which q = 0, p = 1 and r = 0, demonstrates that S5

is an absorbing state which is occupied after two steps. Thus, the probability of

occupying the other states vanishes. Si, which is also an absorbing state, can

never be occupied unless it is initially occupied.

Case b where q = 1, p = 0 and r = 0, has a reflecting barrier at the origin.

Fig.2-46 demonstrates 2i periodic behavior of states S] and S2. For the other states

si(n) = S2(n) = 0, n = 1, 2,...; S3(n > 1) = 0. The period equals v = 2.

Case c for which p > q (q = 0.3, p = 0.7 and r = 0) reveals a transient

behavior of the states as well as their being ergodic, i.e., independent of S(0). The

stationary distribution reads S(25) = [0.045, 0.045, 0.104, 0.242, 0.564].

Case rf, also for p > q (q = 0.1, p = 0.7 and r = 0.2), reveals a similar

behavior as in the previous case. However, the approach towards equilibrium is

less oscillatory by comparison to case c due to the damping parameter r > 0, i.e.,

the probability of remaining in the state. The limiting distribution reads S(24) =

[0.001, 0.003, 0.018, 0.122, 0.856].

Case ^, for p = q = 0.5 is transient with damped oscillations towards an

equilibrium of S(32) = [0.2, 0.2, 0.2, 0.2, 0.2], i.e., the occupation probability of

all states is the same. The states are also ergodic.

Page 137: Applications of Markov Chains in Chemical Engineering

131

Casef with q = 0.2, p = 0.2 and r = 0.6 is transient, free of oscillations as

well as ergodic. The equilibrium state vector reads S(46) = [0.058, 0.236, 0.236,

0.235, 0.235].

Cases g and h with q = 0.8, p = 0.2, r = 0 and q = 0.4, p = 0.2, r = 0.4,

respectively, behave similar to cases e and f, i.e., they are transient and ergodic.

Note that p < q. The ultimate distributions are: S(65) = [0.430, 0.430, 0.107,

0.027, 0.006] for case g and S(20) = [0.210, 0.420, 0.211, 0.106, 0.053] for

case h.

1

0.8

0.4

0.2

0

q = 0,p = 1,r = 0

^.3(") >

•/f.V-'"' s.(n) = s(n)

q=1,p = 0,r = 0

U M") / s,(n) / W i ) /

\ /

^ ^iTi'i 5 0 1

b) 2 3

n q =

~\ ^ s (n)

\L\ • J ^

0.1,p = 0.7, r = 0.2 ' s (n) ' 1

s,(n) 1

15 0 5 10 d) n

15

Page 138: Applications of Markov Chains in Chemical Engineering

132

1

0.8

0.2

0

q = 0.5, p = 0.5, r = 0

s (n)

I/' \^' 2(n) = s (n)

\:J im

s (n) = s (n)

0 5 10 e) n

q = 0.8, p = 0.2, r = 0

q = 0.2, p = \ 1

^ s (n)

0 . 2 , r = 0 . 6

1 1

V , - - - - ' • ! 1 15 0 5 10

f) n q = 0.4, p = 0.2, r = 0.4

15

Fig.2-46. Demonstration of states behavior for S(0) = [0, 0, 1, 0, 0]

2.2 MARKOV CHAINS DISCRETE IN SPACE AND CONTINUOUS IN TIME 2.2-1 Introduction

In the preceding chapter Markov chains has been dealt with as processes

discrete in space and time. These processes involve countably many states Si, S2,

... and depend on a discrete time parameter, that is, changes occur at fixed steps n

= 0, 1, 2, ....

Although this book is mainly concerned with the above mode of Markov

chains, a rather concise presentation will be given in the following on Markov

chains discrete in state space and continuous in time [2, p.57; 4, p. 146; 5, p. 102;

15, p.444]. This is because many processes are associated with this mode, such as

telephone calls, radioactive disintegration and chromosome breakages where

changes, discrete in nature, may occur at any time. In other words, we shall be

concerned with stochastic processes involving only countably many states but

Page 139: Applications of Markov Chains in Chemical Engineering

133

depending on a continuous time parameter. Such processes are also referred to as

discontinuous processes. Unfortunately, application of the basic models presented

in 2.2-3 to chemical reactions, behavior of chemical reactors with respect to RTD,

as well as to chemical processes, is very limited or even inapplicable.

While the distinction between discrete time and continuous time is

mathematically clear-cut, we may in applied work use discrete time approximation

to a continuous time phenomena and vice versa. However, discrete time models

are usually easier for numerical analysis, whereas simple analytical solutions are

more likely to emerge in continuous time.

In the following, we derive the Kolmogorov differential equation on the basis

of a simple model and report its various versions. In principle, this equation gives

the rate at which a certain state is occupied by the system at a certain time. This

equation is of a fundamental importance to obtain models discrete in space and

continuous in time. The models, later discussed, are: the Poisson Process, the

Pure Birth Process, the Polya Process, the Simple Death Process and the Birth-

and-Death Process. In section 2.1-3 this equation, i.e. Eq.2-30, has been derived

for Markov chains discrete in space and time.

2.2-2 The Kolmogorov differential equation The following model represents a simplified version of the Kolmogorov

equation. A big state of a total population NQ consists of many cities. The cities

are arranged in two circles, internal and external. The external circle consists of J

cities, j = 1, 2,..., J, where each city is a state designated as Sj = j =jth city in the

external circle. The internal circle consists of K cities, k = 1, 2,..., K. A state in

this circle is designated by Sk = k = kth city in the internal circle. Inhabitants,

designated as system, are moving from cities in the external circle to cities in the

internal circle, i.e., along the trajectory Sj —> S| -> out of kth city, where from each

external city, inhabitants can move only to each internal city. In other words

inhabitants can not move from one external city to another extemal city.

The movement of inhabitants through an internal city was convincingly

demonstrated by Magritte [20] in his painting Golconda depicted in Fig.2-47.

Page 140: Applications of Markov Chains in Chemical Engineering

134

4|( M i l M l I I I It I l '

l i l t I l t i ^ 4 / It | | | | i I ^M I i I I flBl ^ 1 I H I i

in 1 I ( f V M r t ( ^

Fig.2-47. Motion of inhabitants through an internal city k according to

Magri t te ("Golconda ", 1953, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

The figure has slightly been modified by adding a circle on it, demonstrating

a city, as well as arrows indicating 'in' and 'out' from city k. The painting is one

of Magritte's most scrupulous displays of reordering where he allows his figures

no occupation, no purpose, and despite of their rigid formation, no fixed point.

They could be moving up or down or not moving at all.

In setting up the model, we designate by Nj(t) the number of inhabitants

occupying state j (an extemal city j) at time t. Similarly, Nk(t) corresponds to the

number of inhabitants occupying state k (an internal city k) at time t. The change in

the number of inhabitants in state Sk during time interval At is given by:

Page 141: Applications of Markov Chains in Chemical Engineering

135

ANk(t) = number of inhabitants entering the city in the time interval (t, t + At)

- number of inhabitants leaving the city in the interval (t, t + At)

(2-111)

The quantities on the right-hand side may be expressed by the following

probabilities [2, p.59]:

qj(t) - a rate (1/time) or an intensity function indicating the rate at which

inhabitants leave state Sj (external city j). This function has the following

interpretation, i.e.,

qj(t)At - stands for the probability of the system (inhabitants) to leave state Sj

in the time interval (t, t + At). This quantity is the probability for a change to occur

at Sj not indicating the 'direction' of the change. The following quantity gives the

'direction', i.e., the probability that inhabitants leaving Sj will occupy exactly S^,

namely, they will occupy city k in the internal circle. Thus,

Qjk(0 - gives the direction of the change at the interval (t, t + At), i.e., the

transition probability of the inhabitants from Sj to occupy S^ at time t [15, p.473].

Note that 0<Qjk(t), Qkk(t)< 1.

Other quantities pertaining to the cities in the intemal circle are:

qk(t) - the rate (1/time) at which inhabitants {system) leave state Sk (intemal

cityk). Thus,

qk(t)At - is the probability of the inhabitants to leave Sk at the interval (t, t +

At).

Accounting for the above quantities and applying Eq.(2-111) yields:

J

ANk(t) = X Nj(t)qj(t)AtQjk(t) - Nk(t)qk(t)At (2-11 la)

Defining the following state probabilities, i.e.,

Pk(0 - probability of the system to occupy state Sk at time t,

Pj(t) - probability of the system to occupy Sj at time t

and assuming that the total number of inhabitants NQ in the big state remains

constant, gives

Nj(t) = Pj(t)No; Nk(t) = Pk(t)No (2-112)

Page 142: Applications of Markov Chains in Chemical Engineering

136

Substitution of Eq.(2-112) into Eq.(2-llla) and approaching At to zero, yields a

simplified version of the forward Kolmogorov differential equation for the

transition S; -> S|j~> out of kih city. This equation is continuous in time and

discrete in space; it reads:

dPk(t) ^ - ^ = XPj ^^^J^^^QjkW - Pk(t)qk(t) k = 1, 2, ..., K (2-113)

j=i

It should be noted that Eq.(2-113) may be looked upon as an "unsteady state

probability balance" on city k in the internal circle; J is the total number of cities in

the external circle and K in the internal circle. In other words, the equation gives

the rate of change of the probability of occupying state k at a certain time. Note that

the origin of the equation was an unsteady state "mass balance" on the transition of

inhabitants through city k.

Another version of the Kolmogorov equation is obtained by considering the transition S; -» Sj -» \ . Eq.(2-30) gives the Chapman-Kolmogorov equation,

discrete in time and space. For a continuous time and discrete space, this equation

is generally written as [2, p.61]:

PjkCT 't) = ^Pji(T,s)Pik(s,t) for t > T > 0 (2-114)

and is valid for x < s < t. This relation expresses the fact that a transition from state

Sj at time x to Sk at time t occurs via some state Sj at the intermediate time s, and for

a Markov processes the probability Pik(s,t) of the transition from Si to Sk is

independent of the previous state Sj. Pjk('C,t) is the transition probability of a

system to occupy state k at time t subjected to the fact that the system occupied state

j at time T. Another way of expressing the above is that we shall write Pjk(x,t) for

the conditional probability of finding the system at time t in state Sk, given that at a

previous time T the system occupied Sj. As indicated above, the symbol Pjk('C,t) is

meaningless unless x < t. Differentiation of Eq.(2-114) [15, p.472] yields the

forward Kolmogorov differential equation for the transition S: -> S, —> Sj . It is

continuous in time and discrete in space and reads:

Page 143: Applications of Markov Chains in Chemical Engineering

137

dt = 2 PjiC- OqiCOQikCt) - Pik(T,t)qk(t) (2-115)

Here j and T are fixed so that we have, despite of the formal appearance of the

partial derivative, a system of ordinary differential equations for the function

Pjk(T,t). The parameters] and T appear only in the initial conditions, i.e.:

1 forj = k Pjk(T,T)= (2-115a)

0 otherwise

Thus, the system of ordinary differential equations reads:

^ ^ = X Pi(t)qi(t)Qik(t) - Pk(t)qk(t) (2-116) i

The above equation becomes identical to Eq.(2-113) by replacing the notation of the

state transitions, i.e. i with j .

As Pjk('Tjt) stands in Eq.(2-115), it is not time-homogeneous, since it

depends explicitly on t and x. However, if a restriction is made to the time-

homogeneous or stationary case, then:

Pjk(t,t) = Pjk(t-T) (2-117)

i.e., the transition probability Pjk(T,t) depends only on the duration of the time

interval (t - x) and not on the initial time x. Hence, in the time-homogeneous case,

Eq.(2-114) reads:

PjkC + t^^XV^^Pik^^^ fort>T>0 (2-118) i

The significance of Eq.(2-118) is as follows. Given that at time = 0 a system is at

state Sj = j . If we ask about its state at time (x + t), then the probability that the

system will occupy state k at time (T +1), i.e., Pjk(T + t), may be computed from

the above sunmiation based on the fact that the system will occupy an intermediate

Page 144: Applications of Markov Chains in Chemical Engineering

138

state Si = i before a transition to occupy Sk takes place. Pji(T) is the probability of

the system to occupy Si at time x and Pik(t) is the probability of occupying Sk at

time (T +1).

2.2-3 Some discontinuous models The following models for the transition S; -^ S^ are considered because they

find applications in many comers of our life, later elaborated. It should be noted

that the above transition indicates the basic property of Markov chains, i.e. that

occupation of Sk is conditioned on a prior occupation of Sj and that the past history

is irrelevant. Unfortunately, the application of the models in Chemical Engineering

is very limited, however, some applications are mentioned. For each case we

derive the difference equation describing the probability law of the process and

report the final solution of the differential equation accompanied by its graphical

presentation. In some cases we derive the differential equation from the

Kolmogorov equation.

The Poisson Process. This process is the simplest of the discontinuous

processes which occupies a unique position in the theory of probability and has

found many applications in biology, physics, and telephone engineering. In

physics, the random emission of electrons from the filament of a vacuum tube, or

from a photosensitive substance under influence of light, and the spontaneous

decomposition of radioactive atomic nuclei, lead to phenomena obeying the Poisson

probability law. This law arises frequently in the field of operations research and

management science, since demands for service, whether upon the cashiers or

salesmen of a department store, the stock clerk of a factory, the runways of an

airport, the cargo-handling facilities of a port, the maintenance man of a machine

shop, and the trunk lines of a telephone exchange, and also the rate at which service

is rendered, often lead to random phenomena either exactly or approximately

obeying the Poisson probability law. Part of the above examples are elaborated in

the light of the governing equation describing the law. Finally, it should also be

noted that the Poisson model obeys the Markov chains fundamental property, i.e.,

that future development depends only on the present state, but not on the past

history of the process or the manner in which the present state has been reached.

Page 145: Applications of Markov Chains in Chemical Engineering

139

In deriving the Poisson model, a rather general description arising from the

above applications can be established. We assume that events of a given kind occur

randomly in the course of time. For example, we can think on "service calls" as

(requests for service) arriving randomly at some "server" (service facility) as

events, like inquiries at an information desk, arriving of motorists at a gas station,

telephone calls at an exchange, or emission of electrons from the filament of a

vacuum tube.

Let X(t) be a random variable designating the number of events occurring

during the time interval (0, t). An interesting question regarding to the random

variable X(t) may be presented as what is the probability that the number of events

occurring during the time interval (0, t) = t will be equal to some prescribed value

X. Mathematically it is presented by:

Px(t) = prob{X(t) = X}; X = 0, 1, 2,... (2-119)

where the exact relationship is derived in the following. The above equation

indicates also the realization of the random variable X(t) by acquiring the value x. An expression for Px(t) for the transition S; -^ \ , where the two discrete

states are Sj = X - 1 and Sk = x, may be derived bearing in mind the following

assumptions:

a) The events are independent of one another; more exactly, the random

variables X(ti), X(t2),... are independent of each other if the intervals ti, t2,... are

non-overlapping. In other words, if for example t] = t2 then X(ti) = X(t2). X(ti)

designates the number of events occurring during the time interval (0, ti).

b) The flow of events is stationary, i.e., the distribution of the random

variable X(t) depends only on the length of the interval t and not on the time of its

occurrence.

c) The probability of a change in the time interval (t, t + At), or of a transition

from Sj to Sk in the time interval (t, t + At), or the probability that at least one event

occurs in a small time interval At, is given by:

prob{X(t, t + At) = 1} = U t + o(At) = pjk (2-120)

Page 146: Applications of Markov Chains in Chemical Engineering

140

where X,(events/time) is a positive parameter characterizing the rate (or density or

intensity) of occurrence of the events. A possible interpretation of the above

definition, later elaborated, makes use of the conception birth, i.e., the occurrence

of an event in the time interval (t, t + At) may be looked upon as a single birth.

Thus, the parameter X is the birth rate. If we approach At to zero, no change

occurs. Here o(At) is an infinitesimal of a higher order than At, i.e.:

lim—r— = 0 At

At~>0

o(At) emerges from the expansion pjk(At) = X,o + X.At + Xi^X^ + — The second

term on the right-hand side is responsible for the probability of the occurrence of at

least one event where the third term accounts for more than one event to occur

during At, i.e., "twin birth". XQ should be omitted in the expansion noting that the

probability of the number of events per unit time, i.e., pjk(At)/At approaches

infinity as At is approaching to zero.

d) The probability of no change in (t, t + At), or of remaining in Sj, or that no

events occur during At, is given by:

prob{X(t, t + At) = 0} = 1 - >.At + o(At) = pjj (2-121)

e) The probability of more than one change in the interval (t, t + At) is o(At),

thus it is negligible as At is approaching zero. In other words, this assumption

excludes the possibility of a "twin birth". It should be noted that the above

probabilities are independent of the state of the system.

Having established the one-step transition probabilities pjk and pjj, the

differential equation for Px(t) will be derived by setting up an appropriate

expression for Px(t + At). If the system occupies state Sj = x -1 at time t, then the

probability of occupying Sk = x, i.e., making the transition Sj to Sk, is equal to the

product A,At Px-i(t). If the system already occupies state Sk = x at time t, then the

probability of remaining in this state at (t, t + At) is equal to (1 - AAt )Px(t). Thus,

since the above transition probabilities are independent of each other, and following

Eq.(2-3), we may write that:

Page 147: Applications of Markov Chains in Chemical Engineering

141

Px(t + At) = (1 - >-At)Px(t) + >^t Px-i(t) + o(At) (2-122)

If we transpose the term Px(t) on the right-hand side, divide by At, and approach At

to zero, we obtain the following differential equation:

dPJx) - i - = -?iP,(t) + ?iP,.i(t) X > 1 (2-123)

When X = 0, Px-i(t) = 0; hence:

dPo(t)

dt = -?iPo(t) (2-124)

Eqs.(2-122) and (2-123) characterize the Poisson process and are to be solved with

the initial conditions:

Po(0)=l

Px(0) = 0 forx= 1,2,... (2-125)

Having obtained Po(t), and using Eq.(2-123), it is possible to obtain by induction

[2, p.74] that:

P (t) = Mle-^^; x = 0,1,2,... (2-126)

which is called the Poisson distribution. Some interpretations of Px(t) are: a) It

gives the probability that at time interval (0, t) = t (> 0) the system occupies state x

(x = 0, 1,2, ...). b) It is also the probability of exactly x changes or events

occurring during the time interval of length t. c) Px(t) indicates the probability to

remain at a prescribed state x during the time interval t.

Let us analyze in more details some examples, assuming they obey the

Poisson model:

1) Requests for service arriving randomly at some single service facility,

arrive at a mean rate X calls per time. If the calls have not yet been answered by the

Page 148: Applications of Markov Chains in Chemical Engineering

142

service facility, Eq.(2-126) gives an answer to the following possible questions

which might be of some interest: a) What is the probability that during a prescribed

time interval (0, t), the number of the arriving calls will be equal to a certain value

X? or, b) How long will it take until a call is answered if the probability of

remaining at a certain number x of "waiting calls" is prescribed?

2) A new cemetery of a known size of x graves has been opened at some

town. The mean death rate X in this town is known and the following questions

may arise: a) What is the probability that during the next ten years, i.e., time

interval (0, t) = 10, the cemetery will be full? b) If the probability of occupying

50% of the cemetery is known, how long will it take to reach this state? c) If the

probability to reach a certain time in the future is known, Eq.(2~126) gives an

answer to the occupation state of the cemetery.

3) It has been announced that some urban area became polluted and

consequently men might become infertile. If the process of becoming infertile is

random, thus, Poissonic, a mean parameter X may be defined as the rate of

infertility per day (say!), and Eq.(2-126) becomes applicable yielding the following

information: a) What is the probabiUty that all men in town become infertile during

a week, a month, a year etc.? b) If we prescribe the probability of remaining at

some state x (= number of men which be came infertile), then we may calculate the

time interval of remaining in this state. Certainly, the most important data needed is

X.

4) Electrons are emitted randomly from the filament of a vacuum tube. If the

emission rate, X electron/unit time, is known, then Eq.(2-126) gives the probability

that within the time interval (0, t) the total number of electrons emitted is a

prescribed value x.

5) A mailbox, designated as system, has a finite capacity for letters. The

number of letters is the state of the system x = 0, 1, .... The rate of filling of the

mailbox is X letters per day and Eq.(2-126) gives an answer to several questions of

the type demonstrated above.

Some characteristic properties of the Poisson equation are:

Page 149: Applications of Markov Chains in Chemical Engineering

143

a) Fig.2-48 demonstrates the Px(Xt) - Xi, Px(> t) - x relationships computed from

Eq.(2-126).

0 2

Fig.2-48. The Poisson model

Characteristic behavior observed is:

Px = o(0) = 1, Px>o(0) = 0 and Px^o(oo) = 0 (2-126a)

b) 2 ) Px(t) = 1 (2-127)

x-O

c) The mean number of events, m(t), occurring in the time interval of length t is

given by:

m(t).2)^Px(t) = ?t (2-128) x«0

It should be noted that for the Poisson distribution, the variance is equal to the

mean and as'ki^' co the distribution tends to normality.

d) The mean time until the occurrence of the first event, < t >, reads:

< t > = l / X (2-129)

where K is the mean rate of occurence of the events.

Page 150: Applications of Markov Chains in Chemical Engineering

144

It is interesting to demonstrate the derivation of Eq.(2-123) from the

Kolmogorov Eq.(2-113). In this equation J = K = 1 as well as:

k = Sk = Sk=i=x

j = Sj = Sj=i = x-1

qj(t) = qj=i(t) = X, Pj(t) = Px.i(t) and Qjk(t) = 1

qk(t) = qk=l(t) = >. and Pk(t) = Px(t)

Substitution of the above quantities into Eq.(2-113) gives Eq.(2-123).

Few applications of the Poisson Distribution in Chemical Engineering are:

For X = 0, Eq.(2-126) reduces to:

Po(t) = e-> t (2-130)

where Po(t) stands for the probability of remaining at the state x = 0, i.e., no

change (or event) is expected to occur in the system during the time interval of

length t. The above equation is applicable in the following cases:

a) 1st order chemical reaction, A -> B, for which -dCA/dt = ICCA occurring

in a batch reactor. The solution for the concentration distribution reads:

7 ^ - = l f - - = e-k (2-131)

where CA(t) and CAQ are, respectively, the concentration of A at time t and t = 0. k

is the reaction rate constant. Similarly, NA(t) and NAG are the number of moles of

A at time t and t = 0. The system in this case is a fluid element containing the

species A and B. The states of the system in Eq.(2-126) are: x = 0 designating

species A and x = 1 for species B. As observed, Eqs.(2-130) and (2-131) are

similar, i.e.:

Po(t) = e-kt (2-132)

therefore, the ratio NA(t)/NAO = Po(t) may be looked upon as the probability that

the system remains at x = 0 until time t.

If t = 0, it is obtained that Po(0) = 1, indicating that the probability of

remaining at the initial state at t = 0 is 100%.

Page 151: Applications of Markov Chains in Chemical Engineering

145

If k = 0, Eq.(2-132) yields Po(t) = 1, namely, in the absence of a chemical

reaction, the probability that the system remains in its initial state x = 0 for all t is

100%. If k ^ oo, it is obtained from Eq.(2-132) that Po(t) = 0, namely, the

probability of remaining at the initial state for all t is zero. In other words, in the

presence of an intensive reaction, the initial number of moles will immediately

diminish, i.e., the system will immediately occupy the state x = 1, i.e. in the state

of species B.

If Po(t) = 0.5, this means that the probability of remaining in x = 0 until time t

is 50%. In other words, 50% of A will decompose to B because the probabiUty of

remaining in the initial state until time t is 50%.

b) An additional example is concerned with the introduction of a pulse of an

initial concentration CAO into a single continuous perfectly-mixed reactor, the

system. The states of the system are x = 0, designating the initial concentration

CAO» and X = 1, the concentration of species A at time t, i.e. CACO. The

relationship CACO is given by:

- j ^ = e-^t„ = Po(t) (2-133) ^AO

where tm is the mean residence time of the fluid in the reactor. Thus, If tm -> <», Eq.(2-133) yields Po(t) = 1. This means that the probability of

the pulse to remain at the initial state x = 0 along the time interval t is 100% because

the residence time of the fluid in the reactor is infinity. If tm -» 0, Po(t) = 0. This means that the probability of remaining at x = 0 at

t > tm is zero because of the extremely short mean residence time of the fluid.

c) The following example is concerned with a closed recirculation system

consisting of N perfectly-mixed reactors of identical volumes. If we introduce a

pulse input into the first reactor, then the output signal at the Mh reactors is given

by [21, p.294]:

C = e - ^ ^ ^ J X - ^ ^ J ^ n=l ,2 , . . . ,N (2-134) n=l

Page 152: Applications of Markov Chains in Chemical Engineering

146

where C is a dimensionless concentration at the Mh reactor, tm is the mean

residence time of the fluid in a reactor which is equal for all reactors, n is the

number of passes of the fluid through the system. For short times where only one

term in the series expansion is taken into account, n = 1, Eq.(2-134) is reduced to:

^ (^W^^ ' (t/tj ^ 0^^-Xt (2-i34a) (N~l)! x! "

X = N - 1, X = l/tm where the above equation is exactly the Poisson distribution for

X = N - 1. The description about the behavior of the pulse in the previous example,

is also applicable here.

d) The final example is related to a step change in concentration at the inlet to

a single continuous perfectly-mixed reactor from CA,inlet = 0 to CA,inlet = CAQ.

The response at the exit of the reactor, designated as system, is given by:

CA(t) - p — = l - P o ( t ) (2-135)

where Po(t) is given by Eq.(2-133). CA(t) is the concentration of A inside the

reactor. The states of the system are: x = 0, i.e. CA(0) = 0 and x = 1 for CA(t) >

0. The above result can be interpreted as the probability of not remaining at the

initial state along the time interval t. For example: If tjn -^ 0, Po(t) = 0 and Eq.(2-135) gives CA(t)/CAO = 1, indicating that the

probability of not remaining at the initial state x = 0 along t is 100%. Indeed, this is

plausible because the residence time of the fluid in the reactor is extremely short and

the reactor will acquire instantaneously the final concentration CAQ, i.e, the inlet

concentration. If tm-^ oo, Eq.(2-135) gives CA(t)/CAO = 0, i.e, the probability of remaining

at the initial state x = 0 along t is 100% because of the extremely long residence

time.

The Pure Birth Process. The simplest generalization of the Poisson

process is obtained by permitting the transition probabilities to depend on the actual

state of the system. Thus, if at time t the system occupies state Sj = x (x = 0, 1,2,

Page 153: Applications of Markov Chains in Chemical Engineering

147

...), and at time (t, t + At) the system occupies state Sk = x + 1 (a single birth), then

the following probabilities may be defined:

a) The probability of the transition Sj to Sk is:

Pjk = M t + o(At) (2-136)

b) The probability of no change reads:

Pjj = 1 - >-xAt + o(At) (2-137)

where A is the mean occurrence rate of the events which is a function of the actual

state X. The dependence of X, on x avoids, in the context of birth, the phenomenon

of infertility unless X^ is a constant.

c) The probability of a transition from x to a state different from x + 1 is o(At), i.e.,

twin or multiple birth is impossible.

In view of the above assumptions and following Eq.(2-3), we may write that:

Px(t -h At) = (1 - >.xAt)Px(t) + Ax_iPx_i(t)At + o(At) (2-138)

The reason that x - 1 occurs in the coefficient of Px-i(t) is that the probability of an

event has to be taken conditional on X(t) = x - 1. If we transpose the term Px(t) on

the right-hand side, divide by At, and approach At to zero, we obtain the following

system of differential equations:

dP^(t) = -Xy?^{i) + >.x-iPx-i(t) X > 1 (2-139) dt

dPo(t) = ->.oPo(t) (2-139a)

dt

subjected to the following initial conditions:

Px(0) = 1 for X = xo

P,(0)=:Oforx>xo (^-1^9^)

Page 154: Applications of Markov Chains in Chemical Engineering

148

Depending on the A. - x relationship, two cases will be considered, viz, the linear

birth process and the consecutive-irreversible z-states process.

The linear birth process. This process, sometimes known as the Yule-Furry

process, assumes [2, p.77] that for a constant X:

X^ = Xx x > l , X , > 0 (2-140)

In this case Eq.(2-138) becomes:

Px(t + At) = (1 - ^xAt)Px(t) + X(x - l)At Px_,(t) + o(At) (2-141)

where in terms of the conception of birth, it may be interpreted as

prob{X(t + At) = x} = prob{X(t) = x and no birth occurs in (t, t + At)} +

prob{X(t) = X - 1 and one birth occurs in (t, t + At)}

The following definitions are applicable:

Px(t) = prob{X(t) = x}; 1 - UAt = prob{X(t, t + At) = 0}

Px-l(t) = prob{X(t) = X - 1}; (x - l)XM = prob{X(t, t + At) = 1}

where

X(t) is a random variable designating the population size at time t;

X or X - 1 designate the actual population size

The corresponding physical picture which may be visualized in the light of the

above, is the following one [4, p. 156; 15, p.450]. Consider a population of

members which can, by splitting or otherwise, give birth to new members but can

not die. Assume that during any short time interval of length At, each member has

probability A.At + o(At) to create a new one; the constant X births/(timexmember)

determines the rate of increase of the population. If their is no interactions among

the members and at time t the population size is x, then the probability that an

increase takes place at some time between t and t + At equals X.xAt + o(At).

It is interesting at this stage to compare the "birth characteristics" of the above

process with the Poisson one. In the Pure Birth Process, each member at each time

interval is capable of giving birth. Also each new bom member continues to give

birth and this process repeats its self ad infinitum. Thus, the birth rate is A,x. In

the Poisson process, once a member gave birth, he is becoming impotent but

remains alive, and only the new born member is giving birth and then becoming

Page 155: Applications of Markov Chains in Chemical Engineering

149

also impotent. In this case, the birth rate is X, Table 2-3 in the following is a

numerical comparison between the two process for X = Ibirth per unit time and an

initial population size of x = 1.

Table 2-3. Comparison between Pure Birth and Poisson Processes Pure Birth Process Poisson Process

X

population

at timet

1

2

3

4

5

6

7

8

9

1 10

BR = X,x

births per

unit time

1

2

3

4

5

6

7

8

9

10

At*

/

1

1/2

1/3

1/4

1/5

1/6

1/7

1/8

1/9

t*

0

1

1.50

1.83

2.08

2.28

2.45

2.59

2.72

2.83

BR = X At* t*^

births per

unit time

; /

1 ]

1 ] T 1 1

1 1 I ]

1 ]

1 ]

1

1

o | [ 1

I 2

3

[ 4

5

[ 6

[ 7

1 8

1 9J BR = birth rate; At = Ax ^t+At ^t 1

birthrate (BR)t (BR)t

* number of time units

The major conclusion drawn from Table 2-3 is that the increase of the population in the Pure Birth Process is significantly greater because the members don't become infertile after their first birth giving. For example, in the Pure Birth Process the population size becomes 10 after 2.83 time units where in the Poisson Process it takes 9 time units.

The results in the Table can also be demonstrated in Fig.2-49 by Escher's painting Metamorphose [10, p.326] modified by the author of this book. On the left-hand side of the original upper picture hexagons can be seen which make one think of the cells in a honeycomb, and so in every cell there appears a bee larva. The fully grown larvae tum into bees which fly off into space. But they are not

Page 156: Applications of Markov Chains in Chemical Engineering

150

vouchsafed a long life of freedom, for soon their black silhouettes join together to

form a background for white fish seen on the right-hand side. The modified

painting below the original one demonstrates the Poisson Process of rate ^ = 1

birth-metamorphosis per unit time corresponding to x = 1, 2, 3, 4. An interesting

question is concerned with the birth mechanism in the figure, which is beyond the

scope of this book.

Fig.2-49. Poisson process by the modiHed painting **Metamorphose**

(M.CEscher "Metamorphose" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

Returning to Eq.(2-141), the following basic system of differential equations

are obtained:

dP^(t)

dt

dPo(t)

dt

= ->.xP^(t) + ^(x - 1 )Px i (t) X > 1

= 0

(2-142)

(2-142a)

Page 157: Applications of Markov Chains in Chemical Engineering

151

where the solution reads [2, p.78]:

P^(t) = e-^Kl - e-^')''"^ for x = 1, 2,... (2-143)

= 0 for X = 0

If the initial population size is denoted by XQ and the initial conditions are PXQ(O) = 1, Px(0) = 0 for X > xo, the solution reads [16, p.450]:

^"^'^ = (x-!fp)Kxo-l)!^"^' '°^' " ^"^'^''"" X > Xo > 0 (2-143a)

The above type of process was first studied by Yule [15, p.450] in

connection with the mathematical theory of evolution. The population consists of

the species within a genus, and the creation of a new element is due to mutations.

The assumption that each species has the same probability of throwing out a new

species neglects the difference in species sizes. Since we have also neglected the

possibility that species may die out, Eq.(2-143) can be expected to give only a

crude approximation for a population with initial size of XQ = 1. Thus, if the

mutation rate X is known, the above equation gives the probability that within the

time interval (0, t) the population will remain at some prescribed state x > XQ.

Some characteristic properties of the above distribution are:

a) Fig.2-50 demonstrates the Px(^t) - Xt, Px(^t) - x relationships computed from

Eq.(2-143) with Xi as parameter. It is observed that for x > 1 and for a constant X,t,

the probability of remaining in a certain state decreases by increasing x. For x > 1,

and at a constant Xi, the probability of remaining in the state decreases as time

increases, where in general, Eq.(2-126a) is satisfied.

Page 158: Applications of Markov Chains in Chemical Engineering

152

1—I I — \ — \ — I — r

- A , t = 1

Xt = 0 , x > 0 -

j - . : i":: i " i^ 4 6

X 10

Fig.2-50. The Pure Birth model

b) Px(t) obeys Eq.(2-127). There is, however, some problem with the summation

in the equation. Eq.(2-139) indicates that the solution for Px(t) depends generally

on the value of X . Hence, it is possible for a rapid increase in the A. to lead to the

condition where:

oo

S Px(t) < 1 x=0

i.e., is a dishonest process. However, in order to comply with Eq.(2-127) for all t,

it is necessary that [2, p.81]:

i^ (2-144)

x=0

c) The mean number of events, m(t), occurring in the time interval of length t, is

defined by Eq.(2-128) where for Px(t) given in Eq.(2-143) it reads:

m(t) = ]£xP^(t) = e ^ (2-145)

x=0

The only application of Eq.(2-143) in Chemical Engineering is for x = 1, i.e.,

that during the time interval t only one event occurs with the probability Pi(t) given

by:

Page 159: Applications of Markov Chains in Chemical Engineering

153

Pi(t) = e- ^

This is the same equation obtained by the Poisson model, i.e., Eq.(2-130).

The consecutive-irreversible z-state process. In this process, encountered

in chemical engineering reactions, the system undergoes the following succession

of transitions:

So -> Si -> S2 -> S3 -> ... -> Sj -» ... ^ S,_i ->S^ (2-146)

In this context, the system is defined as a fluid element containing chemical species.

The above scheme may be looked upon as a birth process where the bom member

gives birth to only one new member. Once the new member gives birth, it becomes

infertile for any reason. At a certain time interval, it may happen that all members

are alive at different ages. However, at the end of the birth process, only one

member remains and all previous bom ones disappeared. The present birth scheme

is similar to the Poisson process, however, no disappearance of members versus

time occurs in the latter. In addition, the magnitude of the birth rate Xi in Eq.(2-

146) depends on the state Si, where in the linear birth process, according to which

mankind growth is conducted (in the absence of death), the birth rate is

proportional to the state size x according to Eq.(2-140). In view of the above

assumptions, we may conclude that Eqs.(2-138) to (2-139a) are applicable.

Let us now apply the above birth model to a well-known process, i.e., a

consecutive-irreversible z-stage first order chemical reaction, with a single initial

substance, the "first member of the family". The various states are Sj s Aj (i = 0,

1,..., Z) where A designates concentration of a chemical species i acquiring some

chemical formula; from Px(t) it follows that i = x. Considering Eq.(2-146), the

system occupies the various states at different times, i.e. a fluid element contains

different species along its transitions among the states. Px(t) is the probability of

occupying state x, i.e., occupying the chemical state of species i = x. x = 0 is the

initial chemical substance, x = 1 is the second chemical species, etc., where x = Z

is the "last bom member" of the family which remains alive for ever, according to

Eq.(2-146). Another interpretation of Px(t) is the probability that at time interval

Page 160: Applications of Markov Chains in Chemical Engineering

154

(0, t), the system will still occupy the state of chemical species x of concentration

p^(t) = Ax/(t)Ao(0) where Ao(0) is the initial concentration of species x = 0. On the

basis of Eqs.(2-138) to (2-139a) we may write for the consecutive-irreversible

reactions the following equations, designating X-i = ki where the latter are the

chemical reaction rate constants (1/time):

(2-147a)

for

for

for

dPo(t)

dt "

species 0.

dPi(t)

dt "

species 1.

dPjd)

dt

specie 2.

-'A^

-koPo(t)

-k,P,(t) ^ -koPo(t)

-k2P2(t) + kiP,(t)

= -k .P_ ,m + k Af " -Z- l ' Z-l^"-' ' "-Z-2^ Z-Z^"-'

for specie z-i. For species z the probability or mass balance reads

Px(0) = 1 for X = 0

Px(0) = 0 for X > 0

(2-147b)

(2-147c)

(2-147d)

dP,(t) - ^ = k,_,P,_i(t) (2-147e)

The following are the initial conditions:

(2-147f)

A general solution for the above set is available [22, p.l 1]. For Z = 2 considered

in the following, it reads:

PQ(t) = e-M (2-148a)

Page 161: Applications of Markov Chains in Chemical Engineering

155

Pl(t) = -^-!-r[e-V-e-KV] Iv — 1

(2-148b)

P2(t) = l -Po( t ) - Pi(t) (2-148C)

where K = ki/ko- Some characteristic properties of the above distributions are

demonstrated in Fig.2-51 where the relationship Px(kot) - kot is plotted for K =

0.25.

K = k/k =0.25 1 0

20

Fig.2-51. The Pure Birth model for consecutive reactions

Generally, it is observed that the system, eventually, occupies state 3, i.e.,

only species 3 is present; the other "members of the family" have died during the

time. On the basis of curves in the figure, we may ask also the following questions

which are of some interest. If the occupation probability of some state is, say,

0.368, then what is this state, what is the situation of the other states and during

what time interval are the above occupation probabilities valid? According to

Eq.(2-148c) the summation of all probabilities at each time must be unity.

Therefore, for the above probability, the following is the probability distribution of

the system among the states: Po(l) = 0.368, Pi(l) = 0.548 and P2(l) = 0.084. In

other words, at time kot = 1, 36.8% of the system still occupies the state x = 0 (So

= Ao), 54.8% underwent a transition to state x = 1 (Si = Ai) and 8.4% to state x =

Page 162: Applications of Markov Chains in Chemical Engineering

156

2 (S2 = A2). However, at time kot = 20, the whole system, practically, occupies

state X = 2 (S2 = A2).

To conclude birth models, it is interesting to present in Fig.2-52 the painting

Development II by Escher [10, p.276] which is originally a woodcut in three

colors. The painting demonstrates the development of reptiles and at first glance it

seems that their number is increasing along the radius. Although their birth origin

is not so clear from the figure, it was possible, by counting their number along a

certain circumference, to find out that it contains exactly eight reptiles of the same

size. This number is independent of the distance from the center. Thus, it may be

concluded according to Eq.(2-142) that dPx(t)/dt = 0, namely, Px=8(t) = 1 . In other

words, the probability of remaining at the state x = 8 reptiles during time interval

(0, t) is 100% since no birth takes place along the radius.

^ # . • • •_ • ^p*

« *

*

Fig.2-52. Escher*s demonstration for reptiles' birth rate of X = 0

(M.C.Escher "Development 11" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

Page 163: Applications of Markov Chains in Chemical Engineering

157

The Polya Process. In the above models the rate of occurrence of the

events X was independent of time, i.e., homogeneous. In the present process, it

has been assumed that the probability of a transition from Sj to Sk is time

dependent. Two cases are considered in the following.

The inversely proportional time dependence. It has been assumed [2,

p.82] that in the time interval (t, t + At) the transition probability is given by:

1 + OCX

prob{X(t, t + At) = 1} = '^YT^Hkt^^ " ^^^^^" Pjk (2-149)

where a and X, are non negative constants. The above probability designates an

occurrence of one event or one occupation during the time interval At. By

proceeding as before, we obtain the following differential equations:

dPo(t) 1 - i r = -^TTEalPo( t ) (2-150a)

subjected to the following initial conditions:

Px(0) = 1 for X = 0

P x ( 0 ) = O f o r x > 0 ^2-^^^*')

Eq.(2-150a) yields that:

Po(t) = (1 + aXt)- ' '" (2-151)

and the solution of Eq.(2-150) reads:

Px(t) = - ^ ( 1 + aXt r -^^ '^^J^d + ai) X = 1, 2,.. . (2-152) i = l

Page 164: Applications of Markov Chains in Chemical Engineering

158

For a = 0 the equation reduces to Poisson model. For a = 1 and by introducing

the new time parameter (1 + Xt) = exp(XT), one obtains the Yule-Furry process

given by Eq.(2-143a).

Some characteristic properties of the above distribution are:

a) Fig.2-53 demonstrates the Px(t) - Xt, Px(t) - x relationships computed from

Eq.(2-152) with Xi as parameter and for a = 1, 10. It is observed that generally

Eq.(2-126a) is satisfied.

1

0.8

^ 0.6

QL 0.4

K h x = 0

1—I—I—I—r

x=1 5 2 1 2 • a -/1 1 1 10 1.0

l^n—I—I—I— \— \—r

L xt = o,x = o

Fig.2-53. The Polya model for inversly proportional time

dependence

b) Px(t) obeys Eq.(2-127).

c) The mean number of events, m(t), occurring in the time interval of length t, is

defmed by Eq.(2-128) where for Px(t) given by Eq.(2-152)

m(t) = 2)^Px(t)»?^t x-O

(2-153)

The exponential time dependence. Another dependence on time of the

transition probability from Sj to S^, leading to a closed form solution of the

differential equation, assumes that the process rate parameter X. is given by:

A. = AQC (2-154)

where KQ and k are adjustable parameters. Let us relate X. to the following

humorous example noting that numerous cases fall in this category. A girls'

Page 165: Applications of Markov Chains in Chemical Engineering

159

dormitory accommodates N girls in every academic year. The mean rate of a girl

acquiring a boy-friend throughout the year varies with time according to Eq.(2-154)

where t denotes the number of months lapsed from the beginning of the academic

year. Assume that the same girls stay in the dormitory, there is no departure or

arrival of new girls during the academic year, a girl can have only one boy-friend

throughout the year, and that there are no break-ups during the year. The decrease

of X with time is plausible because as time goes by, the girls become more engaged

in their studies and don't have too much time for other activities. If the system is

the N girls in the dormitory and x is the state of the system, i.e., the number of

girls already acquired a boy-friend, where Px(t) is the probability that at time

interval (0, t) x girls are already occupied, then the set of Eqs.(2-122)-(2-124) is

applicable under the initial conditions Po(0) = 1, Px(0) = 0 for x = 1,2, ..., N.

The solution of Eq.(2-124) with X given by Eq.(2-154) is:

Po(t) = c^^M^M-^^)-i^ (2-155)

The rest of the solution is obtained as follows [23]:

dP,(t) ^ dP,(t) d?i ^ . kt^Px(t)_ , dP,(t) dt " d^ dt —^^0^ dX - ^^ dX

Thus, Eq.(2-123) is expressed in terms of the variable X instead of t by:

dP.(>.) 1

- ^ = ^ [ p , a ) + p , . i a ) ]

where Px(Ao) = 0 for x > 1. If the following solution is assumed:

P^iX) = Po(>.)F,(X) = c^^-'^'^F^iX)

it is possible to obtain for Fx(X,) that:

dF^ (X) 1.

dX = - k F x - i a )

Page 166: Applications of Markov Chains in Chemical Engineering

160

where Fx(X.o) = 0. Applying the solutions for Fx(X.) and the above relationships,

yields the following expression for Px(t) which is a modified Poisson model:

Px( t )« -^{[?^o /kIexp( -k t ) - l ]}VVa^^^^ X - 0 , 1 , 2 , . . . (2-156)

If k = 0, the original Poisson model is obtained, i.e., Eq.(2-126).

Some characteristic properties of the probabiUty distributions computed from

Eq.(2-156) are demonstrated in Fig.2-54. The figure on the left-hand side indicates

that the distributions versus time attain a steady state depending on XQ and k. For

example, for x = 1 and Kolk = 1, namely, one girl has already been occupied, it is

observed that the probability of remaining at this state during time interval (0, oo) is

0.368 where the probability of remaining in state x = 0 is zero at t > 0. Thus, the

probabiUty of occupying the other states in the above time interval is 1 - 0.368 =

0.632 , i.e., each girl will acquire a boy-friend throughout the year with some

probabiUty. The figure on the right-hand side indicates that increasing the number

X of the occupied girls, other parameters remain constant, reduces the probability of

remaining in this state. A possible explanation for this behavior is that an increased

number of occupied girls by a boy-friend, indicates that this is a good situation.

Thus, more giris would like to acquire boy-friends which is nicely predicted by the

model.

K 0.8

^ 06 U

Q- 0.4 1-.

0.2

0

"1—I—I—I— \—r

kt=0,x = 0

x=1 2 2 1 X^=1 1 5 5

1—r

^ - i — I * - ! - - ! —I -H H 4 6

kt 10

*^g^ I I 1 1 1 1 1 L ^ kt = 0.x = 0

r kt=i r 0 * =1 i K ^ h - \ j / 1 ^ 4 h \ 5 5 L V ^ ^ j/ L -^XSr'""^' -V- f' r^4»» r i^'i^^r -

r n

-H - n

1 H

1

J

±z2 4 6

X 10

Fig.2-54. The Polya model for exponential time dependence

Page 167: Applications of Markov Chains in Chemical Engineering

161

The Simple Death Process. In the above birth processes, the transition

of the system is always from state Sj = x - 1 to Sk = x where x > 0. This is clearly

reflected in Eqs.(2-122) and (2-138, 2-141). In the death process, the transition of

the system is in the opposite direction, i.e., from Sj = x + 1 to S^ = x, and the

derivation of the governing equations follows the assumptions:

a) At time zero the system is in a state x = XQ, i.e., the random variable

X(0) = Xo>l .

b) If at time t the system is in state x (x = 1, 2, ...), then the probability of the transition x -» x -1 at the interval (t, t + At) is:

Pjk = |ixAt + o(At) (2-156)

c) The probability of remaining in state x reads:

Pjj = 1 - iLlx t + o(At) (2-157)

where [X is the mean occurrence rate of the events which is a function of the actual

state. d) The probability of the transition x -^ x - i where i > 1 is o(At), i.e., twin or

multiple death is impossible. In view of the above assumptions and following

Eq.(2-3), we may write that:

Px(t -h At) = (1 - HxAt)Px(t) + |Lix+iPx+i WAt + o(At) (2-158)

The reason that x + 1 occurs in the coefficient of Px+i(t) is that the probability of an

event has to be taken conditional on X(t) = x + 1. The differential equation which

follows reads:

dP.(t) — ^ = - H,P,(t) -h ix+iPx+iCt) (2-159)

where |LIX is the death rate which is a function of the state the system occupies.

If |Lix = |Li = constant, we have a constant rate death process. The case of fx =

1 death per unit time may be demonstrated in Fig.2-49 as follows. If one is

observing the figure, while beginning from the middle and continuing to left or to

Page 168: Applications of Markov Chains in Chemical Engineering

162

the right, he can see that the number of fish or beens is reducing up to one, due to

their death for some reason.

If it is assumed that the death rate follows a linear relationship, i.e., jix = Mx

where |i > 0, we obtain the so-called simple death process or linear death process

satisfying:

dP.(t) — ^ = - |ixP,(t) + ^(x + l)Px+,(t) (2-160)

subjected to the following initial conditions:

Px(0)=l for x = xo> 1

Px(0) = 0 for X ;i xo

The solution of the differential equation is [2, p.85]:

(2-161)

Px (t) = , / " ' „e-' o tKe^ t_i)Xo-x o < x < x o (2-162) x!(x — XQ)!

with the following characteristics: a) Fig.2-55 demonstrates the Px(A.t) - x, Px(?tt) - Xi relationships computed from

Eq.(2-162) with \ii as parameter for two initial values of XQ = 5, 10. A typical

behavior revealed in the present model indicates that:

Px = xo(0) = Px = o M = l (2-163)

which is due to XQ > x. On the other hand, in the previous models:

Px = o(0) = 1 and Px>o(oo) = 0 (2-164)

Page 169: Applications of Markov Chains in Chemical Engineering

163

1 — I — I — I — I — I — ! — r 1—I—I—I—I—r

x^ = 5

I >i ^ 1 ^ r-->j \ I I L 2 4 6 8 10 0 2 4 6 8 10

X X

1—\—r—r

x^ = 5

J L-L 2 4 6 8 10 0 2 4 6 8 10

Fig.2-55. The Simple Death model

b) Since XQ > x and depending on the magnitude of XQ, the probabiUty of remaining

in a certain state increases versus time.

c) Px(t) obeys Eq.(2-127).

d) The mean number of events, m(t), occurring in the time interval of length t, is

defined by Eq.(2-128) where for Px(t) given in Eq.(2-162):

m(t) = 2 x P ^ ( t ) = Xoe-»* x-O

(2-165)

The Birth-and-Death Process. An example of such a process are cells

undergoing division, i.e., birth, w^here simultaneously they also undergo

apoptosis, i.e., programmed cell death. The latter might be due to signals coming

from exterior of the cells. The existence of human being may also be looked upon,

approximately, as a simultaneous birth-death process although population in the

Page 170: Applications of Markov Chains in Chemical Engineering

164

world is generally increasing. One should imagine what would have happened if

death phenomenon would not have existed. Finally, it should be noted that birth-

and'death processes are of considerable theoretical interest and are also encountered

in many fields of application.

The derivation of the governing equations is based on the following

assumptions:

a) If at time t the system is in state x (x = 1, 2,...), the probability of the transition X -> X + 1, i.e. Sj to Sk, in the time interval (t, t + At) is given by:

Pjk = >oc t + o(At) (2-166)

where Xx is the mean birth rate which is an arbitrary functions of x.

b) If at time t the system is in the state x (x = 1,2, ...), the probability of the transition x -^ x - 1, i.e. Sj to Sk, in the time interval (t, t + At) is given by:

Pjk = M t + o(At) (2-167)

where |LIX is the death rate.

c) The probability of a transition to a state other than the neighboring state is o(At).

d) If at time t the system is in the state x (x = 1, 2,...), the probability of remaining

in the state is given by:

pjj = 1 - {\ + ixJAt + o(At) (2-168)

e) The state x = 0 is an absorbing or dead state, i.e., poo =1.

In view of the above assumptions and following Eq.(2-3), we may write that:

P^(t + At) = ;ix-iPx-i WAt + |ix+iPx+i(t)At + [1 - (Xx + ^x)^t)]Px(t) + o(At)

(2-169)

leading to the following differential equation:

dP.(t) - ^ = ?^x-iPx-i(t) - ^ x + ^x)Px(t) + Hx+iPx^i(t) (2-170)

Page 171: Applications of Markov Chains in Chemical Engineering

165

In Eq.(2-169), the first two terms on the right-hand side describe the transitions

from states x-1 and x+1 to state x; the third term designates the probability of

remaining in state x. Eq.(2-170) holds for x = 1, 2, ... where for x = 0, noting

assumption e above, we have:

- ^ = ^iPi(t) (2-171)

since X^i = XQ = [XQ = 0. If at t = 0 the system is in the state x = XQ, 0 < XQ < <»,

the initial conditions are:

Px(0)=l forx = xo

Px(0) = 0 forx^txo (2-172)

Depending on the X^-x and |ix " ^ relationships, two cases will be considered,

viz., the linear birth-death process and the z-stage reversible-consecutive process.

The linear birth-death process. It is assumed here that (i^ = fix and A. = X,x

where \i and X are constant values. The solution obtained for XQ = 1[2, p.88],

which satisfies Eqs.(2-172), reads:

Px(t) = [1 - a(t)][l - p(t)][p(t)]^-i X = 1, 2,... (2~173a)

Po(t) = a(t) (2-173b)

where

M[e(^-fi)t - 11 M

m = rK-T-^ (2-174b)

Page 172: Applications of Markov Chains in Chemical Engineering

166

Some characteristics of the solution are:

a) Fig.2-56 demonstrates the Px(t) -1 relationship computed by Eqs.(2-173a,b) for

different values of \i and X designated on the graph.

40 50

Fig.2-56. The Birth-Death model

The following trends, also demonstrated in the figure, are observed regarding to the

extinction probability Po(t), i.e., the probability that the population will eventually

die out by time t The trends are obtained from Eq.(2-173b) by letting t -> oo.

Hence:

lim Po(t) -= 1 for X < |ui t-»oo

« Y f or X > ji (2-175)

From the above results we can see that the population dies out with a probability of

unity when the death rate is greater than the birth rate, but when the birth rate is

greater than the death rate, the probability of eventual extinction is equal to the ratio

Page 173: Applications of Markov Chains in Chemical Engineering

167

of the rates. Additional computations indicate that if X > |Li (left-hand side graphs),

increasing X at constant i decreases Px(t). For example:

for ^1= 1 and A, = 2, Po(t) = 1 i f t > 3

for|Li=land^=10,Po(t) = 0.1 i f t > l

for |i = 1 and X = 50, Po(t) = 0.02 if t > 1.

If |l > A., increasing |Li causes Po(t) to approach unity and the others to faster

approach zero.

b) Px(t) obeys Eq.(2-127).

c) The mean number of events, m(t), occurring in the time interval of length t, is

defined by Eq.(2-128), where for Px(t) given in Eq.(2-173)

m(t) = e( - )t (2-175)

The following asymptotic behavior is obtained:

limm(t) = 0 for X<\i

= 1 for A = |i

= oo for >.>^ (2-176)

From the above we can see that when A = |i, the expected rate of growth is zero

and the mean population size is stationary.

The Z'State reversible-consecutive process. In this process, widely

encountered in chemical engineering reactions, the system, defined below,

undergoes the following succession of simultaneous birth-death transitions:

XQ X J X,2 X^ X J _ | X- X^_2 A,2_j

So -^ Si -» S2 ^ S3 -^ ...-^ Si -» ... -^ S,_i ^ S, (2-177)

^1 1^2 \^3 1^4 \^i \^M ^^z-1 \^z

where the magnitude of the birth rate Ai and the death rate fij depend on the state Sj.

Let us now apply the above simultaneous birth-death model to a well-known

process, i.e., a z-stage irreversible-consecutive first order chemical reaction, with a single initial substance. The various states are Sj = Aj (i = 0, 1, ..., Z) where Ai

designates concentration of a chemical species i acquiring some chemical formula;

Page 174: Applications of Markov Chains in Chemical Engineering

168

considering the quantity Px(t), it follows that i = x. In this context, the system is

defined as a fluid element containing chemical species which undergo a chemical

reaction or not. Considering Eq.(2-177), the system occupies the various states at

different times. Px(t) is the probabiUty of occupying state x, i.e., occupying the

chemical state of species i = x. x = 0 is the initial chemical substance, x = 1 is the

second chemical species, etc., where x = Z is the last born member of the family

which remains alive for ever. Another interpretation of Px(t) is the probabiUty that

at time interval (0, t), the system still occupies the state of chemical species x of

concentration Px(t) = Ax(t)/Ai(0); Ai(0) is the initial concentration of species x = i.

On the basis of Eq.(2-170) we may write the following equations where %{and m

are the chemical reaction rate constant (1/time):

^ ^ = HiPi(t)-?ioPo(t) (2-178a)

for species 0.

^ ^ = |i2P2(t) + K^oii) - Pi(t)[^i + X,] (2-178b)

for species 1.

dPn(t) — ^ = ^3P3(t) + ?LiPi(t) - P2(t)[|Li2 + ^2] (2-178C)

for specie 2.

^ ^ = n,P,(t) + V2P,_2(t) - P.-i(t)[|i.-, + K-{\ (2-178d)

for specie z-1 where for specie z

dp,(t) dt = ^z-iPz-i(t) - HzPz(t) (2-178e)

Page 175: Applications of Markov Chains in Chemical Engineering

169

For Z = 2, a solution is available [22, p.42] if we take species 0 with initial

concentration Ao(0) to be the starting substance. Then, Px(t) = Ax(t)/Ao(0) and the

solution reads:

Po(t) = ^^rr + ; e ^^' + Y1Y2 Y I ( Y I - Y 2 )

YVY2(^ll+^2+^l) + ll i2

Y2(Y2-YI)

-Y2t (2-179a)

P,(t) = >.o LY1Y2 YI(Y,-Y2) Y2(Y2-YI)

(2-179b)

P2(t)=^? i [ ^ 7 7 7 : + - r — : « " ' • ' + - r ^ H LY1Y2 Y I (Y I -Y2 ) Y2(Y2-YI) J

(2-179C)

where YJ and Y2 are roots of the following quadratic equation taken with the reverse

signs:

Y + Y(^ + X,i + M-i + 1I2) + ^^1 + ^|A2 + RIH2 = 0 (2-180)

Some characteristic properties of the above probability distributions, or

dimensionless concentration of the species, are demonstrated in Fig.2-57 where the

relationship Px(t) -1 is plotted.

Page 176: Applications of Markov Chains in Chemical Engineering

170

1

0.8

^ 0.6 X

^ 0.4

0.2

0

"\ x = 0

- \

-

L 1 IZ- , 1 .

1

1

' 1 ' 1 ' J

0 1 J ^1^ = 1 ^^2=1 1

_J J — —-„ : =" * !

1 1 1 1 1

x = 0

^^ 2- ^^ = 1 X^=0.5

, \ ^ 1 = 0.2 t =0.1

2 3 t

10

Fig.2-57. The Birth-Death model for reversible-consecutive reactions

Generally, it is observed that the system, i.e., a fluid element containing

some chemical species, attains a steady state distribution of its occupation

probability among the various states. The distribution depends on the rate

constants designated in the figure and are characteristic of reversible reactions.

Considering the figure on the left-hand side, it may be concluded that the

probability of remaining in states 0, 1 and 2 at time interval (0, ©o) is 1/3 for each

state, i.e., in a form of the initial component, component 1 and component 2. The

above probabilities indicate also that in the above time interval, exactly three

changes have occurred, i.e., the formation of states 2 and 3 due to the reaction and

also the remaining in state 0. Certainly, the quantities Px(oo) = Ax(c»)/Ao(0), x =

0, 1, 2, may be looked upon as dimensionless concentrations encountered in

reaction engineering. On the basis of the figure on the right-hand side, it may be

concluded that if the rate constants |LII and |i2 will approach zero, i.e., the effect of

the irreversibility is diminished, the probability of remaining at states 0 and 1 will

also diminish at time interval (0, ©o), and the only state to be occupied by the

system is 3, i.e., the state of solely component 3.

2.3 MARKOV CHAINS CONTINUOUS IN SPACE AND TIME 2.3-1 Introduction

The above topic is presented in the following because the equations

developed and the basic underlying mechanism, i.e. diffusion, are of fundamental

Page 177: Applications of Markov Chains in Chemical Engineering

171

importance in Chemical Engineering. However, the major problem of applying the

equations is that not always an analytical solution is available.

Let us consider a dancing hall where people are arriving at and leaving from

after they have enjoyed the evening. If the hall is defined as system and the number

X of people present in the hall at some time during the evening is the state of the

system, x = 1, 2, 3,. . . , then this situation may be looked upon as a Markov chain

continuous in time and discrete in space, elaborated in the preceding section 2.2.

More specifically the situation may be categorized as a birth-death model where the

entering people designate birth and the leaving ones symbolize death. The mean

rates of birth and death can easily be determined by counting the people arriving

and leaving the hall. In the above example, one may be interested in the probability

that at a certain time of the evening, the number of dancers in the hall is some

prescribed value. Certainly, early in the evening, x = 0 and Px(0) = 1. The main

characteristic of processes similar to the above one, as well as the ones discussed in

section 2.2, is that in a small time interval there is either no change of state or a

radical change of state. Therefore in a finite interval there is either no change of

state or a finite or possibly denumerably infinite number of discontinuous changes.

Assume now that some time in the evening the dancing hall becomes

extremely congested and the doors are suddenly shut. Under these conditions, the

movement of the dancers in the hall may be governed by random impacts of

neighboring dancers. A typical physical situation, similar to the above one, is that

of particles suspended in a fluid, and moving under the rapid, successive, random

impacts of neighboring particles. If for such a particle the displacement in a given

direction was plotted against time, we would expect to obtain a continuous or

somewhat erratic graph which would, in fact, be a realization of a stochastic

process in continuous time with continuous state space. The characteristic of such

processes is that in a small time interval, displacement or change of state are small.

The physical phenomenon related to the above behavior is known as Brownian

motion, first noticed by the botanist Robert Brown in 1827. The modeling of such

motions is based on the theory of diffusion and kinetic theory of matter. Markov

processes associated with above motions, in which only continuous changes of

state occur versus time, are therefore called diffusion processes. The above points

will be elaborated in the following.

Page 178: Applications of Markov Chains in Chemical Engineering

172

2.3-2 Principles of the modeling Brownian motion of particles is the governing phenomenon associated with

transitions between states in the above examples as well as in the mathematical derivations in the following [4, p.203]. If we consider a particle as system and the states are various locations in the fluid which the particle occupies versus time, then the transition from one state to the other is treated by the well-known random walk

model. In the latter, the particle is moving one step up or down (or, alternatively, right and left) in each time interval. Such an approach gives considerable insight into the continuous process and in many cases we can obtain a complete probabilistic description of the continuous process.

The essence of the above model is demonstrated for the simple random walk

in the following. Let X(n) designate the position at time or step n of the moving particle (n = 0, 1,2,...). Initially the particle is at the origin X(0) = 0. At n = 1, there is a jump of one step, upward to position 1 with probability 1/2, and downwards to position -1 with probability 1/2. At n = 2 there is a further jump, again of one step, upwards or downwards with equal probability. Note that the jumps at times n = 1, 2, 3, ... are independent of each other. The results of this fundamental behavior are demonstrated in Fig.2-58 where two trajectories 1 and 2 for a single particle, out of many possible ones, are shown.

T \ \ I I I I r

J I I \ \ L 0 1 2 3 4 5 6 7 8 9 10 11 12

Time, n

Fig.2-58. Two independent realizations of a simple random walk

Page 179: Applications of Markov Chains in Chemical Engineering

173

In general, the trajectory of the particle is given by:

X(n) = X(n - 1) + Z(n) (2-181)

where Z(n), the jump at the nth step, is such that the random variables Z(l), Z(2),

... are mutually mdependent and all have the same distribution which reads;

prob{Z(n) = 1} = prob{Z(n) = - 1} = 1/2 (n = 1, 2,...) (2-182)

Finally, it should be noted that Eq.(2-181), taken with the initial condition X(0) =

0, is equivalent to:

X(n) = Z(l) + Z(2) + Z(3) + ... + Z(n) (2-183)

The mathematical elements used in the following derivations are:

X(t) - a random variable designating the position of a particle, system, in the fluid

at time t.

X(t) = X, indicates realization of the random variable, i.e., that at time t the random

variable acquired a value x, or, that the system occupied state x.

p(y, x; X, t) - a probability density function, i.e. probability per unit length, where,

p(y, T; X, t)dx - is the probability of finding a Brownian particle at time t in the

interval (x, x + dx) when it is known that the particle occupied state y at an earlier

timex.

It follows from above definitions that:

prob{a < X(t) < b | X(T) = y} = j p(y, x; x, t)dx (t > x) (2-184)

i.e., the probability of finding a particle at time t between states a and b, subjected

to the fact that the particle occupied state y earlier at time x, is given by the integral

on the right-hand side. Assume now that the lower limit is a = 0, then we define a

new ftmction P(y, x; x, t), the transition probability function by:

Page 180: Applications of Markov Chains in Chemical Engineering

174

P(y,T; b, t) = p(y, x; x, t)dx = prob{ X(t) < b | X(T) = y} (t > x) (2-185) Jo

where

p(y,T;x.t) = ^ ^ i i ^ 4 i i ^ (2-186a)

P(y, x; X, t), the probability of finding a particle at time t in the interval (0, b) while

recalUng it occupied state y at time x, also satisfies the following conditions:

lim P(y, x; x, t) = 0

x ^ O 1- T./ A (2-186b) hm P(y, x; X, t) = 1

2.3-3 Some continuous models In the following, two basic models are presented [2, p. 129; 4, p.203]; the

Wiener process and the Kolmogorov equation. Applications of the resulting

equations in Chemical Engineering are also elaborated. The common to all models

concerned is that they are one-dimensional and, certainly, obey the fundamental

Markov concept - thai past is not relevant and thai future may be predicted from the

present and the transition probabilities to the future.

The Wiener process. We consider a particle governed by the transition

probabilities of the simple random walk. The steps of the particle are Z(l), Z(2),

... each having for n = 1, 2,.. . the distribution:

prob{Z(n) = 1} = p, prob{Z(n) = - 1} = q = 1 - p (2-187)

where p and q are constant one-step transition probabilities. Assume that a particle

is in the time coordinate n - 1 . It undergoes a transition to a state coordinate k from

the state coordinates k - 1 and k + 1. Thus, we may write the following forward

equation for a fixed j :

Page 181: Applications of Markov Chains in Chemical Engineering

175

Pjk(n) = Pj,k-l(n - l)p + Pj,k+l(n - l)q (2-188)

where pjk(n) is the probability that a particle occupies state k at time n, if it occupied

state j at n = 0. Considering the definition of p(y, T; X, t)dx above, let:

p(yo, 0; X, t)Ax = p(xo, 0; x, t)Ax = p(xo; x, t)Ax (2-189)

be the conditional probability that the particle is at x at time t, given that it started at

yo = xo at time T = 0; x is a continuous state coordinate. We have also XQ = j Ax,

X = kAx and t = nAt, thus, Eq.(2-188) in terms of the new scale reads:

p(xo; X, t) = p(xo; x - Ax, t - At)p + p(xo; x + Ax, t - At)q (2-190)

the factor Ax canceling throughout. Suppose that p(xo; x, t) can be expanded in a

Taylor series and the first and second derivatives exist, we obtain that:

p(xo; X + Ax, t - At) = p(xo; x, t) - A t | ^ + A x - ^ + 0.5(Ax)^|^ (2-191)

If we expand Eq.(2-190) according to Eq.(2-191) and apply the following

expressions for p and q [4, p.206]:

p = 0.5[l + i ! f ^ ] , q = 0 . 5 [ l - i ! f i ] (2-192)

where |i is the mean (length/time) and a^ is the variance (lengths/time) of the

random variable X(t) designating the position of the particle at time t. If At -> 0,

we obtain the forward equation:

^P(xo; X, t) ^^2^^p{xo,x,i) apCxp; X, t) 5t = ^-^^ ; 2 ^ ^ (2-193)

Page 182: Applications of Markov Chains in Chemical Engineering

176

which is a partial differential equation of the second order with respect to x and is

first order in t. It is an equation in the state variable x at time t for a given initial

state XQ. The solution of the above equation conditional on X(0) = XQ reads:

with the two parameters i and a. Eq.(2-194) satisfies Eqs.(2-185)-(2-186b).

Eq.(2-193) is familiar in Chemical Engineering in the following cases [24]:

a) Diffusive-convective heat transfer. In this case, the equation of energy for

constant properties, ignoring viscous dissipation, taking into account axial heat

diffusion only in the x-direction which is superimposed on the molecular diffusion

of heat, reads:

9T(x, t) ^,3^T(x,t) ,,aT(x,t) . .Q. ^ — 5 t — = ^ 9^2 " ^—^T- (2-195a)

Comparing the above equation with Eq.(2-193) demonstrates the similarity between

the two for XQ = 0. The parameters appearing in Eq.(2-195a), i.e., the heat thermal

diffusivity a which accounts for the molecular diffusion, and the constant axial

velocity U accounting for the axial diffusion, are characteristic to the heat transfer

process. The underlying model of particle's motion associated with the derivation

of Eq.(2-193) may be applied also to heat transfer. In this case, a fluid element (or

a molecule) at some temperature moves due to both, diffusion as well as axial

diffusion generated by the fluid velocity, transferring heat to the other fluid

elements. The motion of the fluid elements may be looked upon also from

probabilistic arguments which led to Eq.(2-193). Finally, it should be mentioned

that in particular problems, the solution of the equations will depend on initial and

boundary conditions.

b) Diffusive-convective mass transfer. The equation of continuity for species

A in a binary solution of A+B, assuming constant density of the fluid mixture as

well as the diffusion coefficient DAB. taking into account axial mass diffusion only

in the x-direction which is superimposed on the molecular diffusion of mass, reads:

Page 183: Applications of Markov Chains in Chemical Engineering

177

dCAx, t) 9^CA(X, t) 9CA(X, t) - V - = D A B — ^ ; 2 U - ^ ^ (2-195b)

CA is the local concentration of species A in the mixture. Other remarks made in

(a) are also applicable here.

c) Heat or mass transfer into semi-infinite bodies by pure conduction or

diffusion. The governing equations may be obtained from the above equations by

ignoring the axial component; thus:

3T(x, t) , .8^(x, t )

^£^ = j£0dl (2-1950

The above equations are similar to Eq.(2-193) for |LI = 0 which assumes that a fluid

element of some concentration, or at some temperature, is moving by pure

molecular diffusion, thus generating mass or heat transfer in the system.

An additional equation of a similar form, belongs to momentum transfer. For

a semi-infinite body of liquid with constant density and viscosity, bounded on one

side by a flat surface which is suddenly set to motion, the equation of motion reads

[24, p. 125]:

^ ^ = v ^ ^ (2-195d)

Ux is the velocity of the liquid in the x-direction, and v is the kinematics viscosity of

the liquid governing the momentum transfer by pure molecular diffusion.

The Kolmogorov equation. In the preceding model, the major

assumption made was that the one-step transition probabilities p and q remain

constant. In the following it is assumed that these probabiUties are state depending,

namely, designated as pk and q . The following forward equation for the present

model reads:

Page 184: Applications of Markov Chains in Chemical Engineering

178

Pjk(n) = Pj,k-i(n - l)pk-i + Pj,k+l(n - l)qk+l + Pj,k(n - 1)(1- Pk - Qk) (2-196)

The equation may be interpreted in the following way: the probability of being at

state k after n steps = the probability of being at state k-1 after n-1 steps times the

one-step transition probability (which depends on state k-1) plus the probability of

being at state k+1 after n-1 steps times the one-step transition probability of moving

from state k+1 to k (which depends on state k+l) plus the probability of being at

state k after n-1 steps times the probabihty of remaining in this state, (1- Pk - Qk); all

transitions take place in the time interval n and n+1. Again we consider the process

as a particle taking small steps -Ax, 0, or +Ax in the small time interval of length

At. From Eq.(2-189) the probability density p(xo; x, t)Ax may be looked upon as

the conditional probabihty that a particle is at location x at time t, given that initially

it occupied XQ. Thus, Eq.(2-196) becomes:

p(xo; X, t) = p(xo; x - Ax, t - At)p(x. AX) + p(xo; x + Ax, t - At)q(x + AX)

+ p(xo; X, t - At)(l - qx - px) (2-197)

the factor Ax canceling throughout. It has been shown elsewhere [4, p.214] that:

l i m { p ^ - q j - ^ = p(x)

Ax,At-^0

lim{p^ + qx - (Px - q x ) ^ } - ^ = a(x) (2-198)

Ax,At-^0

where P(x) and a(x) are the instantaneous mean (length/time) and variance

(lengths/time), respectively. Applying Eqs.(2-197) and (2-198), resulted in the

following forward equation:

ap(xo; X, t) a^{a(x)p(xo; x, t)} 3{(3(x)p(xo; x, t)} — 5 ^ - = 0-5 ^;;2 T. (2-199)

Page 185: Applications of Markov Chains in Chemical Engineering

179

which is a parabolic partial differential equation. A more general equation can be

obtained by allowing the transition mechanism to depend not only on the state

variable x but also on time t. In this case we are led to define p(x, t) and a(x, t),

both depending on x and t, i.e., the instantaneous mean (length/time) and variance

(lengths/time), respectively. Thus, if XQ denotes the state variable at time to and x

that at a later time t, then the transition probability density p(xo, to; x, t) satisfies the

following/<7rwar<i Kolmogorov equation also called the Fokker-Plank equation:

ap(xo, to ; x, t) aS{a(x, t)p(xo, to*, x, t)} 3{p(x, t)p(xo, to; x, t)} 3t = ^'^ ^;;2 ^ (2-200)

where it has been assumed in the derivations that the above partial derivatives exist.

The functions P(x, t) and a(x, t) are sometimes called the infinitesimal mean and

variance of the process. If these functions are assumed as constant values, the

above equation reduces to Eq.(2-193) where a(x, t) = a^ and P(x, t) = |LI.

Eq.(2-200) is familiar in Chemical Engineering in turbulent flow. For

example, the energy equation for one-dimensional flow [24, p.377] for a fluid of

constant properties, in the absence of viscous dissipation effects and for xo = to =

0, reads:

aT(x,t) ,.3^T(x,t) a{u,(x,t)T(x,t)} — 5 r - = ^ -^ ;^2 ^ (2-201)

where T = T + T'; the thermal diffusivity a has been assumed constant. Note that

in turbulent flow the temperature T is a widely oscillating function of time,

fluctuating about the time-smoothed value of the temperature T, where T' is the

temperature fluctuations.

The major problem of Markov chains continuous in time and space is that the

availability of analytical solutions of the governing equations, which depend also

on the boundary conditions, is limited to simplified situations and for more

compUcated cases, numerical solutions are called for.

Page 186: Applications of Markov Chains in Chemical Engineering

180

2.4 CONCLUDING REMARKS In the present chapter, Markov processes discrete in time and space,

processes discrete in space and continuous in time as well as processes continuous

in space and time, have been presented. The major aim of the presentation has been

to provide the reader with a concise summary of the above topics which should

give the reaaer an overview of the subject.

The fundamentals of Markov chains have been presented in an easy and

understandable form where complex mathematical derivations are abandoned on the

one hand, and numerous examples are presented on the other. Despite of the

simplifications made, the author believes that the needed tools have been provided

to the reader so that he can solve complicated problems in reactors, reactions,

reactor plus reactions and other processes encountered in Chemical Engineering,

where Markov chains may provide a useful tool.

The models discrete in space and continuous in time as well as those

continuous in space and time, led many times to non-linear differential equations

for which an analytical solution is extremely difficult or impossible. In order to

solve the equations, simplifications, e.g. linearization of expressions and

assumptions must be carried out. However, if this is not sufficient, one must

apply numerical solutions. This led the author to a major conclusion that there are

many advantages of using Markov chains which are discrete in time and space.

The major reason is that physical models can be presented in a unified description

via state vector and a one-step transition probability matrix. Additional reasons are

detailed in Chapter 1. It will be shown later that this presentation coincides also

with the fact that it yields the finite difference equations of the process under

consideration on the basis of which the differential equations have been derived.

2.5 ARTISTIC ENDING OF THE CHAPTER The present chapter begun with Fig.2-0, the painting Waterfall by Escher,

which made it possible to present in various places of this chapter the basic concept

of conditional probability. We end this chapter by 'Markovization', discrete in time

and space, of the amazing oil on canvas painting by Magritte Carte blanche-

Signature in blank [12, p.45] depicted in Fig.2-59. This demonstrates again a way

of entertaining the combination of art and science. A few words about this

Page 187: Applications of Markov Chains in Chemical Engineering

181

painting. Magritte, in an extremely subtle and deceptive way, demonstrated the

simultaneous movement in two planes of the horsewoman. Between two trunks

the normal backdrop of foliage is visible, and this conceals a portion of the horse

and rein, the horse appearing to be passing between the same two trunks.

Spatially, the rider and the woods become an absurdity, due to this section of the

intruding background, the position of one of the horse's hind legs, and another tree

trunk in the background, part of which passes in front of the horse and rider.

Magritte expressed his thoughts on his painting as follows: "Visible things can be

invisible. If somebody rides a horse through a wood, at first one sees them, and

then not, yet one knows that they are there. In Carte blanche, the rider is hiding the

trees, and the trees are hiding her. However, our powers of thoughts grasp both

the visible and the invisible."

Page 188: Applications of Markov Chains in Chemical Engineering

182

Fig.2-59. The impossible state S5 ("Carte blanche", 1965, © R.Magritte, 1998 c/o Beeldrecht Amstelveen)

Let us now apply Markov chains to investigate the trajectory of the system,

the horsewoman, riding in the forest. The possible states that the system can

occupy on the basis of Fig.2-59 are defined as follows: Si-the passage between

trees 1 and 2 through which the system can move, S2-the passage between trees 4

and 3, S3-the passage between trees 4 and 5, S4-the passage between trees 7 and 6,

Page 189: Applications of Markov Chains in Chemical Engineering

183

and Ss-the impossible situation depicted in Fig.2-59. It is assumed that once the

horsewoman abandons S5, she never retums to it and continues to ride in the forest

according to her mood at the moment. Thus, S5 is an emepheral state. The

policy-making matrix distinguishes among all cases analyzed below where the

conmion to all of them is that S(0) = [0, 0, 0, 0, 1], i.e., the rider is initially at S5.

The results of the computations are depicted in Fig.2-60; on the top of each

figure, the corresponding one-step transition probability matrix is presented.

Generally, in cases a to c the states attain a steady state whereas the stationary

distribution in each case obeys Eq.(2-105a), i.e. the states reveal Ergodic

characteristics. In cases d to f the states exhibit periodic behavior whereas S5 is

eventually abandoned in all cases. In cases a, c, d and f it is abandoned after the

first step whereas in cases b and e, abandoning occurs after a few steps. This is

because in the corresponding matrix, there is also a probability of remaining in the

state, pii ^ 0.

In the following we consider each case separately. In case a the steady state

occupation probability is different for each state; the corresponding state vector

reads, S(l 1) = [0.300, 0.400, 0.200, 0.100, 0.000]. As seen, S2 is of the highest

probability, i.e. S2(l 1) = 0.400. Case b differs from a only by the values of p5i;

thus, causing vanishing of S5(n) to occur after 6 steps rather than after 1 step.

From thereon, the values of S(n) are the same as in a. Case c causes a problem to

the horsewoman because S(2) = [0.25, 0.25, 0.25, 0.25, 0.00]; thus she has a

problem what state to occupy because all states acquire an identical probability.

Case d reveals periodic characteristics of the states with period of v = 2 as follows:

S(21) = [0, 1/3, 0, 2/3, 0]

S(22) = [2/3, 0, 1/3, 0, 0]

S(23) = [0, 1/3, 0, 2/3, 0]

S(24) = [2/3, 0, 1/3, 0, 0]

In other words, in each step two states, S2 and S4 or Si and S3, acquire a certain

occupation probability. Case e demonstrates the following periodic results at

steady state

Page 190: Applications of Markov Chains in Chemical Engineering

184

S(12) = [0.133, 0.267, 0.533, 0.067, 0]

S(13) = [0.067, 0.133, 0.267, 0.533, 0]

S(14) = [0.533, 0.067, 0.133, 0.267, 0]

S(15) = [0.267, 0.533, 0.067, 0.133, 0]

where the period of each state is v = 4. Case f is similar to e whereas the

occupation probability of each state at each step is unity. In conclusion we may say

that the major problem of the horsewoman, once abandoning the strange state S5,

is as William Shakespeare profoundly stated:" to be (in a state) or not to be (in a

state), that is the question''.

Si

S2

P = S3

S4

S5

Case a

Si S2 S3 S4 S5

0 1 0 0 0

1/2 0 1/2 0 0

0 1/2 0 1/2 0

1 0 0 0 0

1/2 0 0 1/2 0

Si

S2

P = S3

S4

S5

Caseb

Si S2 S3 S4 S5

0 1 0 0 0

1/2 0 1/2 0 0

0 1/2 0 1/2 0

1 0 0 0 0

1/3 0 0 1/3 1/3

1

0.8

0.6

I \ 1 r n r (a)

s/n) spx) s (n) s^(n).

n I I I r

/^<")

"1—r~ (b)

s,(n) s(n) s(n) s (n)

8 10 0 8 10

Page 191: Applications of Markov Chains in Chemical Engineering

185

p =

Si

S2

S3

S4

S5

Si

0

1/2

0

1/2

1/2

Casec S2 S3

1/2 0

0 1/2

1/2 0

0 1/2

0 0

S4

1/2

0

1/2

0

1/2

S5

0

0

0

0

0

Si

S2

P = S3

S4

S5

Si

0

0

0

1

0

S2

1/2

0

0

0

0

S3

0

1

0

0

0

S4

1/2

0

1

0

1

S5

0

0

0

0

0

" 1 — I — \ ! — ! — \ 1 I r~ (c)

s.(n), i = 1,...,4

M . I I I I I I I i I I i I

10 0

Si S2 S3 S4 S5

Si

S2

S3

S4

S5

1

0

0

0

0

0

1

0

0

0

0

0

1

0

1

0

0

0

0

0

'\ /.

8 10 0

., ,; \/ 1 ; *)

y nv ''v•• 2 4 6 8 10

n Fig.2-60. Policy-making matrices and state occupation probabilities

of the horsewoman

Page 192: Applications of Markov Chains in Chemical Engineering

186

Chapter 3

APPLICATION OF MARKOV CHAINS IN CHEMICAL REACTIONS

Chemical reactions occur due to collisions between similar or dissimilar

species. The collision process requires, first of all, the coming together of

molecules, which is of probabilistic or stochastic nature. Thus, chemical reactions

may be looked upon as probabilistic or stochastic processes and more specifically

as a Markov chain. According to this model, concentration of the species at time

(n+1) depends only on their concentration at time n and is independent on times

prior to n. Therefore, the governing equations for treating chemical reactions, as

elaborated later, are Eqs.(2-23) and (2-24) below, derived in Chapter 2:

z

Sk(n+1) = X^j(n)pjk (2-23) j=i

S(n+1) = S(n)P) (2-24)

The application of the equations to chemical reactions requires the proper

definition of the above quantities as well as correctly defining the transition

probabilities pjj and pjk*, this is established in the following. It should also be noted

that the models derived below for numerous chemical reactions, are applicable to

chemical reaction occurring in a perfectly-mixed batch reactor or in a single

continuous plug-flow reactor. Other flow systems accompanied with a chemical

reaction will be considered in next chapters.

Page 193: Applications of Markov Chains in Chemical Engineering

187

The attractiveness of Markov chains in chemical reactions is particularly due

to the following reasons: a) Simplicity, elegance and the didactic value,

b) Demonstrating the power of probability theory in handling a pric^n deterministic

problems, c) Applying fundamentals of linear algebra to chemical reaction

problems of practical importance, d) The attractiveness of the method increases

with the number of reacting components, if the reactions are higher-order or if they

are non-isothermal. In such cases the governing differential equations are

nonlinear, and an analytical solution is extremely difficult, if not impossible;

finally, e) The solution of Eq.(2-24) yields the transient response of the reaction

towards attainment of a steady state, which is important in practice. The above

points are demonstrated by the numerous examples presented in this book.

Throughout this chapter it has been decided to apply Markov chains which

are discrete in time and space. By this approach, reactions can be presented in a

unified description via state vector and a one-step transition probability matrix.

Consequently, a process is demonstrated solely by the probability of a system to

occupy or not to occupy a state. In addition, complicated cases for which analytical

solutions are impossible are avoided.

3.1 MODELING THE PROBABILITIES IN CHEMICAL REACTIONS

Definitions. The basic elements of Markov chains associated with Eq.(2-

24) are: the system, the state space, the initial state vector and the one-step

transition probability matrix. Considering refs.[26-30], each of the elements will

be defined in the following with special emphasize to chemical reactions occurring

in a batch perfectly-mixed reactor or in a single continuous plug-flow reactor. In

the latter case, which may simulated by perfectly-mixed reactors in series, all

species reside in the reactor the same time.

The system is simply a molecule. The state of the system is the specific

chemical formula of the molecule, or what kind of species is the molecule. The

state space is the set of all states that a molecule can occupy, where a molecule is

occupying a state if it is in the state. For example, in the following irreversible

consecutive reaction:

Page 194: Applications of Markov Chains in Chemical Engineering

188

2 3 4

A| -^ A2 —> A3 -> A4 -> A5

type Ai = i of a molecule is regarded as the state of the system, i.e. a specific

chemical formula.

The state space SS, which is the set of all states a system can occupy, is

designated by:

55 = [Ai, A2, A3, A4, A5] = [1, 2, 3, 4, 5]

Finally, the reaction from state Ai to state Aj is the transition between the states.

The initial state vector is given by Eq.(2-22), i.e.:

S(0) = [si(0), S2(0), S3(0), ..., Sz(0)] (2-22)

Si(0) designates the probability of the system to occupy state A s i at time zero,

whereas S(0) designates the initial occupation probability of the states [Ai, A2, A3,

..., Az] by the system. Z designates the number of states, i.e. the number of

chemical species involved in the chemical reaction. In the context of chemical

reactions, as shown later, the probabilities Si(0) may be replaced by the initial

concentration of the molecular species and S(0) will contain the initial concentration

of all species. The one-step transition probability matrix is generally given by

Eqs.(2-16) and (2-20) whereas pjk represent the probability that a molecule Aj will

change into a molecule Ak in one step, pjj represent the probability that a molecule

Aj will remain unchanged within one step. In the following, general expressions

are derived for the determination of pjk and pjj for the model below.

The reacting system. Consider a chemically reacting system containing

the species Ai, A2, A3,...., Az. A chemical reaction among the species induces the

change in the state of the mixture. It is also assumed that a certain species Aj can

react simultaneously in several reactions, designated in the following by

superscripts (i), i = 1,2,..., R where R is the total number of reactions in which

Aj is involved. The following scheme of irreversible reactions by which Aj is

converted to products is assumed, where each set of reactions involving reversible

reactions can always be written according to the scheme below in order to apply the

following derivations.

Page 195: Applications of Markov Chains in Chemical Engineering

189

i s t leaction:

2ndieactiQii:

.4%.

. ^ ^ \ .

(1)

(2)

G)

products

products

ith reaction: a^ Ai + ... + ^ Aj + ... -^ ... + a^ Aj^ + ... (R)

Rih reaction: ....f>^... products (3-1)

. ( i ) ; . where a-* is the stoichiometric coefficients of species j in the ith reaction. The rate

of conversion of species j in the ith reaction based on volume of fluid V, i.e. rj*, is

defined by:

(i^ 1 dN: (^^ (i) G) A) J V dt J 1 ^ ' " y •••

(3-2)

where Nj are the number of moles of species j and Cj is its concentration, moles of .0) j/m . kj , in consistent units, is the reaction rate constant with respect to the

(i) conversion of species j in the rth reaction. In the case of a plus sign before kj , this

means moles of j formed/(s m ). The discrete form of Eq.(3-2) reads:

rf(n) = -kS^^C^(n)C^(n)... C[i(n)... (3-2a)

where the reaction rate and the concentrations Cj(n) refer to step n. In addition, the

conservation of the molar rates for all reacting species in the rth reaction in Eq.(3-1)

is governed by:

'1 „(i) Ji)

^k

(3-3)

Eq.(3-3), corresponding to the fth reaction, makes it possible to compute the

reaction rates of all species on the basis of TJ given by Eq.(3-2).

Page 196: Applications of Markov Chains in Chemical Engineering

190

Definition of pjj. If species j is converted simultaneously in R reactions,

the overall rate of conversion of species j is the following sum:

dCi V (i) -dt' = rj = 2^r}'^ (3-4)

i

where the summation is over all reactions in which species j is involved, i.e. i = 1,

2,..., R. The integration of the equation between t and t + At, taking into account

that rj ^ is negative according to Eq.(3-2), yields that:

Cj(n) - Cj (n+1) = ^ r f \n)At (3-5)

where Cj(n) and Cj(n+1) are, respectively, the concentration of species j in the

mixture at time interval t and t + At or at step n and n+1. rj (n) is the reaction rate

corresponding to step n, i.e. the concentrations in Eq.(3-2a) correspond to this

step. Here it is assumed that the variation of rj (n) between step n and n+1 is not

significant. If we consider the following quantity:

VCj(n+l) ^ amount of species j present at time t = (n+1) At . rri < \ VCj(n) " amount of species j present at time t = nAt "" v " ^)

where V is the volume of fluid, we may look on it as the probability pjj that a

molecule Aj will remain unchanged within one step. Thus, from Eq.(3-5) it

follows that:

q(n+l) Pjj~ C3(n)

= 1 - I £(n)l CjCn)

At (3-6)

The summation is over all reactions in which species j is converted to products, i.e.

i = 1, 2,..., R. Eq.(3-6) indicates that if At -^ 0, pjj = 1. Indeed Aj will remain

unchanged under such conditions. However, if At is large enough, the probability

that Aj will remain unchanged approaches zero, as expected. Thus, the above expression may, indeed, serve as a probability term, provided that 0 < pjj < 1 •

Page 197: Applications of Markov Chains in Chemical Engineering

191

Definition of pjk. In deriving an equation for the probability pjk

corresponding to the transition Aj -> Aj , we consider the ith reaction in Eq.(3-1).

In this case, dCj /dt = r j \ and an integration of it yields that:

Cf \n -H) -Cf (n ) rf(n)At

Cj(n) Cj(n)

amount of species j converted in the time interval At in reaction i amount of j available at time t = nAt (3-7)

Since the formation of the products in the ith reaction is associated with reaction

among all reactants, and each species contributes according to its stoichiometric

coefficient, we account for this effect in the determination of pjk by introducing the

ratio between the stoichoimetric coefficients. In addition, the transition probability

for Aj -^ Ay. depends also on the stoichiometric coefficient of Ak. Thus, on the

basis of Eq.(3-7), the probability of the transition Aj -> Aj for the ith reaction in

Eq.(3-1), i.e. Pjk, reads:

Pjk - % r . « A

^ Ji)

-J At Cj(n)

(3-8)

1=1

where N( ) is the number of the reacting species in reaction i. Indeed, the properties

of the above expression are appropriate for describing a transition probability,

namely, as At = 0, pjk = 0 while for relatively large values of At, py^ is increasing

with a limiting value not exceeding unity.

If the ith reaction in Eq.(3-1) is reduced to:

G)

(0. + a^A

then in order to comply with the results obtained by direct integration of the rate

equations, the transition probability for A; —> Aj must read:

Page 198: Applications of Markov Chains in Chemical Engineering

192

a)_ (0 /1 J r f \ n ) | •At (3-8a)

If species j is involved in several reactions, the total of which is R as in

Eq.(3-1), the following expression may be obtained on the basis of Eq.(3-5) for the

overall transition probability of Aj —> Ajj, i.e.:

Pjk=2 r a«) ^

N

3 (i)

Ir;»(n)|

Cj(n) At

1=1

(3-9)

The summation is over all reactions in which species j is involved, i.e. i = 1,2,...,

R. The justification for the above expressions for pjj and pjk is confirmed later by

the agreement with determinations made on the basis of integration of the rate

equations.

If all R reactions are of the type:

Ji) (0, af%-» ... -fa^% +

the transition probability is given by:

r?(n)| V «)/ 1 ^ I £ L ^ At (3-9a)

Finally, if species j undergoes a change solely by a single irreversible reaction

of the type:

... + ajAj + + a Ak + ...

we may ignore, for the sake of simplicity, some of the subscripts and superscripts

in Eq.(3-8), which is reduced to:

Page 199: Applications of Markov Chains in Chemical Engineering

/ Pjk = \ \ N

193

-^—!-At (3-10) Cj(n)

N is the number of the reacting species on the left-hand side of the above reaction.

If the above reaction is reduced to:

ajAj -^ ... + a Ak + ...

the transition probability for Aj —> Aj will read:

. 1X ri(n)

3.2 APPLICATION AND VERIFICATION OF THE MODELING

The modeling in section 3.1 will be applied to non-linear and linear reactions.

In section 3.2-1, all the stoichiometric coefficients of the species equal unity, i.e. aj

= 1. In section 3.2-2 part of the stoichiometric coefficients are different from unity,

aj 9t 1. In section 3.2-3 linear reactions are dealt with whereas in section 3.2-4,

linear-non linear reactions with aj t 1 are demonstrated in detail by case 3.13-6.

The validity of the results for Cj(n+1) computed by Eq.(3-20) on the basis of

Pjj and pjk predicted by Eqs.(3-6), (3-8) to (3-10), will be compared to those

obtained by direct integration of the kinetic equations.

3.2-1 Non-linear reversible reactions with all aj = 1 A procedure for determination of the transition probabilities is demonstrated

by a detailed treatment of the following non-linear irreversible reactions:

Page 200: Applications of Markov Chains in Chemical Engineering

194

Ai+A2 "^ A3 + A4 (3-1 la)

A2 + A3"^ A4 (3-1 lb)

k-2

The following rate equations are assumed satisfying Eq.(3-3):

n = dCi/dt = - kiCiC2 + k.iC3C4 (3-12a)

12 = dC2/dt = - ki C1C2 - k2C2C3 + k. 1C3C4 + k.2C4 (3-12b)

r3 = dCs/dt = - k2C2C3 - k.iC3C4 + kiCiC2 + k.2C4 (3-12c)

14 = dC4/dt = - k-2C4 - k.iC3C4 + kiCiC2 + k2C2C3 (3-12d)

1st step: Transformation of Eqs.(3-11) into the following set of irreversible

reactions which best demonstrate the transition between the states:

(3-13a)

(3-13b)

(3-13c)

(3-13d)

2nd step: Determination of the reaction rates for Eqs.(3-13) on the basis of

Eqs.(3-12). The definition in Eq.(3-2) yields for species j = 1 reacting in the first

reaction, i= 1, that:

T\^\D) = - kS*^Ci(n)C2(n) = - kiCi(n)C2(n) (3-14a)

where k] = kj according Eq.(3-13) for i = 1. Similarly, for species j = 2 reacting

in the first and third reactions, i = 1, 3:

i = l :

i = 2:

i = 3:

i = 4:

"1

Ai + A2 - A3 + A4

A3 + A4 - Ai + A2

A2 + A3 -> A4

k_2

A4 A2 + A3

Page 201: Applications of Markov Chains in Chemical Engineering

195

i[%) = 4^^Ci(n)C2(n) = - kiCi(n)C2(n)

4^\n) = 4^^C2(n)C3(n) = - k2C2(n)C3(n) (3-14b)

wherek2 =kj accordingEq.(3-13)fori= 1 andk2 = k2 for i = 3. For species j

= 3 reacting in the second and third reactions, i = 2, 3:

r^^^n) = - k f C3(n)C4(n) = - k_iC3(n)C4(n)

r'i\n) = - 4^^C2(n)C3(n) = - k2C2(n)C3(n) (3-14c)

For species j = 4 reacting in the fourth reaction, i = 4:

r f (n) = - ki^^C3(n)C4(n) = - k_iC3(n)C4(n)

rj*\n) = - kJ*^C4(n) = - k_2C4(n) (3- 14d)

3rd step: Determination of the probabihties pjj. Applying Eqs.(3-6) and (3-

14a) yields for species j = 1 where i = 1 that:

pil = l-kiC2(n)At (3-15a)

For species j = 2, converted according to Eqs.(3-13), i = 1,3, Eqs.(3-6) and (3-

14b) yield:

P22 = 1 - [kiCi(n) + k2C3(n)]At (3-15b)

For species j = 3, converted according to Eqs.(3-13), i = 2, 3, Eqs.(3-6) and (3-

14c) yield:

P33 = 1 - [k.iC4(n) + k2C2(n)]At (3-15c)

For species j = 4, converted according to Eqs.(3-13), i = 2, 4, Eqs.(3-6) and (3-

14d) yield:

P44 = 1 - [k-iC3(n) + k.2]At (3-15d)

Page 202: Applications of Markov Chains in Chemical Engineering

196

4th step: Determination of the probabilities pjk. As observed in the

reactions given by Eqs.(3-13):

pl2 = 0 and P21 = 0 because Ai is not converted to A2 and vice versa.

Applying Eqs.(3-9) and (3-14a) to species j = 1, converted to species j = 3

and j = 4 according to Eqs.(3-13), i = 1, noting that NW = 2, yields:

P13 = (l/2)kiC2(n)At pi4 = (l/2)kiC2(n)At (3-16a)

Applying Eqs.(3-9) and (3-14b) to species j = 2 which is converted to species

j = 3 according to Eq.(3-13), i = 1, and to species j = 4 according to Eq.(3-13), i =

1, 3, noting that N(l) = N(^) = 2, yields:

P23 = (l/2)kiCi(n)At P24 = (l/2)[kiCi(n) + k2C3(n)]At (3-16b)

Applying Eqs.(3-9) and (3-14c) to species j = 3 which is converted to species

j = 1 according to Eq.(3-13), i = 2, to species j = 2 according to Eq.(3-13), i = 2,

and to species j = 4 according to Eq.(3-13), i = 3, noting that N(2) = N(^) = 2,

yields:

P31 = (l/2)k.iC4(n)At P32 = (l/2)k.iC4(n)At

P34 = (l/2)k2C2(n)At (3-16C)

Finally, applying Eqs.(3-9) and (3-14d) to species j = 4 which is converted to

species j = 1 according to Eq.(3-13), i = 2, to species j = 2 according to Eq.(3-13),

i = 2, 4, and to species j = 3 according to Eq.(3-13), i = 4, noting that N(2) = 2 and

N W = 1 , yields:

P41 = (l/2)k.iC3(n)At P42 = [(l/2)k.iC3(n) + L2]At

P43 = k.2At (3-16d)

The above probabilities may be grouped in the matrix given by Eq.(3-17). It

should be noted that Eq.(2-18) is not satisfied along each row because the one-step

transition probabilities pjk depend on time n. This is known as the non-

homogeneous case defined in Eqs.(2-19) and (2-20), due to non-linear rate

equations, i.e. Eqs.(3-12).

Page 203: Applications of Markov Chains in Chemical Engineering

197

P =

1

2

3

4

Ai = l

1- kiC2(n)At

0

(l/2)k.iC4(n)At

(l/2)k.iC3(n)At

A2 = 2

0

l - [kiCi(n) +

k2C3(n)]At

(l/2)k.iC4(n)At

[(l/2)k.iC3(n) +

k.2]At

A3 = 3

(l/2)kiC2(n)At

(l/2)kiCi(n)At

l-[k.iC4(n) +

k2C2(n)]At

k.2At

A4 = 4

(l/2)kiC2(n)At

(l/2)[kiCi(n) +

k2C3(n)]

(l/2)k2C2(n)At

1- [k.iC3(n) +

k-2]At

(3-17)

Verification of tlie model. Several assumptions were made in section

3.1 which led to Eqs.(3-6), (3-8) to (3-10) for the determination of pjjj and pj^. For

the reactions given by Eqs.(3-13) the results are summarized in the matrix given by

Eq.(3-17). The validity of the results will be tested by writing the Euler integration

algorithm for the differential equations, Eqs.(3-12), which describe the reaction

mechanisms.

Integration of Eq.(3-12a) yields after a few manipulations that:

Ci(n+1) = Ci(n)pii + C2(n)p2i + C3(n)p3i + C4(n)p4i (3-18a)

where p2i = 0. The other pij's, as well as those for the results below, are given in

Eqs.(3-15a to d) and (3-16a to d) which are summarized in the matrix given by

Eqs.(3-17). Integration of Eq.(3-12b) yields:

C2(n+1) = Ci(n)pi2 + C2(n)p22 + C3(n)p32 + C4(n)p42

where pi2 = 0. Integration of Eqs.(3-12c) and (3-12d) yields:

C3(n+1) = Ci(n)pi3 + C2(n)p23 + C3(n)p33 + C4(n)p43

C4(n+1) = Ci(n)pi4 + C2(n)p24 + C3(n)p34 + C4(n)p44

Equations (3-18) reveals the following characteristics:

(3-18b)

(3-18c)

(3-18d)

Page 204: Applications of Markov Chains in Chemical Engineering

198

a) The equations are a function of the transition probabilities pij and pik

detailed in the matrix given by Eq.(3-17).

b) Each of the Eqs.(3-18a to d) obey Eq.(2-23) for a number of states Z = 4

as well as Eq.(2-24). Thus, the following equalities may be obtained:

Sj(n) = Cj(n); Sj(n+1) = C/n+l)

S(n) = C(n) = [Ci(n), CjW, €3(0),..., € > ) ] (3-19)

where C(n) may be looked upon as the state vector of the system at time nAt (step

n). In addition, the initial state vector reads:

S(0) = C(0) = [Ci(0), €2(0), €3(0),..., Cz(0)] (3-19a)

c) Eqs.(3-18a to d) indicate that each Cj(n+1) is a result of the product of the

row vector C(n), defined in Eq.(3-19), by the square matrix P defined in Eq.(3-

17), i.e.:

z

Cj(n+l) = 2^Cj(n)pjk

j=l

C(n+1) = C(n)P (3-20)

where Z is the number of states.

3.2-2 Non-linear reversible and irreversible reactions with a j ^ l

Example a: Aj -^ 2A2 (3-21)

for which:

ri = dCi/dt = - kiCi (3-22a)

It follows from Eq.(3-3) that:

r2 = dC2/dt = 2kiCi (3-22b)

Integration of the above equations yields:

Ci(n+1) = Ci(n)[l - kiAt] (3-23a)

C2(n+1) = Ci(n)[2kiAt] + C2(n) (3-23b)

Page 205: Applications of Markov Chains in Chemical Engineering

199

which can be arranged on the basis of Eq.(3-20) in the following matrix form:

Ai = 1 A2 = 2

P =

1 - kiAt 2kiAt

0 1

Applying Eqs.(3-6) and (3-10) yields identical probabilities.

k,

Example b: Ai "^ 2A2

for which:

ri = dCj/dt = - kjCi + kzCj

It follows from Eq.(3-3) that:

T2 = dCz/dt = 2kiCi - 2k2C2

Integration of the above equations gives:

Ci(n+1) = Ci(n)[l - kiAt] + C2(n)[k2C2(n)At]

C2(n+1) = Ci(n)[2kiAt] + C2(n)[l - 2k2C2(n)At]

yielding the following matrix:

P =

Ai = l 1 - kiAt

A2 = 2 2kiAt

k2C2(n)At l-2k2C2(n)At

(3-24)

(3-25)

(3-26a)

(3-26b)

(3-27a)

(3-27b)

(3-28)

Applying Eqs.(3-6), (3-10) and for p2i Eq.(3-10a), yields identical

probabilities while Eq.(3-25) was expressed as the following two irreversible

reactions:

Aj -> 2A2 and 2A2 -> Aj

Page 206: Applications of Markov Chains in Chemical Engineering

200

Example c: 2Ai -*• A2

for which:

ri = dCi /d t - -k iCf

gives from Eq.(3-3) that:

r2 = dC2/dt- O.SkjCi

Thus, by integration it is obtained that

Ci(n+l) = Ci(n)[l-kiCi(n)At]

C2(n+1) = Ci(n)[0.5kiCi(n)At] + C2(n)

hence:

Ai = l A2 = 2

1 - kiCi(n)At 0.5kiCi(n)At 1

P =

0 1

Identical pnababihties are obtained from Eqs.(3-6) and (3-10a) for pu-

Example d: 2Ai "* A2

for which:

ri = dCj/dt - - kiCi + kjCj

r2 = dCj/dt - O.SkJCi - O.SkjCj

yields by integration that:

Ci(n+1) = Ci(n)[l - kiCi(n)At] + C2(n)[k2At]

C2(n+1) = Ci(n)[0.5kiCi(n)At] + C2(n)[l - 0.5k2At]

Ai = 1 A2 = 2

1 - kiCi(n)At 0.5kiCi(n)At

P =

k2At 1 - 0.5k2At

(3-29)

(3-30a)

(3-30b)

(3-3 la)

(3-3 lb)

(3-32)

(3-33)

(3-34a)

(3-34b)

(3-35a)

(3-35b)

(3-36)

Page 207: Applications of Markov Chains in Chemical Engineering

201

The above probabilities are identical to those computed from Eqs.(3-6), (3-

10) and Eq.(3-10a) for pi2 considering Eq.(3-33) as the irreversible set kj kj

2Ai -> A2 and A2 -» lAj .

3.2-3 Linear reactions Consider the following reactions:

Ai A2 "* A3 -» A4 (3-37)

The kinetics of the reactions satisfying Eq.(3-3) is given by the following

expressions:

ri = dCi/dt = - kiCi + k.iC2

r2 = dC2/dt = - k2C2 - k.iC2 + kiCi + k.2C3

rs = dCa/dt = - ksCs - k.2C3 + k2C2

T4 = dC^dt = k3C3

(3-38a)

(3-38b)

(3-38c)

(3-38d)

1st step: Transformation of Eqs.(3-21) into a set of irreversible reactions as

follows: ki

1 = 1: A i ^ A 2 (3-39a)

= 2: A2->Ai

= 3: A2 — A3

k-2 = 4: A3-J.A2

(3-39b)

(3-39c)

(3-39d)

= 5: A3 —A4 (3-39e)

2nd step: Determination of the reaction rates for Eqs.(3-39) on the basis of

Eqs.(3-38). Following the definition in Eq.(3-2) yields:

Page 208: Applications of Markov Chains in Chemical Engineering

202

T\^\n) = - kS^^Ci(n) = - kiCi(n) (3-40a)

Similarly:

T^i\n) = - k^^^C2(n) = - k_iC2(n) (3-40b)

r^^\n) = - k^^^C2(n) = - k2C2(n) (3-40c)

r ' ^n) = - k^%3(n) = - k_2C3(n) (3-40d)

r^^^n) = ~ kfc^in) = - k3C3(n) (3-40e)

3rd step: Determination of the probabilities pjj. Applying Eqs.(3-6) and (3-

40a) yield for species j = 1 where i = 1 that:

pil = l -k iAt (3-41a)

For species j = 2 which is converted according to Eqs.(3-39b, c), i = 2, 3, Eqs.(3-

6) and (3-40b, c) yield:

P22=l-[k-i+k2]At (3-41b)

For species j = 3 which is converted according to Eqs.(3-39d, e), i = 4, 5, Eqs.(3-

6) and (3-40d, e) yield:

P33=l-[k-2 + k-3]At (3-41C)

For species j = 4, formed according to Eq.(3-39e), i = 5, and for remaining at this

state:

P44 = 1 (3-41d)

4th step: Determination of the probabilities pj^. Applying Eqs.(3-9) or (3-

10) and (3-40a) to species, j = 1 which is converted to species j = 2 according to

Eq.(3-39a), i = 1, noting that N(l) = 1 and that A] is not converted to A3 and A4 in

one step, yields:

Page 209: Applications of Markov Chains in Chemical Engineering

pi2 = kiAt pi3 = pi4 = 0

203

(3-42a)

Applying Eqs.(3-9) or (3-10) and (3-40b) to species j = 2 which is converted to

species 1 according to Eq.(3-39b), i = 2, to species 3 according to Eq.(3-39c), i =

3, noting that N(2) = 1 and N(3) = 1, yields:

P21 = k.iAt P23 = k2At P24 = 0 (3-42b)

Applying Eqs.(3-9) or (3-10) and (3-40c) to species j = 3 which is converted

to species j = 2 according to Eq.(3-39d), i = 4, to species j = 4 according to Eq.(3-

39e), i = 5, noting that N(4) = N(5) = 1, yields:

P31 = 0 P32 = k.2At p34 = k3At (3-42c)

Species j = 4, formed according to Eq.(3-39e), i = 5, remains in its state,

thus:

P41 = 0 P42 = 0 P43 = 0 (3-42d)

The above probabilities may be grouped in the matrix given by Eq.(3-43). It

should be noted that Eq.(2-18) is satisfied along each row because the one-step

transition probabilities pjk are independent of the time n. This is known as the

time-homogeneous case defined in Eqs.(2-14a) and (2-16), due to the fact that the

rate equations (3-23) are linear.

1

2

3

4

A i = l

1 -kiAt

k-i At

0

0

A2 = 2 kiAt

l - [k - i+k2]At

k-2At

0

A3 = 3

0

k2At

l - [k-2 + k3]At

0

A4 = 4

0

0

ksAt

1

p =

(3-43)

Verification of the model. Integration of Eq.(3-38a to d) yields after a

few manipulations Eqs.(3-18a to d). The pij's are given by Eqs.(3-41a to d) and

(3-42a to d), which are summarized in the matrix governed by Eq.(3-43).

Page 210: Applications of Markov Chains in Chemical Engineering

204

3.2-4 Linear-non linear reactions with aj ^ l Example 3.13-6 (chapter 3.13) is presented in details demonstrating also the

derivation of the kinetic equations satisfying Eq.(3-3). This example should be

studied thoroughly since it contains important aspects of applying the equations for

calculating pjj and pj] .

3.3 MAJOR CONCLUSIONS AND GENERAL GUIDELINES FOR APPLYING THE MODELING

The major conclusions drawn from treating the above reactions, and many

others reported in the following without detailed derivations, are:

a) The results obtained by the Euler integration are in complete agreement

with the results obtained by the model presented in section 3.1 yielding Eqs.(3-6)

and (3-8) to (3-10a) for predicting the probabilities pjj and pijk. Thus, one may

apply each of the methods, depending on his conveniece. However, by gaining

enough experience, one starts to Teel' that the method based on the Markov chains

is easier and becomes 'automatic' to apply. In addition, chemical reactions are

presented in unified description via state vector and a one-step transition probability

matrix.

b) Reversible reactions should be transformed into a set of irreversible

reactions.

c) The above reactions, treated in detail, provide the reader with a good

introduction for applying the probabilities pjj and pjk to chemical reactions.

3.4 APPLICATION OF KINETIC MODELS TO ARTISTIC PAINTINGS

Prior to modeling of chemical reactions in the next sections, it is interesting to

demonstrate how simple kinetic models can also be applied to artistic paintings.

No reaction

The first example ^plies to Fig.2-52. The painting in the figure.

Development II by Escher [10, p.276], demonstrates the development of reptiles,

and at first glance it seems that their number is increasing along the radius.

Page 211: Applications of Markov Chains in Chemical Engineering

205

Although their birth origin is not so clear from the figure, it was possible, by counting their number along a certain circumference, to conclude that it contains exactly eight reptiles of the same size. This number is independent of the distance from the center. Thus, if we designate by Ci the number of reptiles at some distance from the origin, it follows that

dCj Ti = —^ = 0 or alternatively in a discrete form Ci(n+1) = Ci(n)

This result indicates the absence of a chemical reaction. Although the reptiles become fatter versus the number of steps (time), their number is unchanged.

zero order reaction The second example refers to Fig.3-la showing various kinds of "winged

creatures" in a drawing of a ceiling decoration designed by Escher in 1951 [10, p.79] for the Philips company in Eindhoven. If the number of the "winged creatures", designated as the "concentration" Ci, is counted along the lines corresponding to steps (time) n = 0, 1, 2, ... shown schematically in Fig.3-lb for cases 1 to 3, the results summarized in Table 3-1 are obtained. The general trend observed is a decrease versus time of the "concentration".

Page 212: Applications of Markov Chains in Chemical Engineering

206

iTi ^ 1 ^ C i ^ ^ W ^ ^ ^^^ r^ 1

!ic4 ^.-:

r : |r #" f ?l | f -€ 1^ fT, -*

Fig.3-la. "Winged creatures'* demonstrating zero order reaction (M.C.Escher "Ceiling Decoration for Philips" © 1998 Cordon Art B.V. - Baam - Holland. All

rights reserved)

Page 213: Applications of Markov Chains in Chemical Engineering

207

0 1 t 9 A 4

n = 9 10 11

U - ^ > — D

-e—©

n = o o—a s—n o

1 1

1

1 A ^ f

: a-1 {

«—«—^-•-' e

T 1

1

• ±

Casel Case 2 Case 3

Fig.3-lb. Three configurations for determination of the "winged

creatures concentration" Ci

Table 3-1. "Concentration" Ci of "winged creatures" versus time n

time n

case 1*, Ci(n)

case 2*, Ci(n)

[case 3*, Ci(n)

0 1 2 3 4 5

12 11 10 9 8 7

25 23 21 19 17 15

48 40 32 24 16 8

6 7

6 5

13 11

8

4

9

9

3

7

10

2

5

111 1

3

* see Fig.3-lb

It should be noted that in case 1 of Fig.3-lb, the hnes for n = 0 and 11 correspond

to twelve butterflies and a single fly, respectively, in Fig.3-la. In case 2 these lines

correspond to twenty five and three "winged creatures", respectively, whereas in

case 3 the lines correspond to forty eight and eight "winged creatures",

respectively.

In order to fit the concentration data Ci (number of "winged creatures" along

a line) versus the time n reported in Table 3-1, the common approach of fitting

experimental data to a kinetic model is applied. Thus, the simplest model of a zero k

order reaction is tested which corresponds Ai -> A2. The rate equation reads

Page 214: Applications of Markov Chains in Chemical Engineering

208

dCi

Eqs.(3-6) and (3-8) are applied for determination of the probabilities which

yield the following matrix:

Ai = l A2 = 2

l"

P =

2

Thus,

1 - [k/Ci(n)]At [k/Ci(n)]At

0 1 (3-45)

Ci(n+l) = Ci(n)-kAt

C2(n+1) = kAt + C2(n) (3-46)

Fitting the data in Table 3-1 by Eq.(3-46) for At = 1, yields the following

equations corresponding to Fig.3-lb:

Case 1: Ci(n+1) = Ci(n) - 1; C2(n+1) = 1 + C2(n) where Ci(0) = 12; C2(0) = 0

Case 2: Ci(n+1) = Ci(n) - 2; C2(n+1) = 2 + C2(n) where Ci(0) = 25; C2(0) = 0

Case 3: Ci(n+1) = Ci(n) - 8; C2(n+1) = 8+ C2(n) where Ci(0) = 48; C2(0) = 0

The excellent fit to Eqs.(3-30) of the data given in Table 3-1, indicates that the

concentration-time dependence of the "winged creatures" in Fig.3-la according to

the configurations depicted in Fig.3-lb, obeys a model of zero order reaction. The

significance of the quantities C2(n) is as follows. Since Ci(n) is decreasing versus

time, i.e. the number of the "winged creatures", the conservation of mass requires

that they are found in state A2 according to the reaction A] -^ A2.

mth order reaction

The third example refers to Fig.3-2 which is a woodcut by Escher [10,

p. 118, 325] showing moving fish of changing size. Here Escher demonstrated an

infinite number by a gradual reduction in size of the figures, until reaching the Umit

of infinite smallness on the straight side of the square. If the number of fish along

the square perimeter, designated as "concentration" Ci, is counted, the obtained

results are summarized in Table 3-2. Fig.3-lb, case 3, shows schematically the

Page 215: Applications of Markov Chains in Chemical Engineering

209

fish orientation along a squre which was counted, where each circle symbolizes a

fish. Also, along a certain square, each fish is located exactly behind (or before)

the other, and all are of the same size. The case of n = 0 corresponds to the the

square located almost at the sides of the square where n = 6 corresponds to the

most inner square comprising of four fish.

, ^>k<!k> • • * . * * . ••>>*««>*»

4 4:

Fig.3-2. Fish orientation for demonstrating an mih order reaction

(M.C.Escher "Square limit" © 1998 Cordon Art B.V. - Baarn - Holland. All rights reserved)

Table 3-2. "Concentration" Ci versus time of moving fish along the square perimeter

1 timen

Ci(n)

1 Ci.calc(n)*

0

760

1

376

399

2

184

185

3

88

84

4

40

37

5

16

15

6 1 4

5 * rounded values computed by Eq.(3-49)

Page 216: Applications of Markov Chains in Chemical Engineering

210

In order to fit the "concentration" data Ci (number of moving fish along a

square) versus the time n reported in Table 3-2, an mth order reaction is tested k

corresponding to Ai -». A2. The rate equation reads:

dCi dt kC^ (3-47)

where the apphcation of Eqs.(3-6) and (3-8) yields the following transition matrix:

Al = 1 A2 = 2

1 - kCim-l(n)At kCim-l(n)At

P =

0

Thus,

1 (3-48)

Ci(n+1) = Ci(n)[l - kAtCr^n)]

C2(n+1) - kACf(n) + CjCn) (3-49)

Fitting the data in Table 3-2 by Eq.(3-49) for At = 1, which was modified to the

following equation [Ci(n) - Ci(n+l)] - kC7(n), yields m = 0.904 and k = 0.896

with a mean deviation of 8.3% between calculated data with respect to coimted

values in Fig.3-2. The above examples indicate that the application of kinetic

models to artistic paintings has been successful.

3.5 INTRODUCTION TO MODELING OF CHEMICAL REACTIONS

In the following, a solution generated by the discrete Markov chains is

presented gr^hically for a large number of chemical reactions of various types.

The solution demonstrates the transient response Cj-nAt and emphasizes some

characteristic behavior of the reaction. The solution is based on the transition

probability matrix P obtained on the basis of the reaction kinetics by applying

Page 217: Applications of Markov Chains in Chemical Engineering

211

Eqs.(3-6), (3-8) to (3-lOa) for computing the probabilities pjj and pjk- It should be

emphasized that the rate equations for the kinetics were tested to satisfy Eq.(3-3).

In order to obtain the transient response, Eq.(3-20) is applied where the initial

state vector S(0) is given by Eq.(3-19a). In each case, the magnitude of S(0) are

the quantities on the Ci axis of the response curve corresponding to t = 0. An

important parameter in the computations is the magnitude of the interval At. This

parameter has been chosen recalling that pjj and pjk should satisfy

0 < Pjj and pj] < 1 on the one hand, and that Cj versus nAt should remain

unchanged under a certain magnitude of At, on the other. In addition, a comparison

with the exact solution has been conducted in many cases, which made it possible

to evaluate the accuracy of the solution obtained by Markov chains. The quantities

reported in the comparison are the maximum deviation, Dmax* and the mean

deviation, Dmean- Oi the basis of these comparisons, a representative value of At =

0.01 is reconmiended, which is the parameter of Markov chains solution. Finally,

it should be emphasized that by equating the reaction rate constant (one or a few) to

zero in a certain case, it is possible to generate numerous interesting situations.

The reactions are presented according to the following categories:

1) Single step irreversible reaction.

2) Single step reversible reaction.

3) Consecutive-irreversible reactions.

4) Consecutive-reversible reactions.

5) Parallel reactions: single and consecutive-irreversible reaction steps.

6) Parallel reactions: single and consecutive-reversible reaction steps.

7) Chain reactions.

8) Oscillating reactions.

The following definitions are applicable [31, 32]:

Consecutive chemical reactions are those in which the initial substance and all

the intermediates products can react in one direction only, i.e.:

Parallel chemical reactions are those in which the initial substance reacts to

produce two different substances simultaneously, i.e.:

Page 218: Applications of Markov Chains in Chemical Engineering

212

- . < ^3

Reversible reactions are those in which two substances entering a single

simple consecutive chain reaction interact in both forward and backward directions,

i.e.:

A^ ^ A^ A^ A^

Conjugated reactions are two simultaneous reactions in which only one

substance Ai is common to both, i.e.:

\ + ^2 ^ ^ 4

Aj + A3 • A5

All three substances Ai, A2 and A3 must be present in the reaction mass in order

for both reactions to take place concurrently.

Consecutive-reversible reactions are those in which two or more reactions,

each of different type, occur simultaneously, for example:

5

Parallel-consecutive reactions belong to the mixed type which have the

characteristics of both parallel and consecutive reactions. The following example

comprises two parallel chains, each composed of three simple reactions:

<

A parallel-consecutive reaction becomes complex when species that belong to

different chains interact as shown below:

Page 219: Applications of Markov Chains in Chemical Engineering

213

Chain reactions. If the initial substance and each intermediate reaction

product interact simultaneously with different substances and in different

directions, such processes are known as chain reactions. For example, the

following scheme is a chain reaction in two stages. Other types of chain reactions

are described in [22].

'^

3.6 SINGLE STEP IRREVERSIBLE REACTION

3.6-1 where

r, = -kCV

(3.6-1)

(3.6-la)

By applying Eqs.(3-6) and (3-10), the following one step transition probability

matrix is obtained:

1 2

1 - kCim-i(n)At kCi'n-i(n)At

P =

0 1 (3.6-lb)

Page 220: Applications of Markov Chains in Chemical Engineering

214

where 1, 2 stand for states (chemical species) A] and A2, respectively. From Eqs.(3-19a), (3-20), one obtains that:

Ci(n+1) = Ci(n)[l - kAtCr^n)]

C2(n+1) = kAtCf (n) + C2(n) (3.6-lc)

The variation of Ci against t = nAt for the initial state vector C(0) = [Ci(0), C2(0)] = [1, 0] is depicted in Figs.3.6-1 (a to d) for different values of the parameter m = 0, 0.5, 1 and 3 where also the effect of the reaction rate constant k is demonstrated.

u

Fig.3.6-la. Ci versus t demonstrating the effect of k for m = 0 in Eq.(3.6-la)

1

0.8

0.6

0.4

0.2

0

U

h-

u

1

i = l \

ly

1

1 1 k = 0.5

1 1

H

H

0.5 1.5 2 0

Fig.3.6-lb. Ci versus t demonstrating the effect of k for m = 0.5 in Eq.(3.6-la)

Page 221: Applications of Markov Chains in Chemical Engineering

215

u

1

0.8

0.6

0.4

0.2

0

-

r

h-

1 1

iX

1 1

1 y

y ^ k = 0.5-

^"""^-cr

1

1 /

l \ /

7

1

1 1

k = l -

1 1 0.5 1.5 2 0 0.5 1.5

Fig.3.6-lc. Ci versus t demonstrating the effect of k for m = 1 in

Eq.(3.6-la)

u

1

0.8

0.6

0.4

0.2

0

u

H

1

i=^^^^^^

1

1 I k = 0.5

1 1

-

-•

Y

1

iV

'y

k = 1

1

-

0.5 1.5 2 0 0.5 1.5

Fig.3.6-ld. Ci versus t demonstrating the effect of k for m = 3 in

Eq.(3.6- la)

It should be noted that exact solutions for the above models are available in

refs.[32, vol.1, p.361; 34, pp.4-5, 4-6]. For At = 0.01, the agreement between

the Markov chain solution and the exact solution is D^ax = 0.4% and Dmean =

0.2%.

3>6-2 ajAj + ^2^1 ~^ ^3^3

where

ri = ~ kCjC^' r2 = - rkcjc^' r = a2/ai

(3.6-2)

(3.6-2a)

The following one step transition probabiUty matrix is obtained:

Page 222: Applications of Markov Chains in Chemical Engineering

216

1

P = 2

3

l-kc'f'(n)C5'(n)At 0 kRcV'(n)C5'(n)At

l-rkC\(n)C^"'(n)At ARC',(n)C^'(n)At

(3.6-2b)

0

0 0 1

where R = 1/(1+ r) and r = aa/ai. 1, 2, 3 stand for states Ai, A2 and A3, respectively.

By applying Eqs.(3-19a), (3-20), one obtains that:

Ci(n+1) = Ci(n)[l-kCl"kn)C^(n)At]

C2(n+1) = C2(n)[l-rkcl(n)C^"'(n)At]

(3.6-2C)

-,1-1 C3(n+1) = Ci(n)[kRCr (n)C^(n)At] + C2(n)[r'kRC;(n)Cr (n)At] + C3(n)

The following cases were explored:

3.6-2a For the initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [1,0.5, 0]:

u

1

0.8

0.6

0.4

0.2

0

r ^

1

J = l

3 , ^ - ^

1

1 r = l

1

1 -H

H

-

1 0.5 1.5

1/ V 2 I

2 0 0.5 1.5

Fig.3.6-2a. Ci versus t demonstrating the effect of r for I = 0, m = 3/2 and k = 5 in Eq.(3.6-2a)

For At = 0.04, the agreement between the Markov chain solution and the

exact solution [32, vol.l, p.361] is Dmax = 1-9% and Dmean = 0.3%.

Page 223: Applications of Markov Chains in Chemical Engineering

217

3>6-2b For an initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [2, 3,0]:

3

2.5

2

u" 1.5

1

0.5

0

_ \ 2

\ i = l

r y 3

- / V

y' y ^

1

1

^ - "

-..., ^ -^^^-^7-^-—-^

0.5 1 1.5 0 t t

Fig.3.6-2b. Ci versus t according to Eq.(3.6-2a).

ri = - kCi l /2c2 (left), ri = - kCiCil'^ (right) for k = 1, r = 1

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [32, vol.1, p.361] is Dmax = 1-2% and Dmean = 0.6%.

3.6-3 where

Aj + A2 + A3 -^ A4

rj - r2 - r3 - - r4 - - kC]C2C3

(3.6-3)

(3.6-3a)

yields the following transition probability matrix

P =

2 0

3 0

4 ^kC2(n)C3(n)At

1 1 -

kC2(n)C3(n)At

0 ^~ 0 |kC,(n)C3(n)At kC,(n)C3(n)At ^

0

0

0 . ^ / L . ^ . T' C,(n)C2{n)At kC,(n)C,(n)At 3 '•

0 0 (3.6-3b)

Page 224: Applications of Markov Chains in Chemical Engineering

218

Ci(n+1) is obtained by applying Eq.(3-20) where the effect of the initial state

vectors C(0) = [Ci(0), C2(0), €3(0), C4(0)] = [1, 2, 3, 0] and [3, 2, 1, 0] is

demonstrated in Fig.3.6-3a.

3

2.5

2

0 - 1.5

1

0.5

0

\ 1

^3

^ \ ^ ^ v

. \ -Lv< 1/ 1

1

4x

/

\

i"" "

1

y

1

>.--I=iz=:^

^ 1

A

-

^ —-0.2 0.4 0.6 0.8

t

X J

""'•---

p\jz 1

k ,

1 1 i

^

— z j

-

-L ' ' ' 1 1 1 1

1 0 0.2 0.4 0.6 0.8 t

Fig.3.6-3. Ci versus t demonstrating the effect of Ci(0) for k = 1

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [31, p.20] is Dmax = 2.3% and Dmean = 1-5%. For At = 0.005,

Dmax = 1.4% and Dmean = !%•

3.6-4 where

rj = - kjCj/Cl + kjCi)

yields the following transition probability matrix:

(3.6-4)

(3.6-4a)

P =

1 2 1 - [kj/(l + k2Ci(n)]At [k,/(l + k2Ci(n)]At

0 1 (3.6-4b)

Fig.3.6-4 demonstrates the effect of the reaction rate constants ki and k2 on

the species concentration distribution for the initial state vector C(0) = [Ci(0),

C2(0)] = [1, 0].

Page 225: Applications of Markov Chains in Chemical Engineering

219

u

1

0.8

0.6

0.4

0.2

0 (

1

0.8

0.6

0.4

0.2

0

\ r~

ly^

/ ^ 1 ) 0.5

V 1

i = l \

\y V , 0 0.5

1 k = l , k = 1

1 2

^ "

1 1 t

i k =5,k =5^

1 ^x

/

1 1 t

"~1

1 1.5

^'^

1.5

-

<

2

^

' y / , 0 0.5

2 (

\ y--V 7\ r V ) 0.5

\ = l ,k =0.1

2

1 1 t

= 5,k =0.5 2

1 t

~T

1 1.5

1

1 1.5

- : :

-

H

A

/ •

Fig.3.6-4. Ci versus t demonstrating the effect of ki and k2

3.7 SINGLE STEP REVERSIBLE REACTIONS

3.7-1 Ai ^ A 2 (3.7-1)

where

Tj — — r2 — — ( k j C j — )£.'^'2) (3.7-la)

By applying the approach detailed in section 3.2-1, i.e., treating the

reversible reactions as two irreversible ones demonstrated in Eqs.(3-10a) and (3-

10b), the following transition probability matrix is obtained:

P =

1 2 1 - kjAt kjAt

koAt 1 - k^At (3.7-lb)

Page 226: Applications of Markov Chains in Chemical Engineering

220

Thus, from Eqs.(3-19a), (3-20), one obtains that:

Ci(n+1) = Ci(n)[l - kjAt] + C2(n)[k2At]

C2(n+1) = Ci(n)[kiAt] + C2(n)[l - k2At] (3.7-lc)

yielding the following curves for the initial state vector C(0) = [Ci(0), C2(0)] = [1, 0]:

u

1

0.8

0.6

0.4

0.2

0

u

u

V

1

i = r\^

2 /

1 k = 1

2

^ '

1

1

1

V^ ' f

-

-

> , T

1 1 k =5

2

1

-J

-J

A

0 0.5 1 1.5 2 0 0.5 1 1.5 2 t t

Fig.3.7-1. Ci versus t demonstrating the effect of k2 for ki = 1

At steady state, the results verified the relationship which follows from

Eq.(3.7-la), i.e., (C2/Ci)eq.= ki/k2. For At = 0.01, the agreement between the

Markov chain solution and the exact solution [34, p.4-7; 48, p.20; 49, p.85] is

Dmax = 0.3% and Dmean = 0.1%.

3.7-2 Ai "^ 2A2 (3.7-2)

k2

The transition matrix based on Eqs.(3-26a,b) was developed before and is

given by Eq.(3-28). The transient response of Ci and C2 for the initial state vector

C(0) = [Ci(0), C2(0)] = [1, 0] is demonstrated in Fig.3.7-2. At equilibrium, the

results verified the relationship which follows from Eqs.(3-26a,b), i.e.,

(C2/Ci2)eq = ki/k2 where no analytical solution is available for comparison.

Page 227: Applications of Markov Chains in Chemical Engineering

o

1

0.8

0.6

0.4

0.2

0 0.5 1.5 2 0 0.5 1.5

221

\ 1

i=^'">^

- ;

/ :

1 1 k =1

2

^ ^

1 1

V 1

1

- 1^ -

1 ,

1 1 k =5

2

1 1

Fig.3.7-2. Ci versus t demonstrating the effect of k2 for ki = 1

according to Eqs.(3-26a,b)

3.7-2.1 Ai ^ A2 + A3 (3.7-2. la)

where ri - -12 - - r3 - - kjCj + k2C2C (3.7-2. lb)

yields the following transition probability matrix:

P =

1 2 3 l-kjAt kjAt k,At

^k2C3(n)At l-k2C3(n)At 0

•5-k2C2(n)At 0 l-k2C2(n)At (3.7-2. Ic)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is demonstrated in Fig.3.7-2.1.

Page 228: Applications of Markov Chains in Chemical Engineering

222

1

0,8

0.6

0.4

0.2

0

U

h- ^

L_

1/'

1

2

L.

C 3

1 1 k =1

2

. . . . . — • ' - " ' ' "

1 i

H

-H

\ k =5

I ^ ^ I

- c = c U 2 3 - J

0 0.5 1 1.5 2 0 0.5 1 1.5 2 t t

Fig.3,7-2.1. Ci versus t demonstrating the effect of k i f o r k i = 1

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [32, p.79; 44; 48, p.20; 49, p.85] is Dbax = 0.36% and Dmean =

0.23%.

3,7-3 2Ai ^ A2 (3.7-3)

The transition matrix based on Eqs.(3-34a,b) was developed before and is

given by Eq.(3-36). The transient response of Ci and C2 for the initial state vector

C(0) = [Ci(0), C2(0)] = [1,0] is demonstrated in Fig.3.7-3. The remarks made in

3.7-2 are also applicable here.

u

11

0.8

0.6

0.4

0.2

0 (

i = r

2

3

^

1 0.5

1 k =1

2

1 1 t

"T

1 1.5 2 (

- r"

2

3

1

1 0.5

1 k =5

2

1 1 t

1

1 1.5

H

H

-H

Fig.3,7-3. Ci versus t demonstrating the effect of kz

for ki = 1 according to Eqs.(3-34a,b)

Page 229: Applications of Markov Chains in Chemical Engineering

3.7-4

where

2Ai ~^ A2 + A3

Ti — - kCj + k2C2C3

Tj = rg = 0.5kCi - O.SkjCjCj

yields the following transition probability matrix:

223

(3.7-4)

(3.7-4a)

(3.7-4b)

P =

l-kiCi(n)At

•jk2C3(n)At

^k2C2(n)At

^kiCi(n)At

l-jk2C3(n)At

0

^k,C,(n)At

l-^k2C2(n)At (3.7-4C)

where the computation of pi2 = pi3 was made by Eq.(3-10a).

The transient response of Ci, C2 and C3 for the initial state vectors C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0.1, 0] and [1, 0, 0] is demonstrated in Fig.3.7-4.

u

1

0.8

0.6

0.4

0.2

0

r-

^

[^

A = l

> -

1

1

k = 1

— -"

1 '

.

1

^

H

-

-

0 0.5 1 1.5 0 t t

Fig.3.7-4. Ci versus t demonstrating the effect of ki for k2 = 1

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [32, p.35; 49, p.86] is Dmax = 3.1% and Dmean = 0.7%. For At =

0.01, Dmax = 7.2% and Dmean = 2.6%.

Page 230: Applications of Markov Chains in Chemical Engineering

224

3.7-5

where

2A1 + A2 ^ 2 A 3

rj = - rg = - IkiCjCj + 2k2Cl

2 ^ — '^1^1^2 " 2 3

(3.7-5)

(3.7-5a)

(3.7-5b)

yields the following transition probability matrix:

P =

l-2k,Ci(n)C2(n)At

2k2C3(n)At

2

0

-kiC?(n)At

k2C3(n)At

3

3-kiC,(n)C2(n)At

|kiC^(n)At

l-2k2C3(n)At (3.7-5C)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), CBCO)] = [1, 1, 0] is demonstrated in Fig.3.7-5.

1

0.8

0.6

0.4

0.2

0

- \

h

r/ V'

1

1

1

/

^ -— -

k = 2

. - - - - I "

_\;/(c^^)_

= 1 1

-

-\

-\

0.5 1.5 2 0 0.5

\ ^ 2 .

- \l^ v Y \ >" ~

1 1 1

—j

-J c;/(c;9 \

k =5 2

1 1 1.5

Fig.3.7-5. Ciand the ratio C3/(CiC2) versus t demonstrating the

effect of k2 for ki = 1

No exact solution is available for this reaction [32, vol.2, p.76]. However, it

should be noted that the ratio C3/(C|C2) approaches at steady state the ratio ki/k2

as predicted from Eqs.(3.7-5a,b).

Page 231: Applications of Markov Chains in Chemical Engineering

225

3.7-6 A1 + A2 ^ A 3

where

Tj — T2 — — 1*2 — — K2Cx|V-'2 " ^ 2 3

(3.7-6)

(3.7-6a)

yields the following transition probability matrix:

P =

1

2

3

1 l-kiC2(n)At

0

kjAt

2

0

l-kiCi(n)At

kjAt

3

•^k,C2(n)At

^k,C,(n)At

l-kjAt (3.7-6b)

The transient response of Ci, C2, C3 and C3/(CiC2) for the initial state vector

C(0) = [Ci(0), C2(0), C3(0)] = [1, 0.5, 0] is demonstrated in Fig.3.7-6.

u

1

0.8

0.6

0.4

0.2

0

2 ^ S/^^1^2)

/ 3 ^ —

k = 1 2

\£1-

^-^ 1 1 1 1 1

K_. 2 • ~

C /(C C ) / 3 ^ 1 2 ^

J

k =5-] 2

—J

/ " 3 k "' 1 1 1

0.5 1.5 2 0 0.5 1.5

Fig.3.7-6. Ci and the ratio C3/(CiC2) versus t demonstrating the effect of ki for ki = 1

For At = 0.01, the agreement between the Markov chain solution and the exact solution [33, p.43; 48, p.20] is Dmax = 6.5% and Dmean = 1-5%. In addition, the ratio C3/(CiC2) approaches at steady state the ratio ki/k2 as predicted from Eqs.(3.7-6a,b).

Page 232: Applications of Markov Chains in Chemical Engineering

226

3,7-7 Ai + A2 "^A3 + A4 (3.7-7)

where rj = r2 = " 13 = - r4 = - kiCiC2 + k2C^C^ (3.7-7a)

yields the following transition probability matrix:

P =

1 2 3 4

l-kiC2(n)At 0 lkiC2(n)At ^kiC2(n)At

0 l-kiCi(n)At |kiCi(n)At ^kjC^WAt

^k2C4(n)At ^k2C4(n)At l-k2C4(n)At 0

^k2C3(n)At ^k2C3(n)At 0 l-k2C3(n)At (3.7-7b)

k = l , k =1 1 2

C C /(C C ) . 3 4 ^ 1 2^

U

i = l

— ~3 '

k"

" k =l',k =4 I 2

The transient response of Ci to C4 and C3C4/(CiC2) for the initial state

vector C(0) = [Ci(0), CiCO), CsCO), €4(0)] = [1, 0.5, 0.25, 0] is demonstrated in

Fig.3.7-7.

1

0.8

0.6

0.4

0.2

0

C C /(C C ) 3 4 ^ 1 2

K

0.5 1.5 2 0 0.5 1.5

Fig.3.7-7. Ci and the ratio C3C4/(CiC2) versus t demonstrating the

effect of 1 2 for ki = 1

For At = 0.01, the agreement between the Markov chain solution and the

exact solution, not existing for ki = k2, [31, p. 187; 44; 48, p.20; 49, p.86] IS l^max

Page 233: Applications of Markov Chains in Chemical Engineering

227

= 8.1% and Dmean = 5.3%. The ratio C3C4/(CiC2) approaches at steady state the

ratio ki/k2 as predicted from Eqs.(3.7-7a).

3,7-8 Ai + A2 + A3 "^ A4

where fl - r2 - "3 ~ ~ ^4 ~ ~ kjCjC2C4 + k2C4

(3.7-8)

(3.7-8a)

yields the following transition probability matrix:

P =

1 1 -

kiC2(n)C3(n)At

0

0

k2At

2 3 4 0 0 jkiC2(n)C3(n)At

1 ^ / N " ^ . X. 0 |k iC i (n )C3(n)At kiCi(n)C3(n)At 3 ^ * ^

0 . r.}^r. ^ ^. |k,Ci(n)C2(n)At kiCi(n)C2(n)At 3 ^ ^ ^

k2At k2At l -k2At (3.7-8b)

The transient response of Ci to C4 and C4/(CiC2C3) for the initial state vector C(0) = [Ci(0), C2(0), €3(0), €4(0)] = [1, 0.9, 0.8, 0] is demonstrated in Fig.3.7-8.

u

0.8

0.6

0.4

0.2

0 "(

V 1 1 ^ \ ^ i

"v \ 2^^^^^""^^" - —

' ^ C /(C C C ) 4 ^ 1 2 3^

- • ^ - " "

1 ^1 } 0.5 1

t

.1 . - - '

1 1.5 2 0 0.5

V I I I 1

^ " 2 ^ _ _ ^ 3 ^ ....

C /(C C C ) 1 ' 4 ^ 1 2 3

— —

-

l- 1 1 1 1.5

Fig.3.7-8. Ci and the ratio C4/(CiC2C3) versus t demonstrating the effect of k2 for ki = 1

Page 234: Applications of Markov Chains in Chemical Engineering

228

The present reaction appears in [35, p. 148] with no exact solution. As seen

in Fig.3.7-8, the ratio C4/(CiC2C3) approaches at steady state the ratio k;i/k2 as

predicted from Eqs.(3.7-8a).

3.8 CONSECUTIVE IRREVERSIBLE REACTIONS

3.8-1 aiAi _> A2

a2A2 -^ A3 (3.8-1)

where

ri = - kjCii T2 = - (l/ai)dCi/dt - kzCj rg = - (l/a2)dC2/dt (3.8-la)

yields the following transition probability matrix:

P =

1 2 3

l-k,C^-'(n)At l.kiC?'-'(n)At 0

0

0

l-kjC^^ kn)At J-k2Cr'(n)At a2

0 1 (3.8-lb)

where pi2 and p23 were computed by Eq.(3-10a). The transient response of Ci,

C2 and C3 for the initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [1, 0, 0] is

demonstrated in Figs.3.8-la to d for various combinations of ai and a2 in Eq.(3.8-

1).

Page 235: Applications of Markov Chains in Chemical Engineering

229

3,8-la Ai -> A2 -» A3

u

1

0.8

0.6

0.4

0.2

0

y 1 1 1 \ i = 1

\ y^

\ ^

L2 /XT \ . v 1 1 1^-—•

u - \

k =1 2

;;;v^~— 2 3

t

Fig.3.8-la. Ci versus t for ai = a2 = 1 in Eq.(3.8-1) demonstrating the effect of k2 for ki = 1

For At = 0.015, the agreement between the Markov chain solution and the

exact solution [31, p.l66; 49, p.90; 51] is Dmax = 2.2% and Dmean = 1.1%.

3,8-lb 2Ai -.> A2 _> A3

u

1

0.8

0.6

0.4

0.2

0

V 1 1 1 \ i = l 1^2=1

f V ^

1

^ — -

— • - . - -

3.^ L ^ \ / l ^ 2 - , _ _

1 1 1 1 k =5

2

--X J J 1

1 2 3 t

5 0 1 2 3 4 t

Fig.3.8-lb. Ci versus t for ai = 2, a2 = 1 in Eq.(3.8-1) demonstrating the effect of k2 for ki = 1

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [33, p.95], which is very complicated, is Dmax = 0-4% and Dmean =

Page 236: Applications of Markov Chains in Chemical Engineering

230

0.3%. It should be noted that for large values of t, C3 should approach 1 whereas for t = 150, C3 = 0.497.

3.8-lc 2Ai ^ A2 2A2 -4 A3

Fig.3.8-lc. Ci versus t for ai = 2, ai = 2 in Eq.(3.8-1) demonstrating the effect of k2 and C2(0) for ki = 1

For At = 0.02, the agreement between the Markov chain solution and the

exact solution [36; 51] is Dmax = 0.7% and Dmean = 0-5%.

Page 237: Applications of Markov Chains in Chemical Engineering

231

3.8-ld Ai _> A2 2A2 -> A3

5%

\

_ 3 . "

/ j

\

1

_^

^ '"" '

l_

JHZ

_ I - - 4 - -1

J

C (0) = 0.5 1

2 3 t

Fig.3.8-ld. Ci versus t for ai = 1, ai = 2 in Eq.(3.8-1)

demonstrating the effect of Ci(0) for ki = 1 and ki = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [36; 32, vol.2, p.51] is Dmax = 2.4% and Dmean = 4.9%.

3.8-2

kK A2

(3.8-2)

where

ri = - k iCi + k3C3 r2 = ~ k2C2 + kjCj 13 = - k3C3 + k2C2 (3.8-2a)

yields the following transition probability matrix:

p = 1

2

3

1

l-kjAt

0

kjAt

2

kjAt

l-kjAt

0

3

0

k2At

l-kjAt (3.8-2b)

Page 238: Applications of Markov Chains in Chemical Engineering

232

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is demonstrated in Fig.3.8-2 for various

combinations of the reaction rate constants. No exact solution is available for

comparison.

h-3/ V_

k 1

1 1 1 = 3,k =2

2 J

--

I [y 1 1 1 4 0

Fig.3.8-2. Ci versus t demonstrating the effect of ki and k2

for k3 = 1

3.8-3

A2 + A3 -> A4

where

Fj = —kjCj r2 = -k2C2C3+kiCi r3 = - r 4 =-k2C2C3

yields the following transition probability matrix:

(3.8-3)

(3.8-3a)

Page 239: Applications of Markov Chains in Chemical Engineering

233

1

2

3

4

1 1-kiAt

0

0

0

2 kjAt

l-k2C3(n)At

0

0

3

0

0

l-k2C2(n)At

0

4

0

^k2C3(n)At

•2-k2C2(n)At

1

p =

(3.8-3b)

The transient response of C\ to C4 for the initial state vectors C(0) = [Ci(0),

C2(0), C3(0), C4(0) ] = [1, 0, 1, 0] and [1, 0, 0.25, 0] is demonstrated in Fig.3.8-

3 where the effect of €3(0) is demonstrated.

u

1

0.8

0.6

0.4

0.2

0

r 1 \ \ Cj(0)=l J \

— \

-A L-

1 C (0) = 0.5

3

^ 2 . ' ^ " "

-J 1

" \

Fig.3.8-3. Ci versus t demonstrating the effect of €3(0)

for ki = k2 = 5

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [36] is Dmax = 3.1% and Dmean = 1-5%.

Page 240: Applications of Markov Chains in Chemical Engineering

234

3.8-4 2 A i ^ A2

A2 + A3 - • A4 (3.8-4)

where

r i - - k i C i rj = - kjCjCg + O.SkjCi rj - - r4 = - kjCjCj (3.8-4a)

yields the following transition probability matrix:

1

2

3

4

1 2 l-kiCi(n)At ikiCi(n)At

0 l-kjCjWAt

0 0

0 0

3

0

0

l-k2C2(n)At

0

4

0

^k2C3(n)At

•jkzCjWAt

1

p =

(3.8-4b)

where P12 was computed by Eq.(3-10a). The transient response of Ci to C4 for the initial state vectors C(0) = [Ci(0), CiCO), €3(0), €4(0)] = [1, 0, 1, 0] and [1, 0,0.5, 0] is demonstrated in Fig.3.8-4 where the effect of €3(0) is demonstrated.

i 1

^ 1

PH \

1 C3(0)

1

= 0.5

^4 .

"T"

1

-

• ^ , ^

2 0 0.5 1.5

Fig.3.8-4. Ci versus t demonstrating the effect of €3(0) for ki = kz = S

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [36] is Dmax = 0.6% and Dmean = 0.3%.

Page 241: Applications of Markov Chains in Chemical Engineering

3>8-5 where

Ai + A2 -> A3 -^ A4

Tj — r2 — — kjL-|C-2 3 — — ''2^3 " kjv jt 2 ^4 ~ ^2^3

yields the following transition probability matrix:

235

(3.8-5)

(3.8-5a)

P =

1

2

3

4

1 l-k,C2(n)At

0

0

0

2

0

l-kiCi(n)At

0

0

3

^k,C2(n)At

^kiC,(n)At

l-k2At

0

4

0

0

kjAt

1 (3.8-5b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 0, 1, 0] is depicted in Fig.3.8-5 where the effect of ki is

demonstrated.

1 ' ' k = 5 ' V '

i i'-'"' \^K.-[V "-

H

5 0 2 3 t

Fig.3.8-5. Ci versus t demonstrating the effect of ki for ki = 1

3,8-6 where

Ai --» A2 -^ A3 -» A4 (3.8-6)

r — — kjCj 2 ~"" ^2^2 + k|Cj r3 — — k3C3 + k2C2 r — k3C3 (3.8-6a)

Page 242: Applications of Markov Chains in Chemical Engineering

236

yields the following transition probability matrix:

1

P = 2

3

4

1 2 3 4

l-kjAt k,At 0 0

0 l-k2At k2At 0

0 0 l-ksAt k3At

0 0 0 1 (3.8-6b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.8-6 where the effect of ks is

demonstrated.

i 1

\ + 1 \

r\ / "• \ L\ / [•''V-^2 ." ' K'-'-/ . - ' I - - - -

1 1 1 k 3 = l

^ 4. - "

^ ^

^ ^ / \ J

\ 3 1

-==

-::;J.,..- 1 2 0 0.5 1.5

Fig.3.8-6. Ci versus t demonstrating the effect of ka

for ki = k2 = 5

A very complicated exact solution is available [33, p.46].

Kj K2 * 3 ^Z-1

3>8-7 Ai -> A2 -> A3 -- A4 ... Az-i _-> Az

where

Tj = kjCj 2 = - k2C2 + kjCj ^3 ~ •" ' S^B " ^^2^2

V l = "" kz- lCz-1 + ^Z-l^Z-1 n = kz_iCz_i

(3.8-7)

(3.8-7a)

yields the following transition probability matrix:

Page 243: Applications of Markov Chains in Chemical Engineering

237

1

2

3

1 l-kjAt

0

0

2 kjAt

l-kjAt

0

3

0 k2At

i-M* ...

Z-1

0

0

0

Z

0

0

0

p =

z-1 z

0

0

0

0

0

0

l-kz_iAt k^iAt

0 1 (3.8-7b)

where from Eqs.(3-19a), (3-20), one obtains that:

Ci(n+l)-Ci(n)[l-kiAt] C2(n+1) - Ci(n)[kiAt] + C2(n)[l - kjM]

CjCn+l) - C2(n)[k2At] + C3(n)[l - kg At]

Cz_i(n+1) - C^2(n)[kz^2At] + Cz_i(n)[l - kz_iAt] Cz(n+1) = Cz_i(n)[kz.iAt] + Cz(n) (3.8-7c)

An exact solution for the present case appears in [22, p.9; 33, p.52].

Particular solutions by Markov chains appear above in cases 3.8-la (n = 3) and

3.8-6 (n = 4).

3.9 CONSECUTIVE REVERSIBLE REACTIONS

3.9-1 Ai -». A2 A3

k-2

where

Tj " — kjV^j ^2 " ~ k2V-'2 " —2^^ " 1 1 ^3 " "" —2^3 "^ k2v^2

yields the following transition probability matrix:

(3.9-1)

(3.9-la)

Page 244: Applications of Markov Chains in Chemical Engineering

238

1

= 2

3

1 l-kjAt

0

0

2 kjAt

l-ksAt

k_2At

3

0 kjAt

l-k_2At (3.9-lb)

The transient response of Ci, C2, C3 and the ratio C3/C2 for the initial state vectors

C(0) = [Ci(0), C2(0), C3(0)] = [1, 0, 0] and [1, 0, 1] is depicted in Fig.3.9-1

where the effect of €3(0) is demonstrated.

1

0.8

0.6

0.4

0.2

0

U

| \ :

u h-

k'

= 1

^f ^i f

y

T 1 r

V'" 1 1 ^ ^ ^

w* " "

~"

TT^^^

C/O): = 0

H

•~j

/ \

I 1 1 ^"":r-^^

-1 - — 1

C^(0) = ll

H

0 0.2 0.4 0.6 t

0.8 1 0 0.2 0.4 0.6 t

0.8

Fig.3.9-1. Ci and the ratio C3/C2 versus t demonstrating tlie effect

of C3(0) for Iti = 1 2 = k.2 = 5

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.26] is Dmax = 6.2% and Dmean = 3.1%. As seen also

in Fig.3.9-1, the ratio C3/C2 approaches at steady state the value k2/k-2 as predicted

from Eqs.(3.9-la).

3.9-2 Ai "^ A2 - • A3 (3.9-2)

where

ri - - kjCi + k.iC2 r2 - - (k.i + ^.^^Q^ + ^\^\ 3 - ^2^2 (3.9-2a)

yields the following transition probability matrix:

Page 245: Applications of Markov Chains in Chemical Engineering

239

P =

1

2

3

1

1-kiAt

k.iAt

0

2

kjAt

l-(k_i+ k2)At

0

3

0

kjAt

1 (3.9-2b)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is demonstrated in Fig.3.9-2.

u

1

0.8

0.6

0.4

0.2

0

^ i = l

V \

I r

3

1 / 1

r k =0-i

2

-

H

1

i 1

L \ /

1

JHEL

^

":>-*->-

k =5 H 2

J J

0.5 1.5 2 0 0.5 1.5

Fig.3.9-2. Ci versus t demonstrating the efTect of 1 2

for \L\ = k.1 = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [22, p.24; 51] is Dmax = 3-6% and Dmean= O.J

3.9-3 kj kj

Ai ^ A2 ] ^ A3

k, k .

(3.9-3)

where

rj = — KjC| + k_2(J2 r2 = "~ vK_j + K2/C2 + kjCj + k_2C3

r3 - kjCj - k_2C3 (3.9-3a)

yields the following transition probability matrix:

Page 246: Applications of Markov Chains in Chemical Engineering

240

P =

1-kiAt

k_iAt

2 3 kjAt 0

l-(k_i+ k2)At kjAt

0 k_2At l-k_2At I (3.9-3b)

The transient response of Ci, C2, C3 and the ratios C2/C1, C3/C2 for the

initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [1, 0, 0] is demonstrated in

Fig.3.9-3.

Fig.3.9-3. Q , C2/C1 and C3/C2 versus t demonstrating the effect of

ki and k.ifor ki = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [22, p.42; 31, p. 175; 42] is Dmax = 1-4% and Dmean = 0.5%. As

observed in Fig.3.9-3, the ratios C2/C1 and C3/C2 approach at steady state the

limits ki/k-i and k2/k.2, respectively.

3.9-4 (3.9-4)

Page 247: Applications of Markov Chains in Chemical Engineering

241

where r = - (kj + k_3)Ci + k_iC2 + k3C3

r2 = - (k i + k2)C2 + kiCi + k_2C3

r3 = - (k3 + k.2)C3 + k_3Ci + k2C2 (3.9-4a)

From Eq.(3.9-4a), the steady state conditions for the system follows from ri = r2 =

rs = 0, yielding

r ^2 1 kik_2 + kik3 -f k_2k,3 Lr^Jeq. " *:—; ;—;; TT" ""^i

1 k_2k_2 " k_j k3 + k2k3

'-C2-'^'J-

C31 kik2 + k_2k,3 + k2k_3

kik„2 + kik3 + k.2k_3

K_iK!_2 + —1* 3 " * 2 3

•=K,

L^aJeq kik2 + k_ik_3 + KiK 1^2

From these conditions it follows that

k-ik-2k-3 = kik2k3

Eqs.(3.9-4a) yield the following transition probability matrix:

(3.9-4b)

(3.9-4C)

P =

1 2 3 l-(ki+ k_3)At kjAt k_3At

k iAt l-(k_i+ k2)At k2At

k3At k_2At l-(k.2+ ^3^^^ (3.9-4d)

The transient response of Ci, C2, C3 and the ratios C1/C3, C2/C1 and C3/C2

for the initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [1, 0, 0] are demonstrated

in Fig.3.9-4.

Page 248: Applications of Markov Chains in Chemical Engineering

242

Fig.3.9-4. Ci and Ci/Cj versus t demonstrating the effect of k.i

for ki = k2 = k3 = k4 = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.31; 35, p.92] is Dmax = 2.0% and Dmean = 0.3%. As

observed in Fig.3.9-4, the ratios Ci/Cj approach at steady state values predicted by

Eq.(3.9-4b).

3.9-5 -> Ai A2 -^ A3 --> A4 (3.9-5)

where r| = - kjCi + k_iC2 r2 = - (k_i + k2)C2 + k^Ci

r3 = - k3C3 + k2C2 u = k3C3 (3.9-5a)

yields the following transition probability matrix:

Page 249: Applications of Markov Chains in Chemical Engineering

243

1

P = 2

3

4

1 2 3 4 l-kjAt kjAt 0 0

k_iAt l-(k_i+k2)At k2At o

0 0 l-k3At k3At

0 0 0 1 (3.9-5b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1,0,0,0] is depicted in Fig.3.9-5 where the effect of ks is

demonstrated.

Fig.3.9-5. Ci versus t demonstrating the effect of ks for

111 = k.i = I12 = 5 and At = 0.01

A complicated exact solution is available [22, p.27].

3.9-6 1 1 1 2 1 3

Ai A2 A3 A4 «- ^ <-

(3.9-6)

^-2 •^-3

where

rj = - kjCi + k_iC2 12 = - (k^i + k2)C2 + kjCi + k^2C3

13 = k2C2 + k_3C4 - (k_2 + k3)C3 14 = k3C3 - k_3C4 (3.9-6a)

yields the following transition probability matrix:

Page 250: Applications of Markov Chains in Chemical Engineering

244

1

P = 2

3

4

1 l-kiAt

k.jAt

0

0

2 kjAt

l-(k_i+ k2)At

k_2At

0

3

0

k2At

l-(k_2+ k3)At

k_3At

4

0

0 kgAt

l-k_3At (3.9-6b)

The transient response of Ci to C4 for an initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1.0,0,0] is depicted in Rg.3.9-6 where the effect of ki is

demonstrated.

Sv 1 1 1 V k =k =k =1

r--—-- '

L 4

H

— _~~]

0.5 1 1.5 0 0.5 1 t t

Fig.3.9-6, Ci versus t demonstrating the effect of kt for k.1 = k.2 = k-3 = 5 and At = 0.01

1.5

A complicated exact solution is available [22, p.44].

3.9-7 Ai A2 A3 ^ A4 -»• A5 (3.9-7)

where

Fj » - kjCj + k_iC2 r2 =" - (k_i + ^ ^ 2 + k iCi + k_2C3

3 " ' ^ 2 2 "" ' ^-2 "'' ' 3 ^3 ^4 "* ^^3^3 "" k4C4 V^ = K4C4 (3.9-7a)

yields the following transition probability matrix:

Page 251: Applications of Markov Chains in Chemical Engineering

245

1

2

= 3

4

5

1 l-kjAt

k_iAt

0

0

0

2 kjAt

l-(k_i+ k2)At

k_2At

0

0

3

0 kjAt

l-(k_2+ k3)At

0

0

4

0

0 k3At

l-k4At

0

5

0

0

0 k4At

1 (3.9-7b)

The transient response of Ci to C5 for the initial state vector C(0) = [Ci(0), C2(0), C3(0), C4(0), C5(0)] = [1, 0, 0, 0, 0] and [1, 0, 0.5, 0] is depicted in Fig.3.9-7 where the effect of k4 is demonstrated.

u

11

0.8

0.6

0.4

0.2

0

« k = l ' 1 4

i-i = l

\ ' k = 5 . - « - -

\ ' • '

\ 2

- - - - -|

- J

H

* "* *1l M 1 1

1 0.5 1 l.f 0 0.5

t t

Fig.3.9-7. Ci versus t demonstrating the effect of k4 for ki = k2 = ka = 10, hL 1 = k.2 = 1 and At = 0.01

A complicated exact solution is available [22, p.29].

1.5

3.9-8

where

A i + A2 ^ A3 ^ A4

k, k .

(3.9-8)

r ** 2 *" "" *^i^i^2 " —13

r3 = - (k^i + k2)C3 + k iCiC2 + ^^2^4

1*4 "" ~k_2C4 + k2C3 (3.9-8a)

Page 252: Applications of Markov Chains in Chemical Engineering

246

yields the following transition probability matrix:

P =

1

2

3

4

1 l-kiC2(n)At

0

k_iAt

0

2

0

l-kiCi(n)At

k_,At

0

3

^kiC2(n)At

^kiCi(n)At

l-(k_,+ k2)At

k_2At

4

0

0

kjAt

l-k_2At (3.9-8b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 0.9, 0, 0] is depicted in Fig.3.9-8 where the effect of

ki, k-i is demonstrated.

1

0.8

0.6

0.4

0.2

0

h

[ k:

-i Lj "•2 - - ->_

3 - ^ ^ _ _ 4- - - > - -

1 1

k =k =1 1 2 J

k =k =5 1 -1 -2

i-"V

/

r -' I'

1

4 , - -

1

1 1

k = k = 5

k =k = 1 ~

1 0 0.5 1 1.5 0 0.5 1 1.5

t t

Fig.3.9-8. Ci versus t demonstrating the effect of kf and k.i for

At = 0.01

The present reaction is considered in [35, p. 149] without an analytical

solution.

3>9-9 Ai + A2 "^ A3 A4 ^ A5 + Ai (3.9-9)

where

Page 253: Applications of Markov Chains in Chemical Engineering

247

ri = - kjCiCj + k_iC3 - lesCiCj + k3C4 rj = - kjCjCj + k_iC3

r3 = - (k_i + k2)C3 + kiCjC2 + k_2C4

r4 = - (k_2 + k3)C4 + k2C3 + k_3CiC5 rs = - k_3CiC5 + k3C4 (3.9-9a)

yields the following transition probability matrix:

1

2

P = 3

4

5

1 1-

2 3 4 5

0 •i-k,C2(n)At ^k_3C5(n)At o [k,C2(n)+k_3C5(n)]At

k_iAt k.jAt

k3At

1-(k_i+k2)At

k_2At

0

k2At

1-(k_2+k3)At

lk_3Ci(n)At

0

k3At

1-k_3Ci(n )At

(3.9-9b)

The transient response of Ci to C5 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 0, 0, 0, 1] is depicted in Fig.3.9-9 where the

effect of ki is demonstrated.

u

1

0.8

0.6

0.4

0.2

0

\ i = l

[/

1 I 1 k =k =k =10

1 2 3

5

4

T - 2 ^ , -

\1 1 i

k =k =k =1 1 2 3

2

/

1 " 1 ' ' • ~ ^ • • " ^ - -

-

0 0.5 1.5 2 0 0.5 1.5

Fig.3.9-9. Ci versus t demonstrating the effect of ki = k i = k3

for k.i = k.2 = k.3 = 5 and At = 0.01

Page 254: Applications of Markov Chains in Chemical Engineering

248

The present reaction is considered in [35, p. 170] without an analytical

solution.

3.9-10

For the above reaction of Z states, which simulates signal transmition in a T-

cell [37], the following equations are applicable:

rj = Tj = - kjCiCz + k_i(C3 + C4 + C4+

r3 = - (k_ i+k2)C3 + kiC,C2

r4 = - (k_i + k2)C4 + k2C3

rj = - (k_i + k2)C 5 + k2C4

+ Cz_i + Cz)

Tz-l - ~ (k_i + k2)C 2;_i + k2Cz_2 tz = - k-j C z + k2Cz_] (3.9-lOa)

yielding the foUowing transition probability matrix:

P =

1

2

3

4

5

Z-1

Z

1

Pll k_iAt k_iAt

k.jAt

k_iAt

k_,At

k_iAt

2 0

P22 k_iAt

k_iAt

k_iAt

k_iAt

k_iAt

3

Pl3

P23 P33

0

0

0

0

4

0

0

k2At

P44 0

0

0

5 0

0

0

k2At

P55

0

0

z-1 0

0

0

0

0

Pz-i,z-i P7.7..1

z 0

0

0

0

0

k2/5

Pz (3.9-lOb)

Page 255: Applications of Markov Chains in Chemical Engineering

249

where

Pn = 1 - kiC2(n)At; p,3 = 0.5k,C2(n)At; P22 = 1 " kiC,(n)At; P23 = 0.5kjC,(n)At

P33 = P44 = P55 = ••• = Pz-i,z-i = 1 - (2k_, + k2)At; p , , = 1 - 2k_,At; (3.9-lOc)

From Eqs.(3-19a), (3-20), one obtains that:

Ci(n+1) = Ci(n)[l - kiC2(n)At] + {CjCn) + CgCn) + - + Cz(n)}[k_iAt]

C2(n+1) = C2(n)[l - kiCi(n)At] + {CjCn) + €4^) + ••• + Cz(n)}[k_iAt]

Cjin+l) = Ci(n)[0.5kiC2(n)At] + C2(n)[0.5kiC,(n)At] + C3(n)[l-(2k_, + k2)At]

C4(n+1) = C3(n)[k2At] + C4(n)[l - (2k_i + k2)At]

Cz_i(n+1) = Cz_2(n)[k2At] + Cz_i(n)[l - (2k_, + k2)At]

Cz(n+1) = Cz_i(n)[k2At] + Cz(n)[l - 2k_iAt] (3.9-lOd)

For Z = 5 the transient response of Ci to C5 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0), C4(0), C5(0) = [1, 1, 0, 0, 0] is depicted in Fig.3.9-10

where the effect of ki is demonstrated.

\ \h2

" V L / \

U—:

1

— K

4

">-— 5 -

1

1 1 1 k = 7

-- J

-

-

1 1 1 1 0 0.2 0.4 0.6 0.8

t

Fig.3.9-10. Ci versus t demonstrating the effect of ki for It-i = 1,

k2 = 5 and At = 0.003

Page 256: Applications of Markov Chains in Chemical Engineering

250

3.10 PARALLEL REACTIONS: SINGLE AND CONSECUTIVE IRREVERSIBLE REACTION STEPS

3.10-1 Ai ^ A 2 (3.10-1)

where [38, chap., problem C48]

Ti = - (ki + k2)Ci r2 = -k3C2 + kiCi r3 = k2Ci + k3C2 (3.10-la)

yields the following transition probability matrix:

1

P = 2

3

1 1 - (ki + k2)At

0

0

2 kjAt

1 - k3At

0

3 k2At

k3At

1 (3.10-lb)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is depicted in Fig.3.10-1 where the effect of ks is

demonstrated.

1

0.8

0.6

0.4

0.2

0

u

-\i = l

V

1 1

X /

/

1 i"~"~^—

1 ^ J __ -—

k3 = 5]

.

\ , '

^z^' \

I ' r^^

1 1 1 1

-] k =0 3 J

3 -J

0.2 0.4 0.6 0.8 t

1 0 0.2 0.4 0.6 0.8 t

Fig.3.10-1. Ci versus t demonstrating the effect of ka

for ki = 5, k2 = 1 and At = 0.005

Page 257: Applications of Markov Chains in Chemical Engineering

3.10-2

251

(3.10-2)

where [38, chap., problem C55]

r i = - k i C i r2 = - (kj + k3)C2 + k,Ci r3 = k2C2 r4 = k3C2 (3.10-2a)

yields the following transition probability matrix:

1

P = 2

3

4

1 l-kjAt

0

0

0

2 kjAt

l-(k2 + k3)At

0

0

3

0 k2At

1

0

4

0 k3At

0

1 (3.10-2b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.10-2 where the effect of ks

is demonstrated.

1

0.8

0.6

0.4

0.2

0

u

4 i= 1

\ 2 - -V

L <•'

1 1 k =1

3

^ ^ 4 .

r^-—•— 1 '^^

-

\

\j '"^ '>--^^^^^i:i_.j=^

-J

__ —

0.5 1 1.5 0 0.5 1 t t

Fig.3.10-2. Ci versus t demonstrating the effect of ka

for ki = 5, k2 = 2 and At = 0.01

1.5

Page 258: Applications of Markov Chains in Chemical Engineering

252

3.10-3

1

1 ^

A 2 - : i l ^ A 4 - ^ I - ^ A 6

A, — - . ^ - As . >-A7 k6 MO

where

ri = - (ki + k2)Ci X2= - (kj + k4)C2 + kjCj

rj = - (ks + k6)C3 + kjCi u = - (ky + k8)C4 + kjCj + kjCs

rj = - (k9 + kio)C 5 + k4C2 + k^Cj

ig = k7C4 + k9C5 r-j = k8C4 + kjoCs

yields the following transition probability matrix:

P =

1

2

3 4

5

6

7

1

Pll 0

0

0

0

0

0

2 kjAt

P22 0

0

0

0

0

3 k2At

0

P33 0

0

0

0

4

0 kjAt

ksAt

P44 0

0

0

5

0 k4At

k At

0

P55 0

0

6

0

0

0 kyAt

kjAt

1

0

7

0

0

0 kgAt

k,oAt

0

1

where

(3.10-3)

(3.10-3a)

(3.10-3b)

Pll = 1 - (ki + k2)At P22 = 1 - ( 3 + k4)At P33 = 1 - {^^ + k6)At P44 = 1 - (k7 + k8)At P55 = 1 - (k9 + kio)At (3.10-3d)

It should be noted that by equating to zero one (or more) of the rate constants ki in

Eq.(3.10-3b), many interesting reactions can be generated.

The transient response of Ci to C7 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), ..., C7(0)] = [1, 0, 0, ..., 0] is depicted in Fig.3.10-3 where the

effect of k7 is demonstrated.

Page 259: Applications of Markov Chains in Chemical Engineering

253

u

0.8

0.6

0.4

0.2

0

1' 4 -r""'

A / " ^ "• -

I k =1

7

c =c 6 7

^ ^ - . c =c " " - 1 ^

1 " ^ ' '

\

tc \ 1

1

/L^

' k = 5 ' 7

c ,.. 6

c 7

. c

-

_

-

0 0.5 1 1.5 0 0.5 1 1.5 t t

Fig.3.10-3. Ci versus t demonstrating the effect of k? for

ki = k2 = k4 = ks = 5, k3 = k6 = 10, kg = kp = kio = 1 and At = 0.005

An extremely complicated exact solution is available [22, p.55].

3.10-4 3 ^ A 5

ly< Ai + A s ' ^ |k5

1 ^ A4 • >• Ae k4

where [38, chap.6, problem B37]

(3.10-4)

rj = r2 = - (kj + k2)CiC2

r3 = - k3C3 + kjCiC2 r4 = - k4C4 + k2CjC2

r5 = - ksCs + k3C3 r = )^^C^ + ^^C^ (3.10-4a)

yields the following transition probability matrix:

Page 260: Applications of Markov Chains in Chemical Engineering

254

P =

1

2

3

4

5

6

1

Pll

0

0

0

0

0

2

0

P22

0

0

0

0

3 lkiC2(n)At

ikiCi(n)At

l-kgAt

0

0

0

4 Yk2C2(n)At

lk2Ci(n)At

0

l-k4At

0

0

5

0

0

k3At

0

l-k5At

0

6

0

0

0

k4At

k^M

1

where Pll = 1 - (kj + k2)C2(n)At; P22 = 1 - (^i + k2)Ci(n)At

(3.10-4b)

(3.10-4C)

The transient response of Ci to C^ for the initial state vector C(0) = [Ci(0), C2(0),

C3(0), ..., C6(0)] = [1, 0, 0, ..., 0] is depicted in Fig.3.10-4 where the effect of

C2(0) is demonstrated.

1

0.8

0.6 r 0.4

0.2

0

L \ ^

' 0^(0) = 0.5 ' 1

• c 6

" c =c • ^ 3 4

r-"-:^-^-^i . .

I LA

k r - ^ ^ > -

' c^(0)

c =c / 3 4

" • ^ - i ^

- T~~--

= 0 . 6 '

-\

c J

c -:-^k.,-1 2 3 0 1 2 3

t t

Fig.3.10-4. Ci versus t demonstrating tlie ej^ect of C2(0)

for Ici = k2 = 5, k3 = li4 = ks = 2 and At = 0.015

3.10-5

ajAi + a j A j — ^ ^ • A 4

(3.10-5)

Page 261: Applications of Markov Chains in Chemical Engineering

255

where [32, vol.2, p.77]

ii = - aikC?iC22 12 = - a2kCi>C22

k = ki + k2 + k3

(3.10-5a)

If the initial conditions are CsCO) = €4(0) = €5(0) = 0, it follows that

ri/r3 = k2/ki or C4/C3 = k2/ki

r5r3 = k3/ki or C5/C3 = k3/ki (3.10-5b)

Thus, the ratio of the amounts of the products is constant during the reaction and

independent of its order. Eqs.(3.10-5a) yield the following transition probability

matrix:

1 2

P = 3 4 5

1

Pll

0

0

0

0

2 0

P2

0

0

0

where

3 4 5

0 NikiC i"' (n)C 2(n)At Nik2Cii"kn)C^2(n)At N^k^Cl^^HnK^MM

P22 N2kiCji(n)C^"Vn)At ^jh^^]'^^^^!^'^^^^^^ N2k3CjKn)C^"Hn)At

1

0

0

0

1

0

Pll = 1 - aikC i""Hn)C22(n)At; P22 = l-a2kCii(n)C22"Hn)At

^1 XT ^ k = k, + ko + ko; Ni = • ; N , = .

(3.10-5C)

(3.10-5d)

For ai = a2 =1, the transient response of Ci to C5 is depicted in Fig.3.10-5

where the effect of C2(0) is demonstrated. It should be noted also that Eqs.(3.10-

5b) are verified by the numerical results.

Page 262: Applications of Markov Chains in Chemical Engineering

256

0 0.5 1 1.5 0 0.5 1 1.5 t t

Fig.3.10-5. Ci versus t demonstrating the effect of CiCO) for

Ci(0) = 2, ki = 3, k2 = 2, k3 = 1 and At = 0.005

3 . 1 0 - 6 i = 1: Ai_^A2

i = 2: 2Ai-^A3 (3.10-6)

The derivation of the kinetic equations, based on Eqs.(3-2), (3-3), is:

fi -~^2 - *^iCi

1 J 2 ) _ . ( 2 ) _ _ , , p 2 2" ti - - r3 - - K2i-.i

where from Eq.(3-4) follows that

ri = rV^ -I- xf = - (k,Ci -i- 2k2Ci)

, Ji) u r f - r(2) _ u r2 ^2-h - *'i*-i ^3-r3 -is.2^1 (3.10-6a)

yielding the following transition probability matrix:

1

P = 2

3

1 2 3 l-[ki-I-2k2Ci(n)]At kjAt k2Ci(n)At

0 1 0

0 0 1 (3.10-6b)

where pi4 is computed by Eq.(3-10a).

Page 263: Applications of Markov Chains in Chemical Engineering

257

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is depicted in Fig.3.10-6 where the effect of k2 is

demonstrated.

Fig.3.10-6. Ci versus t demonstrating the effect of kz for Iti = 5

For At = 0.0025, the agreement between the Markov chain solution and the

exact solution [33, p.35; 39, p.32] is Dmax = 4.8% and Dmean = 4.0%.

3.10-7

where

Ai .^ A2 -> A3

2Ai —> A4

Tj = - kjCj - 2k3Ci

Ta = k2C2 r4 = k3Cj

yields the following transition probability matrix:

T2 — — k2i-^2 " ki^--!

2

1

P = 2

3

4

1 2 3 4 l-[ki + 2k3Ci(n)]At k,At 0 k3C,(n)At

0 l-k2At k2At 0

0 0 1 0

0 0 0 1

(3.10-7)

(3.10-7a)

(3.10-7b)

Page 264: Applications of Markov Chains in Chemical Engineering

258

The transient response of Ci to C4 and ZCi for the initial state vector C(0) =

[Ci(0), C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.10-7 where the

effect of ks is demonstrated.

u

3.5

3

2.5

^ 2

1.5

1 0.5

0 \^'^ ^ ^ r- — r 0.5 1 1.5 0 0.5

t t Fig.3.10-7. Ci versus t demonstrating the effect of ka

for ki = k2 = 1 and At = 0.015

No exact solution is available. However, it should be noted in the above

figure that XCi approaches the limits (2 for k3 = 1 and 3 for ks = 0) according the

stoichiometry in Eq.(3.10-7).

3.10-7,1 Ai -» A2 A3

Ai + A2 -^ A4

where

Ai + A3 -> A4 (3.10-7.1)

ri = - kiCi - k3CiC2 - k4CiC3 x^ = ^\^\ " k2C2 - k3CiC2

1*3 — k2C2 — k4C],C3 r4 = k3CiC2 + k4CiC3 (3.10-7. la)

yields the following transition probabiUty matrix:

Page 265: Applications of Markov Chains in Chemical Engineering

259

P =

1

2

3

4

1

l - [ki+ k3C2(n)+k4C3(n)]At

0

0

0

2

kjAt

l-[k2+k3Ci(n)]At

0

0

3

0

k2At

l-k4Ci(n)]At

0

4

•i-[k3C2(n)

+k4C3(n)]At

i-k3Ci(n)At

lk4Ci(n)]At

1

(3.10-7. lb)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0), C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.10-7.1 where the effect of ki is demonstrated.

0 2 4 6 0 2 4 6 t t

Fig.3.10-7.1. Ci versus t demonstrating the effect of ki for k2 = k3 = k4 = 1 and At = 0.01

3.10-7.2

where

k[ kj

Ai + A2 -4 A3 -> A4

^ 3

Ai + A2 -» A5

ri = r2 = ~ (ki + k3)CiC2

3 " l^lCiC2 - k2C3

r4 = k2C3 r5 = k3CiC2

(3.10-7.2)

(3.10-7.2a)

Page 266: Applications of Markov Chains in Chemical Engineering

260

yields the following transition probability matrix:

P =

1

2

3

4

5

1 l~(ki+ k3)C2(n)At

0

0

0

0

2

0

l - (ki+ k3)Ci(n)At

0

0

0

3 lkiC2(n)At

ikjCiWAt

l-k2At

0

0

4

0

0

k2At

1

0

5 yk3C2(n)At

lk3Ci(n)At

0

0

1

(3.10-7.2b) The transient response of Ci to C5 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 0, 0, 0, 0] is depicted in Fig.3.10-7.2 where the effect of ki is demonstrated.

1 1 1 k =5

1

\ ^ |1C =C . - ' 1 2

Q_ ^ M \c

\V Pv ^ ' L V ^ B '^^^ "V^ . . . .

\' ^\ """ r

c 4

c 5

6 0

Fig.3.10-7.2. Ci versus t demonstrating the effect of ki for k2 = ii3 = 1 and At = 0.01

3.10-8

where

A i ^ A 2

Ai + A2 -> A3

T] — — vk|Cj + k2C|C2) r2 — k]Cj ~ k20jC-2 '"3 — k2C jC 2

yields the following transition probability matrix:

(3.10-8)

(3.10-8a)

Page 267: Applications of Markov Chains in Chemical Engineering

261

1

P =

1, l-[ki + k2C2(n)]At k,At ^k2C2(n)At

0 l-k2Ci(n)At lk2C,(n)At

0 0 1 (3.10-8b)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0,0] is depicted in Fig.3.10-8 where the effect of Ci(0)

is demonstrated.

4

3 k

U

-

C (0) = 1

J = l

- 3-

-

-

\ - \

\ \ \ \

/ . ' ;

C (0) = 4

2

H

3— H

0.5 1.5 0 0.5 1.5

Fig.3,10-8. Ci versus t demonstrating the effect of Ci(0) for ki = k2 = 2

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [33, p.95; 44; 49, p.91] is Dmax = 4.1% and Dmean = 2.4%.

3.10-9

where

Ai-^A4

Ai + A2-> A3

ri = - kiCj - k2CiC2 r2 = - r3 = - k2CiC2 i^ = k2Ci

yields the following transition probability matrix:

(3.10-9)

(3.10-9a)

Page 268: Applications of Markov Chains in Chemical Engineering

262

1 2 3 4

0 4-k2C2(n)At kjAt

1

l-[k, + k2C2(n)]At

0 l-k2Cj(n)At i.k2C,(n)At 0

0 0 1 0

0 0 0 1 (3.10-9b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.10-9 where the effect of ki

is demonstrated.

- \ -"^ -'-

\' 1

k 1

— 4 -

1 = 2

1

3

— 1 -

-

H

0.5 1.5

Fig.3.10-9. Ci versus t demonstrating the effect of ki

for li2 = 2

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.45] is Dmax = 10% and Dmean = 0.5%. It should be

noted that in [27], the transition probability matrix is incorrect.

3,10-9,1

where [53, p.201]

2Ai->A4

Ai + A2 -> A3 (3.10-9.1)

ri = - 2kiCi2 - k2CiC2 r2 = - r3 = - k2C|C2 i^ = k^Cf (3.10-9.la)

Page 269: Applications of Markov Chains in Chemical Engineering

263

yields the following transition probability matrix:

1

P =

2 0 ^k2C2(n)At kiCi(n)At l-[2kiCi(n) + k2C2(n)]At

0 l-k2Ci(n)At |k2Ci(n)At 0

0 0 1 0

0 0 0 1

where pi4 is computed by Eq.(3-10a). (3.10-9. lb)

The transient response of Ci to C4 for an initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.10-9.1 where the effect of

ki is demonstrated.

1

0.8

0.6

0.4

0.2

0

u

r nyi=i

Vr ^ \

\

^2

3^

Ar

1

1 1 k =5

1

• - "

"1 h-

1 1

^

-

[-

0.2 0,4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 t t

Fig.3.10-9.1 Ci versus t demonstrating tlie effect of l i

for 1 2 = 5 and At = 0.02

3,10-10

where

Ai -> A3 + A4

Ai + A2 -» A3 + A5 (3.10-10)

Tj — — 13 — — kjv.'i "~ 2^1^2 2 "" ~ ^2^1^2

14 = kjCj 15 = k2CiC2

yields the following transition probability matrix:

(3.10-lOa)

Page 270: Applications of Markov Chains in Chemical Engineering

264

P =

2

3

4

5

1 1 -

[kj + k2C2(n)]At

2 3 4 5 0 ''1^^+ k,At ik2C2(n)At

ik2C2(n)At

0 k2c'^)At i'^^^'^"^^^ 0 T'^^'^"^^'

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1 (3.10-lOb)

The transient response of Ci to C5 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 1, 0, 0, 0] is depicted in Fig.3.10-10 where the

effect of k2 is demonstrated.

1

0.8

0.6

0.4

0.2

0

i\\ . ' 3

r \^ ^ 1 ^ ^

. 4 - - -

• - • -

-'-"r 1

~^ 1 = 5 2

— 1, 0 0.2 0.4

t 0.6 0.8 0

Fig.3.10-10. Ci versus t demonstrating the effect of k2

for ki = 5 and At = 0.015

3>10-11 Ai + Ai -»A2

Ai + A3 -^ A4

Ai + A2-^A3

Ai + Az-i -> Az Ai + Az-^ Az+i (3.10-1 la)

Page 271: Applications of Markov Chains in Chemical Engineering

where

Ti = klCi + 2j^m^l^n m=2

12 = O.SkiCf - k2CiCi 13 = k2CiC2 - k^C^C^

265

yields the following transition probability matrix:

P =

1

2

3

4

5

:

Z

Z+1

1

Pll 0

0

0

0

0

1 0

2

Pl2

22 0

0

0

0

0

0

3

Pl3

P23

P33

0

0

0

0

0

4

Pl4 0

P34

P44

0

0

0

0

5

Pl5 0

0

P45

P55 ••

0

0

0

z Piz 0

0

0

0

0

Pzz 0

z+1

Pl,Z+l

0

0

0

0

0

Pz,z+i 1

where

Pii = l - 2kiCi(n) + ^k„C„(n) m=l

At P12 = kjCi(n)At

Pi.m+1 = 0.5k^C Jn)At m = 2,..., Z

and for the deteraunation of pn see section 3.3.

P22 = 1 - k2Ci(n)At P23 = 0.5k2Ci(n)At

P33 = 1 - k3Ci(n)At P34 = 0.5k3Ci(n)At

P44 = 1 - k4Ci(n)At P45 = 0.5k4Ci (n)At

(3.110-llb)

(3.10-1 Ic)

Page 272: Applications of Markov Chains in Chemical Engineering

266

P55 = 1 - k5Ci(n)At p56 = O.SksCi(n)At

pzz = 1 - kzCi(n)At pz,z+i = 0.5kzCi(n)At

k,

(3.10-1 Id)

For Z = 2:

where

2Ai_»A2 kj

Ai + A2 -> A3 (3.10-1 le)

ri = -2k iCi -k2CiC2 r2 = -k2C,C2 + k,C^ r3 = k2CiC2 (3.10-1 If)

yields the following transition probability matrix:

p =

1

2

3

l-[2kiCi(n) + k2C2(n)]At kiCi(n)At ^k2C2(n)At 1

0 l-k2Cj(n)At ik2Ci(n)At

0 0 1 (3.10-1 Ig)

where pi2 is computed by Eq.(3-10a).

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1, 0, 0] is depicted in Fig.3.10-11 where the effect of k2

is demonstrated.

Fig.3.10-11. Ci versus t demonstrating the effect of k2

for ki = 1 and At = 0.03

Page 273: Applications of Markov Chains in Chemical Engineering

267

It should be noted that refs.[32, vol.2, p.69; 39, p.47] predict oscillations,

not observed here.

3>10-12 2Ai -^ A2 + A4

Ai + A2 -> A3 + A4

where [32, vol.2, p.70]

(3.10-12)

r — — kiC i — k2v-'iv 2 2 — 1 1 — 12 1 2

3 "" k2CiC2 r4 = kiC? + k2CiC2 (3.10-12a)

yields the following transition probability matrix:

1 1 -

[2kiCi(n)4-k2C2(n)]At

0

0

0

2 3 4 kiCi(n)At ik2C2(n)At [|k2C2(n)+kiCi(n)]At

l-kiCi(n)At ik2Ci(n)At

0 1

0 0

ik2Ci(n)At

0

1

(3.10-12b)

where pi2 was computed by Eq.(3-10a). The transient response of Ci to C4 for

the initial state vectors C(0) = [Ci(0), C2(0), €3(0), €4(0), €5(0)] = [1, 1, 0, 0, 0]

and [0.5, 1, 0, 0, 0] is depicted in Fig.3.10-12 where the effect of Ci(0) is

demonstrated.

Page 274: Applications of Markov Chains in Chemical Engineering

268

1

0.8

0.6

0.4

0.2

0

l \ 1 1 1 ^ C(0)=1

V 4 - - - " " 1 \ ' ' / X r ^ - -

1 " " ~ ~ " ~ i — • — H -

-

0.5 1.5 2 0

Fig.3.10-12. Ci versus t demonstrating the effect of Ci(0) for ki = k2 = 2 and At = 0.03

3,10-13

where

Ai + 2A2 -4 A3

Ai + A2 -> A4

Tj — - k iCiC2 - k2CiC2 1*2 - - 2kiCiC2 - k2C|C2

yields the following transition probability matrix:

(3.10-13)

(3.10-13a)

P =

1 1 -

[kiC2(n)+k2C2(n)]At

0

0

0

2

0

3 4 lkiC2(n)At ik2C2(n)At

l-[2kiCi(n)C2(n) |k,Ci(n)C2(n)At ik2Ci(n)At

+k2Ci(n)]At

0

0

(3.10-13b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.10-13 where the effect of

k2 is demonstrated.

Page 275: Applications of Markov Chains in Chemical Engineering

269

u

1

0.8

0.6

0.4

0.2

0

r 2 \

~ l r ' ' ^ If

1 1 1 k = 5

2

^ J>- - -^

.. ^ ^ 4

1 1 - 4 -

-

0.5 1.5 2 0

Fig.3.10-13. Ci versus t demonstrating the effect of k i for ki = 5 and At = 0.01

3 , 1 0 - 1 4 A i - ^ A 3

Ai + A2 -^ A4

2 A i ^ A 5

where

13 = kjCj r4 = k2CiC2 15 = k3Cj

yields the following transition probability matrix:

1

P =

1 l-[ki+k2C2(n)+

2k3Ci(ii)]At

0

2

0 kjAt ik2C2(n)At k3Ci(n)At

(3.10-14)

(3.10-14a)

l-k2Ci(n)]At 0

3 0 0 1

4 I 0 0 0

5 I 0 0 0 0 1 I (3.10-14b)

where pi5 was calculated by Eq.(3-10a). The transient response of Ci to C5 for the

initial state vector C(0) = [Ci(0), C2(0), €3(0), €4(0), €5(0)] = [1, 1, 0, 0, 0] is

depicted in Fig.3.10-14 where the effect of ki is demonstrated.

i.k2Ci(n)At

0

1

0

0

0

0

1

Page 276: Applications of Markov Chains in Chemical Engineering

Fig.3.10-14. Ci versus t demonstrating the effect of ki for k2 = 2 and k3 = 5

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.48] is Dmax = 3.3% and Dmean = 0.6%. For ks = 0, an

exact solution is available [51].

3>10-15 2Ai-^A3

Ai + A2 -> A4

2A2->A5

where [33, p.75; 38, chap.6, problem D82]

(3.10-15)

rj — — ZkjC j — k2v_ j>-'2 r2 - ~ 2k3C2 - k2CiC2

3 ~ '^i^i ^4"" k2CiC2 r5 - k3C2 (3.10-15a)

yields the following transition probability matrix:

Page 277: Applications of Markov Chains in Chemical Engineering

271

P =

1

2

3

4

5

1 2

0 kiCi(n)At lk2C2(n)At o l-[2kiCi(n)

+k2C2(n)]At

Q HkjCiCn) Q lk2Ci(n)At k3C2(n)At +k3C2(n)]At

0

0

0

0

0

0

0

0

1 (3.10-15b)

The transient response of C\ to C5 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 1, 0, 0, 0] is depicted in Fig.3.10-15 where the

effect of k3 is demonstrated.

u

1

0.8

0.6

0.4

0.2

0

1-

\ ^

r

1

5 - •

_. __ -.

1

' ' \

-

- 3

- r 1.

0.5

[

't \\ \ .

L ^ -!>^

1

- 3 -

""1"^

1 1 1 k =1 3

H

__ 1 ' 5

1.5 1 1.5 2 0 0.5 1 t t

Fig.3.10-15 Ci versus t demonstrating the effect of k^ for ki = k2 = 5 and At = 0.01

3.10-15.1

where

r i = - k i C ,

A i ^ A 2

2A2 -> A3

A 2 ^ A 4

2 ~ '^iCl - k2C2 - k3C2

Tj = O.SkjCj u = kgCj

(3.10-15.1)

(3.10-15.1a)

Page 278: Applications of Markov Chains in Chemical Engineering

272

yields the following transition probability matrix:

P =

1

2

3

4

1

l-kjAt

0

0

0

2

kjAt

l-[k2C2(n)+k3]At

0

0

3

0

"J^l^i

1

0

4

0

k3At

0

1 (3.10-15.1b)

where p23 was calculated by Eq.(3-10a). The transient response of Ci to C4 for

the initial state vector C(0) = [Ci(0), C2(0), €3(0), C4(0)] = [1,0,0,0] is depicted

in Fig.3.10-15.1 where the effect of ks is demonstrated.

Fig.3.10-15.1 Ci versus t demonstrating the effect of ks for ki = k2 = 5 and At = 0.01

A rather compUcated exact solution is available [50].

3,10-16 Ai -> A4 + A5

A5 + A2-^A3

Ai+A2->A3 + A4 (3.10-16)

Page 279: Applications of Markov Chains in Chemical Engineering

273

where [33, p.86]

fj = - kjCi - k3CiC2 12 = - 13 = - k2C2C5 - k3CiC2

14 = kjCj + k3CiC2 15 = kjCj - k2C2C5

yields the following transition probability matrix:

(3.10-16a)

P =

1 1-

[ki+k3C2(n)]At

0

2 3 4

0 ik3C2(n)]At [ki+lk3C2(n)]At

l-[k2C5(n) [ |k2C5(n) ik3C i(n)At +k3Ci(n)]At :

+i.k3Ci(n)]At

kjAt

0

0

0

0

0

^ i.k2C2(n)]At

0

1

0

0

0

l-k2C2(n)]At

(3.10-16b)

The transient response of C\ to C5 for the initial state vectors C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 1, 0, 0, 0] and [0.5, 1, 0, 0, 01 is depicted in

Fig.3.10-16 where the effect of Ci(0) is demonstrated.

1

0.8

0.6

0.4

0.2

0 0 0.5 1 1.5 0

t

Fig.3.10-16. Ci versus t demonstrating the effect of Ci(0) for ki = k2 = k3 = 5 and At = 0.005

I -' X/

p \=, "" —-

1

C ( 0 ) = 1

-

1

Page 280: Applications of Markov Chains in Chemical Engineering

274

3 .10 -17

where

Ai + A2 -^ A3

A2 + A3 ^ A4 (3.10-17)

i i — - k j C i C 2 r 2 - - k i C ] C 2 - k 2 C 2 C 3

T^ ^ lCjV jv 2 ~ ^2^-2^3 4 ^ K2C2^3

yields the following transition probability matrix:

(3.10-17a)

P =

1 2 3 4 l-kiC2(n)]At 0 Yk,C2(n)At 0

0 l-[klCi(n) ikiC,(n)At lk2C3(n)At +k2C3(n)]At

0 l-k2C2(n)]At lk2C2(n)At 0

0 0 (3.10-17b)

The transient response of Ci to C4 for the initial state vectors C(0) = [Ci(0), C2(0), C3(0), C4(0)] = [1, 1, 0, 0], [1, 0.5, 0, 0] and [0.5, 1, 0, 0] is depicted in Fig.3.10-17 where the effect of C2(0) is demonstrated.

y 1

_\1

t K

1 1 C (0) = 0.5

2

4

^.L i

H

2 0 0.5 1.5

Page 281: Applications of Markov Chains in Chemical Engineering

275

1.5

U^ 1

0.5

n

[ I I I 1 \ C (0) = 2

\ - 2

v\>--^ v~^^Cj^:_-~^ - - ^ ~~^ _

0.5 1.5

Fig.3.10-17. Ci versus t demonstrating the effect of C2(0) for ki = k2 = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [33, p. 100; 43; 51] is Dmax = 2.5% and Dmean = 1.9%.

3>10-18

where

Ai + A2 -» A3 + A4

Ai + A3 -> A5 + A4

rJ = — JC|C'2C^2 ~ k2v-'iC-'3

3 - ^\^\^2 "" '^2CiC3

4 ^ k.2C|C2 " " 2 1 3

yields the following transition probability matrix:

(3.10-18)

r 2 - - k i C | C 2

^5 = ^\^\^3 (3.10-18a)

P =

1 l-[kiC2(n)

+k2C3(n)]At

0

0

0

0

2 3 4 5

0 ikiC2(n)]At ^[kiC2(n) ik2C3(n)]At

+k2C3(n)]At

l-kiCi(n)]At ikiC,(n)]At i.kjCi(n)]At Q

0 l-k2Ci(n)]At ^k^c,(n)At lk2Ci(n)At

0

0

1

0

0

1

(3.10-18b)

Page 282: Applications of Markov Chains in Chemical Engineering

276

The transient response of Ci to C5 for the initial state vectors C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0)] = [1, 1, 0, 0, 0] and [1, 0.5, 0, 0, 0] is depicted in

Fig.3.10-18 where the effect of C2(0) is demonstrated.

Fig.3.10-18. Ci versus t demonstrating the effect of C2(0) for ki = k2 = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.61; 47] is Dmax = 0.7% and Dmean = 0.3%. It should

be noted that an exact solution is available only for Ci(0) = 2C2(0) in the first

reference where in the other one it is a comphcated solution.

3,10-19

where

Ai + A2 ^ A3 + A5

Ai + A3 _> A4 + A6 (3J0-19)

Ti - •" \CiC2 - k2C|C3 r2 - - kiC|C2

3 = '^iCiC2 - k2CiC3 r4 = r = k2CiC3

r5 = kiCiC2 (3.10-19a)

yields the following transition probability matrix:

Page 283: Applications of Markov Chains in Chemical Engineering

p =

277

1 2 3 4 5 6

P n 0 •jkiC2(n)At lk2C3(n)]At lkiC2(n)At lk2C3(n)]At

0 l-kiCi(n)]At ikiCi(n)At ^kiCi(n)At

l-k2Ci(n)]At ik2Ci(n)At o ~-k2Ci(n)At

0

0

0

0

0

0

0 1 0 0

0 0 1 0

0 0 0 1

(3.10-1%) where Pn = l-[kiC2(n) + k2C3(n)]At

The transient response of Ci to C^ is depicted in Fig.3.10-19 where the

effect of C2(0) is demonstrated for ki = 2 and k2 = 1.

Fig.3.10-19. Ci versus t demonstrating the effect of CiCO)

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.65] is Dmax = 10% and Dmean = 0.7%.

3.10-20 Ai + A2 -» A3 + A6

Ai + A3 -^ A4 + A6

Ai + A4 -> A5 + A6 (3.10-20)

where

Page 284: Applications of Markov Chains in Chemical Engineering

278

rj - - kiCiC2 - k2CiC3 - k3CiC4 ^2-^ k]C|C2

3 ~ 1^1^102 - k2C|C3 14 -- k2C2C3 — k3C|C4

1 15 = k3CiC4 16 = 2-(kiCiC2 + k2CiC3 + k3CiC4)

yields the following transition probability matrix:

(3.10-20a)

P =

1

2

3

4

5

6

1 2 3 4 5 6

Pll 0 YkiC2(n)At ik2C3(n)]At ik3C4(n)]At p^^

0 l-kiCi(n)]At ikiCi(n)At 0 0 ^kjCiWAt

0 0 l-k2Ci(n)]At ik2C,(n)At 0 |k2Ci(n)At

l-k3Ci(n)]At ik3Ci(n)At lk3Ci(n)At 0

0

0

0

0

0

0

0

where Pll = l-[kiC2(n) + k2C3(n) + k3C4(n)]At

P16 = y[kiC2(n) + k2C3(n) + k3C4(n)]At

(3.10-20b)

(3.10-20C)

The transient response of Ci to C6 for an initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0), C6(0)] = [3, 1, 0, 0, 0, 0] is depicted in Fig.3.10-20

where the effect of C2(0) is demonstrated.

0.6 0

Fig.3.10-20. Ci versus t demonstrating the effect of k2 for ki = k3 = 10

Page 285: Applications of Markov Chains in Chemical Engineering

279

For At = 0.0025, the agreement between the Markov chain solution and the

exact solution [32, voL2, p.66; 45; 46] is Dmax = 3.6% and Dmean = 2.6%. It

should be noted that an exact solution is available only for Ci(0) = 3C2(0).

3>10-21 Ai + A2 -^ A3

A3 + A2 -> A4

A4 + A5 -^ A6

where [32, vol.2, p.66]

r^ = - kiCiC2 1*2 ~ ~ kiCiC2 ~ k2C2C3

^3 == k |CiC2 - ^^2^1^^

yields the following transition probability matrix:

(3.10-21)

(3.10-21a)

P =

1

2

3

4

5

6

1 l"kiC2(n)]At

0

0

0

0

0

2

0

P22

0

0

0

0

3 •ikiC2(n)At

ikjCiWAt

l-k2C2(n)]At

0

0

0

4

0

ik2C3(n)]At

ik2C2(n)At

l-k3C5(n)At

0

0

5

0

0

0

0

l-k3C4(n)]At

0

6

0

0

0

k3C5(n)At

k3C4(n)lAt

1

(3.10-21b)

where

P22 = l-[kiCi(n) + k2C3(n)]At (3.10-21C)

The transient response of Ci to C6 for an initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0), C6(0)] = [1, 1, 0, 0, 1, 0] is depicted in Fig.3.10-21

where the effect of k2 is demonstrated.

Page 286: Applications of Markov Chains in Chemical Engineering

280

u

1

0.8

U.6

04

0.2

0

I - 1 1 \ \ \ N

\ . - - ^ 3

1

_ 5

6

-j

k =10 2 J

L.

\

^

[--.-

- - - i^_^ 1

V, 2 •---^.__

^ ^ ^ ^ - 1 ^ . - -^ ••- ^ u ~

1 ^5

3

_''~~rrt

-J

k =1 2

— _ 0.5 1.5 2 0 0.5 1.5

Fig.3.10-21. Ci versus t demonstrating the effect of k2 for ki = k3 = 10 and At = 0.01

3,10-22

where

Ai-^A2 + A3 k2

A2 + A3 -4 A6

A3 + A4 ^ A5 + A6

A2 + A5 -^ A6

Tj = - k^Cj 12 = k |Ci - k2C2C3 -- k4C2C5

13 = k |Ci - k2C2C3 - k3C3C4 14= - k3C3C4

^5 "= '^3C3C4 - k4C2C5

^6 "= ^':^'^z + ^30304 + k4C2C5

yields the following transition probability matrix:

(3.10-22)

(3.10-22a)

Page 287: Applications of Markov Chains in Chemical Engineering

281

P =

1

2

3

4

5

6

1 l-kjAt

0

0

0

0

0

2 kjAt

P22

0

0

0

0

3 kjAt

0

P33

0

0

0

4

0

0

0

l-k3C3(n)At

0

0

5

0

0

lk3C4(n)At

yk3C3(n)At

l-k4C2(n)At

0

6

0

|[k2C3(n)

+k4C5(n)]At

^[k2C2(n)

+k3C4(n)]At

ik3C3(n)At

•i.k4C2(n)At

1 (3.10-22b)

where P22 = 1 - [k2C3(n) + k4C5(n)]At

P33 = 1 - [k2C2(n) + k3C4(n)]At (3.10-22c)

The transient response of Ci to C6 for the initial state vectors C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0), C6(0)] = [1, 0, 0, 1, 0, 0] and [1, 0, 0, 1, 1, 0] is

depicted in Fig.3.10-22 where the effect of C5(0) is demonstrated.

2

1.5

0.5

0

i = 6-

^ \ l \ '

C (0) = 0 '

- '

^ - - - ~ - ^

0 1 3 0 t t

Fig.3.10-22. Ci versus t demonstrating the effect of CsCO)

for ki = 5 (i = 1, ..., 5) and At = 0.02

An exact solution is available [31, p. 80] only for extreme conditions.

Page 288: Applications of Markov Chains in Chemical Engineering

282

3.10-23 Ai + A2 -^ A3+ A6

Ai + A3 ^ A4+ A7 k3

Ai + A2 -4 A5 + A7

k4 Ai + A5 -^ A4 + A6

where [32, vol.2, p.67; 40]

tj = - (kj + k3)C2 - kjCj - k4C5 T2= - (ki + k3)C2

13 = kjC2 - k2C3 r4 = k2C3 + k4C5

^5 ~ '^3^2" ^ 4 5 Tg = kiC2 + k4C5 r-j = k2C3 + k3C2

yields the following transition probability matrix:

(3.10-23)

(3.10-23a)

P =

where

1

2

3

4

5

6

7

1

Pii

0

0

0

0

0

0

(ki

2

0

1 -+ k3)At

0

0

0

0

0

3

Pl3

| k , A t

l-kjAt

0

0

0

0

C,(n)

4

Pl4

0

ik^At

1

Ik^At

0

0

C.(n)

5

Pl5

ik3At

0

0

l-k4At

0

0

, Cc(n)-

6

PI6

lk ,At

0

0

^ M t

1

0

7

Pl7

ik3At

l k , A t

0

0

0

1

(3.10-23b)

K C '2(n) .

ir C2(n) CsCii)-] ![- C,(n) C2(n)l» .0 1m^^^ P'^ = IL* ! C > ) ^ •' CM^ P'^ == iL'^^ C > ) ^ "3 c l ^ j A t (3.10-23d)

Page 289: Applications of Markov Chains in Chemical Engineering

283

The transient response of Ci to C7 for the initial state vector C(0) = [Ci(0),

C2(0), ..., C7(0)] = [1, 0.5, 0, 0, 0, 0, 0] is depicted in Fig.3.10-23 where the

effect of ki is demonstrated.

2.5 0

Fig.3.10-23. Ci versus t demonstrating the effect of ki

for ii2 = k4 = land ka = 2

For At = 0.0025, the agreement between the Markov chain solution and the

exact solution [32, vol.2, p.67; 40] is Dmax = 4.9% and Dmean = 2.4%. It should

be noted that in the exact solution, x should be replaced by t.

3,10-24 ^ A . > l .Ag

^-Aio >^A 12" M3

(3.10-24)

where

Page 290: Applications of Markov Chains in Chemical Engineering

284

ri = - ki2Ci r2 = - (k23 + k24 + k25)C2 + ki2Ci

13 = - (k36 + k37 + k3g)C3 + k23C2

14 = — (k45 + k47 + k4g)C4 + k24C2

5 ~ ~ ( 56 + ^ 57 " 1^58)^5 + k25C2

6 = " ( 69 + ^^,10 " ^511)05 + k35C3 + k46C4 + k56C5

Ty = - (k79 + k-jiQ + k7j i )C7 + k37C3 + k47C4 + k57C5

ig = - (kgQ + kg 10 + kg i i)Cg + k3gC3 + k4gC4 + k5gC5

TQ = - k9 12C9 + k59C5 + k79C7 + kg9Cg

^ 10 = - ^10,12^10 "•• 6,10^6 + ^7 10C7 + kgjoCg

^11="" ^11,12^11 + ^M^6 "•• k7,iiC7 + kg i iCg

^12 = "" 12,13^12"*" ^9,12^9"*" ^10,12^10 + k ^ 12^11

^ 13 = ^12,13^12

yields the following transition probability matrix:

P =

(3.10-24a)

1 2 3 4 5 6 7 8 9 10 11 12 13

1 P i i

0 0 0 0 0 0 0 0 0 0 0 0

2 P12

P22

0 0 0 0 0 0 0 0 0 0 0

3 0

P23

P33

0 0 0 0 0 0 0 0 0 0

4 0

P24

0 P44

0 0 0 0 0 0 0 0 0

5 0

P25

0 0

P55

0 0 0 0 0 0 0 0

6 0 0

P36

P46

P56

P66

0 0 0 0 0 0 0

7 0 0

P37

P47

P57

0 P77

0 0 0 0 0 0

8 0 0

P38

P48

P58

0 0

Pss 0 0 0 0 0

9 0 0 0 0 0

P69

P79

P89

P99

0 0 0 0

10 0 0 0 0 0

P6,10

P7.10

P8,10

0 PlO.lO

0 0 0

11 0 0 0 0 0

P6,ll

P7,ll

P8,ll

6 0

P u . i i

0 0

12 0 0 0 0 0 0 0 0

P9,12

PlO.lO

P11.12

P12.12

0

13 0 0 0 0 0 0 0 0 0 0 0

Pl2,13

1

where

pll = l-ki2At; pi2 = ki2At

P22 = 1 - (k23+k24 + k25)At p23 = k23At p24 = k24At P25

P33 = 1 - (k36 + k37 + k38)At p36 = k36At P37 = ksTAt P38

I

(3.10-24b)

= k25At

= k38At

Page 291: Applications of Markov Chains in Chemical Engineering

285

P44 = 1 - (k46 + k47 + k48)At p46 = k46At P47 = k47At p48 = k48At

P55 = 1 - (k56 + k57 + k58)At p56 = ksgAt P57 = ks-jAt psg = ksgAt

P66 = 1 - (k69 + k6,10 + k6,l l)At P69 = k69At p6,10 = ke.loAt

P6,ll='f6,llAt

P77 = 1 - (k79 + k7,io + k7,i i)At P79 = k79At P7jo = k7,10At

P7, i i=k7, i lAt

P88 = 1 - (k89 + k8,lO + k8,ll)At p89 = k89At p8,io = k8,ioAt

P8, l l=k8, l lAt

P99 = 1 - k9,i2At p9,i2 = k9,i2At; pio,10 = 1 - kio,12At pio,12 = kio,12At

Pll.ll = 1 -kii,i2At Pii,i2 = kii,i2At

P12,12 = 1 - ki2,i3At pi2,i3 = ki2,l3At (3.10-24c)

The transient response of Ci to C13 C(0) = [Ci(0), C2(0),..., Ci3(0)] = [1,

0, 0, ..., 0] is depicted in Fig.3.10-24.

0.8

0.6 H

0.4 h

0.2

T

|1

\ \

f K / \ ^

V 3^V <^v/

1

y ^

/ /

/ l'^

/

/

. ' 6 / /

/ / -,.5

-. 4

' J~^

1:C 1

2:C^ 3:C =C =C

3 4 5

4:C =C =C 6 7 8

5:C =C =C 9 10 11 6:C

12 7:C

13

- — i

-

0.5 1.5

Fig.3.10-24. Ci versus t for k n = 15, ky = 5 and At = 0.01

A complicated exact solution is available [22, p.59].

Page 292: Applications of Markov Chains in Chemical Engineering

286

3.10-25 A + B _>AB k2

AB + B _> AB2 k3

AB2 + B -> AB3

ABz-i+B ^ A B z

where [41]

B = ~ I^JCACB - k2CABCB ~ ^30^1^5, 3 - kvC Z'^ABZ_,CB

(3.10-25)

l AB - • 'ICA^B - k2CABCB rAB2 " '^2CABCB " k3CAB2CB

^ABj = k3CAB2CB - k4CAB3CB

I'AB^,, - ^Z-ICAB^.I^B - kzCAB2_,^B ABz " kzCAB2_,CB (3- 10-25a)

yields the following transition probability matrix for Z = 4:

A B AB

P =

A

B

AB

AB2

AB3

AB4

AB2

k,c'(„)At « i' ' B " ^ «

AB3

0

AB4

0

0 PBB |k,CA(n)At ik2CAB(n)At IkjCAB/n)^ ik4CAB/n)At

0 0

0 0

^ lk2CB(n)At 0 k2CB(n)At

0 0 0 l-k3CB(n)]At i.k3CB(n)]At

0 l-k4CB(n)]At l.k4CB(n)]At

(3.10-25b)

Page 293: Applications of Markov Chains in Chemical Engineering

287

where PBB = l-tkiC^di) + k2CAB(n) + 1^30^3/") + k4CAB/n)]At (3.10-25c)

The transient response of A to AB4 is depicted in Fig.3.10-25 where the effect of CA(0) is demonstrated.

1.5

u"" 1

0.5

0

1 ' C (0)=1 ' \

\ _ B

\ - ^ V " '' <:: - ^ A A B •~'~ -- _ V ^ . - - ^ ' - ' - - A B ^

^ ' ' . . . . . ,3 -L ' ^ ^ - ~ - ~ - * - . - 1 - - ,- -

;

AB / 4

1 "~~ '"^i'-^i^

.. 1 • ' 1

\ - ' - ~

1

r BV A

C (0) = 2 ' A

AB

AB AB AB

^ : • : : : ' : , : ' : •

-

-

0 0.5 1 1.5 0 0.5 1 t t

Fig.3.10-25. Ci versus t demonstrating the effect of CA(0) for ki = 10, ki = 5 (i = 1, 2, 3) and At = 0.005

A complicated exact solution is available [41].

3.11 PARALLEL REACTIONS: SINGLE AND CONSECUTIVE REVERSIBLE REACTION STEPS

1.5

3.11-1

> ^ 1

(3.11-1)

where

rj = - (kj + k2)Ci + k_iC2 r2 = kjC] - k_iC2

yields the following transition probability matrix:

(3.11-la)

Page 294: Applications of Markov Chains in Chemical Engineering

288

P =

1

2

3

1 - (kj + k2)At

k_iAt

0

kjAt

1 - k_iAt

0

k2At

0

1 (3.11-lb)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1,0,0] is depicted in Fig.3.11-1 where the effect of k.i is

demonstrated.

1

0.8

0.6

0.4

0.2

0

u

\i =

r

:1

/ /

1

" - 2

I

1 k =

-1

1

= 4 -

1 •

:i_jr-—-___

0 0.5 1.5 2 0

Fig.3.11-1. Ci versus t demonstrating the effect of k-i

for ki = 5 and k2 = 3

For At = 0.005, the agreement between the Markov chain solution and the

exact solution [33, p.73] is Dmax = 1-5% and Dmean = 0.9%.

3.11-2 k, ^ A ,

Ai

(3.11-2)

where

ri = 12 = - (k, + k2)CiC2 + k_iC3 rg = kiCjCj - k_iC3

r4 = k2CiC2 (3.11-2a)

Page 295: Applications of Markov Chains in Chemical Engineering

289

yields the following transition probability matrix:

1

2

3

4

1 1-

(ki+k2)C2(n)At

0

k_iAt

0

2

0

1-(ki+k2)Ci(n)At

k.jAt

0

3 4 ikiC2(n)At lk2C2(n)At

ikiCi(n)At •i-k2Ci(n)At

l - k . j A t 0

0 1 (3.11-2b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.11-2 where the effect of

C2(0) is demonstrated.

u

1

0.8

0.6

0.4

0.2

0

' k =4 -1

fc =C - " '

rt-K y \ ^

/ S .

F e- r ' "'" "—•— 3 ^ - ^

1

4 -\

_

-

u

t 1

0.8

0.6

0.4

0.2

0

6 0

' k =1 ' -1

ic =c

1 1 1

- ' "c 4

• - « = = : — _ _ _ _ _ _

\c =c

\K'" f\^_^

' k =0 ' 1 -1

c 3 . . . _ i

c 4 —

T ~ • — 1

Fig.3.11-2. Ci versus t demonstrating tlie effect of k-i for ki = 5, k2 = 3 and At = 0.005

Page 296: Applications of Markov Chains in Chemical Engineering

290

3.11-3

where

Jf" -2

kl

ri = - (kj + k2)C, + k_iC2 + k_2C3

r2 = kjCi - k-iC2 r3 = k2Ci-k_2C3

yields the following transition probability matrix:

(3.11-3)

(3.11-3a)

p =

1

2

3

1 l-(ki+ k2)At

k_iAt

k_2At

2 kiAt

l-k_jAt

0

3 k2At

0

l-k_2At (3.11-3b)

The transient response of Ci to C3 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0)] = [1, 0, 0] is depicted in Fig.3.11-3 where the effect of ki and k-i is

demonstrated.

u

1

0.8

0.6

0.4

0.2

0

L A 1 - 1 I

I \ I

0 0.1 0.2 0.3 t

\ ' -\

\ 1 [- ^

r c >

Ks_

\.^

1 1 1 k =l,k =5 1 -1

'^~z:zr-:=~rz=~

~\ 1 1

-H

H

"1

— -

0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 t

Page 297: Applications of Markov Chains in Chemical Engineering

291

u

1

0.8

0.6

0.4

0.2

«.

\ 1

_ \

1 3

— 1 —

1 0.2

1 1 i 1 k = 5 , k = 1

1 -1 1

^ ^ ^ 2

c — ^

1 1 1 0.4 0.6 0.8 ]

t

Fig.3.11-3. Ci versus t demonstrating the effect of ki and k-i

for k2 = k-2 = 5

For At = 0.01, the agreement between the Markov chain solution and the

exact solution [54, p. 140] is Dmax = 3.8% and Dmean = 1.3%.

3.11-4

A1+A2. '

-2

where [35, p. 149] rj = rj = - (ki + k2)CiC2 + k_,C3 + k_2C4

r3 = kiCjC2 - k_jC3 r4 = k2CjC2 - k_2C4

(3.11-4)

(3.11-4a)

yields the following transition probability matrix:

P =

1

2

3

4

1 1-

(ki+k2)C2(n)At

0

k_iAt

k_2At

2

0

1-(ki+k2)Ci(n)At

k_iAt

k_2At

3 lkiC2(n)At

lkiCi(n)At

1 - k_iAt

0

4 lk2C2(n)At

ik2Ci(n)At

0

1 - k_2At (3.11-4b)

Page 298: Applications of Markov Chains in Chemical Engineering

292

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.11-4 where the effect of ki

and k-i is demonstrated.

1

0.8

0.6

0.4

0.2

0

u

y 1

h- X. ^ ^^

V 1

1 1 1 1 k =k =5

1 -1

C =C H

-

C =C

3 4

i 1 1 0 0.1

\ 1

L " - -k

1 /

U - r

1 = l ,k =

-1

c =c 1 2

c 4

c 3 1

1 5

1

~^_

0.4 0.5

Fig.3.11-4. Ci versus t demonstrating the effect of ki and k-i

for k2 = k-2 = 5 and At = 0.01

3.11-5

(3.11-5)

Page 299: Applications of Markov Chains in Chemical Engineering

293

where [35, p.lOl]

rj = - (kj + kj + k3)Ci + k_,C2 + k_2C3 + k_3C4

rj = kjCi - k_jC2 r3 = k2Ci-k_2C3 14 = k3Ci - k_3C4 (3.11-5a)

yields the following transition probability matrix:

p =

1

2

3

4

1 1-

(ki+k2+k3)At

k_iAt

k^2^t

k_3At

2 kjAt

1 - k.jAt

0

0

3 k2At

0

1 ~ k_2At

0

4 k3At

0

0

1 - k_3At (3.11-5b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1,0, 0, 0] is depicted in Fig.3.11-5 where the effect of k-i

is demonstrated.

0.5 1 1.5 0 0.5 t t

Fig.3.11-5. Ci versus t demonstrating the effect of k-i

for ki = k-3 = 5, k2 = k-2 = ks = 1 and At = 0.01

1.5

3.11-6 Ai-^A2

-^ 2 A i - ' A 3 (3.11-6)

Page 300: Applications of Markov Chains in Chemical Engineering

294

where:

Ti = - (kjCi + 2k2Cf) + k^jCj r2 = k,Ci

r3 — 2k2Ci - k_2C3

yielding the following transition probability matrix:

(3.11-6a)

1

P = 2

3

1 2 3 l-[ki + 2k2Ci(n)]At kjAt 2k2C,(n)At

0 1 0 k_2At 0 l-k_2At (3.11-6b)

where pi3 was computed by Eq.(3-10a). The transient response of Ci, C2 and C3

for the initial state vector C(0) = [Ci(0), C2(0), €3(0)] = [1, 0, 0] is depicted in

Fig.3.11-6 where the effect of k-2 is demonstrated.

Fig.3.11-6. Ci versus t demonstrating the effect of k-2

for ki = k2 = 5 and At = 0.01

3.11-7

where [35, p.3]

A l ^ A 2

k., k2

Ai + A2 ^ A3

K (3.11-7)

Page 301: Applications of Markov Chains in Chemical Engineering

295

rj = - (kjCi + kjCjCj) + k_,C2 + k_2C3

rj = - (kjCjCj + k_,C2) + k]Ci + k_2C3

yields the following transition probability matrix:

(3.11-7a)

1

1

P = 2

3

1, l-[ki+k2C2(n)]At kiAt ^k2C2(n)At

k_iAt l-[k_i+k2Ci(n)]At lk2Ci(n)At

k_2At k_2At l-k_2At (3.11-7b)

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0)] = [1,0, 0] is depicted in Fig.3.11-7 where the effect of k-2 is

demonstrated for ki = k-i = k2 = 5 and At = 0.01.

- \ , \

_ 2

(^ 1

1 1 k =0.5

-2

1 1

1

1 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8

t t Fig.3.11-7. Ci versus t demonstrating the effect of k-2

3,11-8

where [28]

Ai + A2 A3 A3 + A2 -^ A4 (3.11-8)

Page 302: Applications of Markov Chains in Chemical Engineering

296

12 = - (kjCiCz + kjCzCj) + k_,C3

rj = kiCjCj - (k_iC3 + kjCjCj) 14 = kjCjCj

yields the following transition probability matrix:

(3.11-8a)

P =

1 l-kiC2(n)At

0

k_,At

2

0 lk,C2(n)At

4

0

[k,C,(n)+k,C,(n)]At 2 ' ' '^ '^"^^ '

k_iAt

0

1 -

LK_IH~K2V_'2'

^k2C3(n)At

, M. |k3C2(n)At (n)]At 2 ^ ^

0 1 (3.11-8b)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0), C2(0),

C3(0), C4(0)] = [1, 1, 0, 0] is depicted in Fig.3.11-8 where the effect of k-i is

demonstrated.

1

0.8

0.6

0.4

0.2

^0 0.5 1 1.5 0 0.5 1 t t

Fig.3.11-8. Ci versus t demonstrating the effect of k-i

for ki = k2 = 5 and At = 0.01

u

[ 1 1

2 • —

1 J-^ - - " " "

-\

-

Page 303: Applications of Markov Chains in Chemical Engineering

297

3.11-9 Ai + A2 A3 + A4

Ai + A5 " A3 + A6

where Tj — — (kjC|C2 + '^2^1^5-' ''" '^-1^3^4 ''" *^2^3^6

r2 = — kjC-jC-2 " k_jC3C4

r3 = (k_iC3C4+ k_2C3C6) + k , C , C 2 + kjCjCs

14 = K|C|C2 ~ k_jC3C4

15 = k2C|C5 - k_2C3C5

yields the following transition probability matrix:

1

P =

1-

[kjCjCn) +

k2C5(n)]At

0 l-kiCi(n)At ikiCi(n)At i-k,Ci(n)At 0

(3.11-9)

(3.11-9a)

2 3 4 5 6 0 ^{^iC^in) ykiC2(n)At 0 Y^2C5(n)At

+k2C5(n)]At

P31 ik_iC4(n)At l-[k_iC4(n) ^ ik_2C6(n)At Q +k_2C6(n)]At

P41 P41 0 l-k_iC3(n)At 0 0

0 0 |k2Ci(n)At 0 l-k2Ci(n)At ik2Ci(n)At

0 0 ~k^2C3(n)At l-k_2C3(n)At

(3.11-9b)

U-k_2C3(n)At 0

where P31 = y[k_iC4(n) + k_2C6(n)]At; P41 = ik_iC3(n)At (3.11-9C)

Page 304: Applications of Markov Chains in Chemical Engineering

298

The transient response of Ci to Ce for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0), C6(0)] = [1, 1, 0, 0, 1, 0] is depicted in Fig.3.11-9

where the effect of ki is demonstrated.

c\ r ' \ L c x r 3

/

r/ ; . - '

1 1 c c • ...

c 6

' 'c 4

1 1

1 1 k =1

1

-rrr-r-:. - ^

1 1 0 0.2 0.4 0.6 0.8

t Fig.3.11-9. Cj versus t demonstrating the effect of ki

for k-i = k2 = k-2 = 2 and At = 0.005

An extremely complicated exact solution is available [32, vol. 2, p.49].

3.11-10 Ai + A 2 ^ A 3

where

Ai + A 4 ^ A 5

A3 + A4 ^ A 6

ri = - (kjCiCj + k2CiC4) + k_]C3

^2 ^ — '^1^1^2 " — 1 3

r3 = - (k_iC3 + k3C3C4) + k,C,C2

14 = — vk2CjC4 + k3C3C4)

15 = k2CiC4 r = k3C3C4

yields the following transition probability matrix:

(3.11-10)

(3.11-lOa)

Page 305: Applications of Markov Chains in Chemical Engineering

299

P =

1 2 3 4 5 6 l-[kiC2(n)+ Q lkiC2(n)At o ^k2C4(n)At o |k2C4(n)]At

0 l-kiCi(n)At ikiCi(n)At

k_iAt k_iAt l-[k_i+ k3C4(n)]At

0 •i.k3C4(n)At

0

0

0

0

0 l-[k2C,(n)+ lk2Ci(n)At lk3C3(n)At k3C3(n)]At

0 0

0 0

(3.11-lOb)

The transient response of Ci to C^ for the initial state vectors C(0) = [Ci(0),

C2(0), C3(0), C4(0), C5(0), C6(0)] = [1, 1, 0, 1, 0, 0] and [1, 1, 0, 0, 0, 0] is

depicted in Fig.3.11-10 where the effect of €4(0) is demonstrated.

1

0.8

0.6

0.4

0.2

^0 0.5 1 1.5 0 0.5 1 t t

Fig.3.11-10. Ci versus t demonstrating the effect of €4(0)

for ki = k.i = k2 = k3 = 5 and At = 0.01

u

U I \

l \ ' \ l ' -V \ '

— \ ^ '\ '*'

k^.> L^

1

5 .

v 4

1

C(0) = 4

, - -

• ' • • ^ ~

-

1 '

" - • •

6 ^ ^ - -

~~ l~"

^

'

— -r=r-~::::r-.-~

An exact solution is available [32, vol. 2, p.75] only for limiting cases.

Page 306: Applications of Markov Chains in Chemical Engineering

300

3,11-11

where

rJ = — (k|2 + k|3 + k |4)C| + k2iC2 + ^3^03 H- K41C4

12 = - (k2i + k23 + k24)C2 + ki2Ci + k32C3 + k42C4

13 = - {k^i + k32 + k34)C3 + ki3Ci H- k23C2 + k43C4

14 = - (k4i H- k42 H- k43)C4 + ^i^Ci + k24C2 + k34C3

yields the following transition probability matrix:

(3.11-11)

(3.11-lla)

1 1-

[ki2+ki3+ki4]At

k2iAt

k3iAt

k4iAt

2 ki2At

1-[k2i+k23+k24]At

k32At

k42At

3 ki3At

k23At

1-[k3i+k32+k34]At

k43At [k4

4 ki4At

k24At

k34At

1-l+k42+k43]At

1

2 P =

3

I LK4i+K42+K43ja

(3.11-llb)

The transient response of Ci to C4 for the initial state vector C(0) = [Ci(0),

C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.11-11 where the effect of

ki2 is demonstrated.

Page 307: Applications of Markov Chains in Chemical Engineering

301

u

1

0.8

0.6

0.4

0.2

0 "(

K ^ 1 \ k =1 \ ^

z^" """C^C =C y^ 2 3 4

X 1 1 3 0.5 1 t

r

-4-

A 1.5 0

1 1 k =10

12

--^c^

c =c 3 4

1 i 0.5 1

t

-

1

Fig.3.11-11. Ci versus t demonstrating the effect of ki2 for kij = 1 (U 5t 12) and At = 0.01

An exact solution is available [31, p. 172; 52].

3.12 CHAIN REACTIONS

3.12-1 ks.

where

"kH^ 3

\K,

ri = - (ki + k2)Ci

rj = - (k3 + k4)C2+ kiCj rj = - (kj + k6)C3+ kjCj

14 = k3C2 Tj = k4C2 Tg = k5C3 Tj = k6C3

(3.12-1)

(3.12-la)

yields the following transition probability matrix:

Page 308: Applications of Markov Chains in Chemical Engineering

302

p =

1

2

3

4

5

6

7

1 l-[ki+k2]At

0

0

0

0

0

0

2 kjAt

l-[k3+k4]At

0

0

0

0

0

3 k2At

0

l-[k5+k6]At

0

0

0

0

4

0

k At

0

1

0

0

0

5

0

k4At

0

0

1

0

0

6

0

0

ksAt

0

0

1

0

7

0

0

k^At

0

0

0

1

(3.12-lb) The transient response of Ci to C7 for the initial state vector C(0) = [Ci(0),

C2(0),..., C7(0)] = [1, 0,..., 0] is depicted in Fig.3.12-1 where the effect of ki is demonstrated.

i 1

V 1

k =10 1

7 6

5

4

H

. , -1

0.5 1 1.5 0 0.5 1 1.5 t t

Fig.3.12-1. Ci versus t demonstrating the effect of ki for k2 = 2, k3 = 3, k4 = 4, ks = 6 and At = 0.005

An exact solution is available [22, p.52].

Page 309: Applications of Markov Chains in Chemical Engineering

303

3.12-2 ^ ^' }

f vL vL WW W W W

g A9 Alo All A12 Ai3 Ai4 Ai5 (3.12-2)

where

r i = - ( k i 2 + ki3)Ci

2 = - (k24 + k25)C2 + ki2Ci

14 = — (k4g + k49)C4 + k24C2

6 = "" (k^ 12 + k i3)C6 + k36C3

8 k4gC4 Tg = k49C4

42 - k6,12C6 "13 = k< 6,13^6

13 - - (k36 + k37)C3 + ki3Ci

5 = ~ (k5 10 + k5 ii)C5 H- k25C2

17 = - (k7 14 + k7 i5)C7 + k37C3

^io = k5 10C5 rii = k5 11C5

ri4 = k7,i4C7 ri5 = k7,i5C7 (3.12.2a)

yields the following transition probability matrix:

P =

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 Pii 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2 P12

P22 0 0 0 0 0 0 0 0 0 0 0 0 0

3 Pl3 0 P33 0 0 0 0 0 0 0 0 0 0 0 0

4 0 P24 0 P44 0 0 0 0 0 0 0 0 0 0 0

5 0 P25 0 0 P55 0 0 0 0 0 0 0 0 0 0

6 0 0 P36 0 0 P66 0 0 0 0 0 0 0 0 0

7 0 0 P37 0 0 0 P77 0 0 0 0 0 0 0 0

8 0 0 0 P48 0 0 0 1

0 0 0 0 0 0 0

9 0 0 0 P49 0 0 0 0 1

0 0 0 0 0 0

10 0 0 0 0

P5,10

0 0 0 0 1

0 0 0 0 0

11 0 0 0 0

P5,ll 0 0 0 0 0 1

0 0 0 0

12 0 0 0 0 0

P6,12

0 0 0 0 0 1

0 0 0

13 0 0 0 0 0

P6,13

0 0 0 0 0 0 1

0 0

14 0 0 0 0 0 0

P7,14

0 0 0 0 0 0 1

0

15 0 0 0 0 0 0

P7,15

0 0 0 0 0 0 0 1

(3.12-2b)

where

Pll = l - ( k l 2 + ki3)At pi2 = ki2At pi3 = ki3At

P22 = 1 - (k24 + k25)At P24 = k24At P25 = k25At

P33 = 1 - (k36 + k37)At P36 = k36At P37 = ks-jAt

Page 310: Applications of Markov Chains in Chemical Engineering

304

P44 = 1 - (k48 + k49)At P48 = k^gAt P49 = k49At

P55 = 1 - (k5,io + k5ji)At p5,io = k5,ioAt p5,ii = ks^nAt

P66 = 1 - (k6,12 + k6,13)At P6,12 = k6,12At P6,13 = k6,13At

P77 = 1 - (k7,i4 + k7j5)At P7,i4 = k7,i4At P7,i5 = k7,i5At (3.12-2C)

The transient response of Ci to C15 for the initial state vector C(0) = [Ci(0),

C2(0),..., Ci5(0)] = [1, 0, ..., 0] is depicted in Fig.3.12-2 where the effect of ki3

is demonstrated.

0.8 k \ l

0.6

U

0.4

0.2

\r'

6:C =C =C =C 9 10 11

7:C =C =C =C 12 13 14 V.

0.5 1.5

Page 311: Applications of Markov Chains in Chemical Engineering

305

0.8 h \ l

0.6

U

0.4

0.2

^13 = 2

1:C 1

2:C =C 2 3

3:C =C =C =C 4 5 6 7

4:C =C =C =C = 8 9 10 11

c =c =c =c 12 13 14 15

0.5 1.5

Fig.3.12-2. Ci versus t demonstrating the effect of ki3 for ki2 = 2,

kij = l(ij ^ 12,13) and At = 0.01

An exact solution is available [22, p.70].

3.13 OSCILLATING REACTIONS [55-69]

3 .13-1 (Ai)gas -> A i ki

Ai-->A2

A2-^A3 (3.13-1)

where [55] (Ai)gas denotes a saturated vapor of gas A] in equilibrium with its

condensed phase containing species Ai, A2 and A3. It is assumed that equilibrium

between the phases is established immediately and that the condensed phase is

perfectly mixed so that diffusion effects are negligible. H is the rate of supply of Ai

in moles/sec from the vapor phase (Ai)gas into the condensed phase. The

governing equations are:

Page 312: Applications of Markov Chains in Chemical Engineering

306

rj = H - kC, = H - kjCjC,

2 ^ ^^1 ~" ^^2^2 ~ '^1^2^1 "~ ^2^2

13 = k2C2 (3.13-la)

The fact that k = kiC2 indicates species A2 influence autocatalytically its own rate of

formation. The above equations yield the following transition probability matrix:

P =

1 2 3

1 - [kiC2(n)]At + [H/Ci(n)]At kiC2(n)At 0

0 1 - k2At k2At

0 0 1 (3.13-lb)

The term [H/Ci(n)]At, where H is a constant supply rate of Ai, must be added in

pil in order to comply with the integrated form of ri in Eq.(3.13-la).

The transient response of Ci, Ci exact and C2 is depicted in Fig.3.11-3 where

the effect of ki and H is demonstrated. The initial state vector is C(0) = [Ci(0),

C2(0)] = [0.5, 1].

50 100 t

200 400 600 800 1000 lOOOOt

Page 313: Applications of Markov Chains in Chemical Engineering

307

1.5

U

0.5 k

A , i.'

c 1,exact

K /^\

1

C 2

1

I 1

k =2, H = 0.5 1

(c)

1 1 10 20

t 30 40

Fig.3.13-1. Ci versus t demonstrating the effect of ki and H

for k2 = 3

An exact solution is available [55]. However, it is restricted to relatively

large values of t. Indeed, as observed in case (a), the exact solution does provide

reasonable results for small t, i.e. Ci < 0. In addition, the agreement between the

Markov chain solution and the exact solution depends on the parameters ki, k2 and

H as shown in Fig.3.13-la,b,c for At = 0.08, 0.00001 and 0.02, respectively. It

should be noted that oscillations occur when H < 4k2^/ki as observed in Fig.3.13-

la,c. A non-oscillatory behavior occurs when H > 4k2^/ki as shown in Fig.3.13-

Ib. This has been obtained by varying ki from 0.1 to 400.

3 > 1 3 - 2 Ai + A2 -> A3 + A4

A3 + A2 -^ 2A4

Ai + A3->2A3 + A5

2A3 -^ Ai + A6

A5^fA2 (3.13-2)

where for the above reaction, known as the Belousov-Zhabotinski reaction [59],

Page 314: Applications of Markov Chains in Chemical Engineering

308

T2 = - kjCiCj - k2C2C3 + fkjCs

r3 = kjCiCz - k2C2C3 + k3CiC3 - 2k4C3

r5 = k3CiC3-k5C5 r6 = k4C? (3.13-2a)

For a detailed derivation, the attention of the reader is addressed to case 5.2-1(1).

The following transition probability matrix is obtained:

1 l-[kiC2(n)+

Lc3(n)]At

0

k4C3(n)At

0

0

0

2

0

l-[kiCi(n)+

k2C3(n)]At

0

0

fkjAt

0

3 [i-kiC2(n)+

2k3C3(n)]At

ikiCi(n)At

l-[k2C2(n)+

k3Ci(n)+

2k4C3(n)]At

0

0

0

4 ikiC2(n)At

•i-[kiCi(n)+

2k2C3(n)]At

k2C2(n)At

1

0

0

5 lk3C3(n)At

0

ik3Ci(n)At

0

l-ksAt

0

6

0

0

k4C3(

0

0

1

p =

4

5

6 I

(3.13-2b)

where p3i = p36 was computed by Eq.(3-10a). The transient response of C3 and

C5 is depicted in Fig.3.13-1 where the effect of f in Eq.(3.13-2) is demonstrated

for At = 5x10-5, ki = 0.1, k2 = 6xl08, ks = L5xl05, k4 = 4xl08 and ks = 5x10^

and the initial state vector C(0) = [0.05, 10-6, 10-12,0, 10-12, Q].

Page 315: Applications of Markov Chains in Chemical Engineering

309

t — t

oo w-> O

so «n

u- 2

o

cs - ;

, \

c 5

1 1

,* ' ;' '. ;

' .

c 3

V "—^

1

—._^ T

1

.-' '•--• - J

-f = 0.530

-

1

-

i ,.„/,.

.• 1 1

C ' 5

. c

1 1

-

f = 0.535 1

1 1 0 0.01 0.02 0.03 0.04 0.05 0 0.01 0.02 0.03 0.04 0.05

t t

Fig.3.13-2. Ci versus t demonstrating the effect of f

The differential Eqs.(3.13-2) were solved numerically [59] for the case of a

continuous perfectly mixed reactor.

3,13-3 Ai + A2 "^ A3

2 A2 -^ A4

A3 ->A2

A3 + A4 -> 3A2

where [65]

A 4 - ^ A5

rj = - k |CiC2 + k_iC3

^2^^ — k l ^ l ^ 2 " — 1 3 " 2 3 — '^'^3^2 "'" •^J'^4^3^4

r3 = kiC|C2 - k_|C3 - k2C3 - k4C3C4

^4 ~ ^302 - k4C3C4 - k5C4

^5 = ^^5^4

(3.13-3)

(3.13-3a)

For a detailed derivation, the attention of the reader is addressed to case 5.2-1(4).

The following transition probabiUty matrix is obtained:

Page 316: Applications of Markov Chains in Chemical Engineering

310

P =

1 2 3 4 l-kiC2(n)At 0 ^kiC2(n)At o

0

5

0

l-[kiCi(n)+ lkjCi(n)At k3C2(n)At 2k3C2(n)]At

k_iAt f V l-[k_i+k2+ Q

|k4C4(n)]At k4C4(n)]At

Q l~[k4C3(n)+

kslAt

0 0

0

0

|k4C3(n)At ksAt

(3.13-3b)

where p24 was computed by Eq.(3-10a). The transient response of Ci to C5 for an

initial state vector C(0) = [Ci(0), C2(0), €3(0), €4(0), C5(0)] = [1, 1, 0, 0, 0] is

depicted in Fig.3.13-3 where the effect of ks in Eq.(3.13-3) is demonstrated for At

= 0.01, ki = 2.5, k.1 = 0.1, k2 = 1, k4 = 10 and ks = 1.

k =0.2 3

10 15 0 t t

Fig.3.13-3. Ci versus t demonstrating the effect of ka

An analytical solution is available only for a simplified case [65].

Page 317: Applications of Markov Chains in Chemical Engineering

311

3>13-4 (Ai)gas-^Ai

2A3 + A2 -^ 3A3

A3-^A4

Ai ->A3

A3->A2

(3.13-4)

where (Ai)gas denotes a saturated vapor of gas Ai in equilibrium with its condensed

phase containing species Ai, A2, A3 and A4. It is assumed that equilibrium

between the phases is established immediately and that the condensed phase is

perfectly mixed so that diffusion effects are negligible. H is the rate of supply of Ai

in moles/sec from the vapor phase (Ai)gas into the condensed phase. The following

equations, a detailed derivation of which without H appears in case 5.2-1(1), are

known as the Brusselator model [60]:

ri = H - k i C i

1*2 - ~ ^20203 + k3C3

r3 = kiC,

'•4 = ^403

fj = kjCi - 2k2C2C3 + SkjCjCg - (kg + k4)C3

The above equations yield the following transition probability matrix:

(3.13-4a)

P =

1

2

3

4

1 2 l-kiAt+[H/Ci(n)]At Q

0 l-k2Cf(n)At

0 kjAt

0 0

3 k,At

k2C|(n)At

l-(k3+k4) At

0

4

0

0

k4At

1 (3.13-4b)

The term [H/Ci(n)]At, where H is a constant supply rate of Ai, must be added in

pil in order to comply with the integrated form of ri in Eq.(3.13-4a).

The calculation of P33 requires some clarification since according to Eq.(3.13-

4), A3 is simultaneously consumed and formed. This fact must be taken into

Page 318: Applications of Markov Chains in Chemical Engineering

312

account in calculating P33 in order to satisfy the result obtained by integration of 13

given by Eq.(3.13-4a). Thus, considering the latter equation, yields

P33 = 1 - [2k2C2C3 + (k3 + k4)]At + (2/3)[3k2C2C32/C3]At

= l - [k3 + k4]At

where the term 1 - [2k2C2C3 + (k3 + k4)]At stands for the probability to remain in

state A3 and (2/3)[3k2C2C32/C3]At is the transition probabiUty from 2A3 to 3A3

for the reaction 2A3 + A2 -^ 3 A3 according to Eq.(3-8).

The transient response of Ci, C2 and C3 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0), C4(0)] = [1, 0, 0, 0] is depicted in Fig.3.13-4 where the

effect of k3 and H in Eq.(3.13-4a) is demonstrated for At = 0.03, ki = 10, k2 = 1,

k 4 = l , H = l a n d 0 .

k = 2 , H = 1 3

40 0 20 t

40

Page 319: Applications of Markov Chains in Chemical Engineering

313

r

-\ ^ ^

/ \ 2y'

f X 1 . , - , „ . 1

,K '

1

,

— J

1

• ' " * " "1

J

k = 2 . 5 , H = 0 3 _

1

20 30 40 0 0.5 t t

Fig.3.13-4. Ci versus t demonstrating the effect of ka and H

An analytical solution is available [60].

3 > 1 3 - 5 (Ai)gas -> Ai

i = 1: Ai + A2 "^ A3 + A4 i = 2: A3 + A2"^2A4

k.

i = 3: Ai + A3 "^ 2A5 i = 4:A6 + A5 '/' A3 + A7

i = 5: 2A3 ^ Ai + A4 i = 6: A7-4gA2 + A6 (3.13-5)

where (Ai)gas denotes a saturated vapor of gas Ai in equilibrium with its condensed

phase containing species Ai to A7. It is assumed that equilibrium between the

phases is established immediately and that the condensed phase is perfectly mixed

so that diffusion effects are negligible. H is the rate of supply of Ai in moles/sec

from the vapor phase (Ai)gas into the condensed phase. The last six equations

describe the modified Oregonator mechanism [57] consisting of six steps. The rate

equations, a detailed derivation of which appears in case 5.2-1(3), are:

Page 320: Applications of Markov Chains in Chemical Engineering

314

ri = H - kiCjCj - le,C3C4 - kgCjCj + k_^cl + kjCj - k_5C,C4

rj = - kiCjC2 + k_iC3C4 - kjCjCj + k_2C^ + gk6C7

Tj = KjL-]C 2 ~ k_|C^3v-4 — k2C 2 3 "'" k_2^4 — k3C ]C 3 + k_3C5 + k4C5Cg

- k_4C3C7 - 2k5C3 + 2k_5CiC4

T^ = kjCjC-2 ~ k_|C3C4 + 2k2C2C3 — 2k_2C4 + k5C3 — k_5CjC4

fj = 2k3(^jv_3 ~ 2k_3L-5 — k/^\^^\^^ + k_4C 3C,'7

r = - k4C5C6 + k_4C3C7 + k6C7

Tj = k4C5C6 - k_4C3C7 - k6C7

which yield the following transition probability matrix:

(3.13-5a)

P =

1

2

3

4

5

6

7

1 2 3 4 5 6 7

Pll 0 Pi3 Pi4 Pi5 0 0

0 P22 P23 P24 0 0 0

P31 P32 P33 P34 P35 P36 0

P41 P42 P43 P44 0 0 0

P51 0 P53 0 P55 0 P57

0 0 P63 0 0 P66 P67

0 P72 0 0 P75 P76 P77 (3.13-5b)

where

Page 321: Applications of Markov Chains in Chemical Engineering

315

Pll = 1 - [kiCzCn) + kgCjdi) + k_5C4(n)]At + [H/Ci(n)]At

Pi3 = [O.SkiCjCn) + k_5C4(n)]At

Pi4 = 0.5kiC2(n)At pi5 = k3C3(n)A

P22 = 1 - [kiCi(n) + k2C3(n)]At P23 = 0.5k,Ci(n)At

P24 = [0.5kiCi(n) + k2C3(n)]At

P31 = [0.5k_iC4(n) + k5C3(n)]At P32 = 0.5k_iC4(n)At

P33 = 1 - [k_iC4(n) + k2C2(n) + k3Ci(n) + k_4C7(n) + 2k5C3(n)]At

P34 = [k2C2(n) + k5C3(n)]At P35 = [k3C,(n) + 0.5k_4C7(n)]At

P36 = 0.5k_4C7(n)At P41 = 0.5k_iC3(n)At

P42 = [0.5k_iC3(n) + k_2C4(n)]At P43 = [k_2C4(n) + k_5Ci (n)]At

P44 = 1 - [k_iC3(n) + 2k^2C4(n) + k_5C,(n)]At

P51 = k.3C5(n)At P53 = [k_3C5(n) + 0.5 k4C6(n)] At

P55 = 1 - [2k_3C5(n) + k4C6(n)]At P57 = 0.5k4C6(n)At

P63 = 0.5k4C5(n)At P66 = 1 - k4C5(n)At P67 = 0.5k4C5(n)At

P72 = gkgAt P75 = 0.5k_4C3(n)At P76 = [0.5k_4C3(n) + k lAt

P77 = 1 - [kg + k_4C3(n)]At (3.13-5c)

The term [H/Ci(n)]At, where H is a constant supply rate of Ai, must be added in

Pll in order to comply with the integrated form of ri in Eq.(3.13-5a).

The transient response of Ci, C2, €5 and C7 for the initial state vector C(0) =

[Ci(0), C2(0), C3(0), C4(0), C5(0), C6(0), CTCO)] = [1, 1, 0. 0, 0, 0, 1] is depicted

in Fig.3.13-5 where the effect of €7(0) = 3 and 1 is demonstrated. The parameters

of the results are [57]: Ci(0) = C2(0) = 1, ki = 1.5, k2 = ks = k4 = ks = 1, k-i = k-

3 = k-4 = k-5 = 0.005, k-2 = 0, kfi = 2, g = 3, H = 1 and At = 0.1.

Page 322: Applications of Markov Chains in Chemical Engineering

10 20 ^0 10 20 t t

Fig.3.13-5. Ci versus t demonstrating the effect of Cj

40

In [57] the equations were integrated numerically for the case where the

reactions take place in a continuous perfectly mixed reactor.

3»13-6 The derivations of this example are detailed and serve as a completion to

chapter 3.2 and in particular to section 3.2-4. Of special importance is the

calculation of the probabilities P33 and P34 elaborated below. The following

reactions are considered showing at some conditions a complicated mixed-mode

behavior [69] .

i = 1: Ai + A2 + A3 -> 2A3

i = 2:

i = 3:

i = 4:

i = 5:

i = 6:

i = 7:

Ai

2A3 -^ 2A4

+ A2 + A4 ^ 2A3 k4

A3-»A5

A 4 ^ A 6

A3,0 -> A3

k'7

Ai,o ^ Ai

k'» i = 8: A2,0 -^ A2 (3.13-6)

Page 323: Applications of Markov Chains in Chemical Engineering

317

where Ai,o indicate that the initial concentration Ci(0) of species Ai (i = 1,2, 3)

remains unchanged. The following kinetic equations may be derived by

considering the above reactions and the basic relationship given by Eq.(3-3):

i = 1: rS"(n) = ~ kS^^Ci(n)C2(n)C3(n) = - kiCi(n)C2(n)C3(n)

= 12 (Xi) = r3 (nj = - -J r3 Kn)

i = 2: I xf\n) = - k?^C (n) = - k2C3 (n) = - ^ ri'\n)

i = 3: if in) = - k f Ci(n)C2(n)C4(n) = - k3Ci(n)C2(n)C4(n)

= r2 (n) = r4 (n) = - ^ r3 (nj

i = 4: r^\n) = - ^^^€3^) = - k4C3(n) = - r?\n)

i = 5: r f (n) = - ki ^C4(n) = ~ k5C4(n) = - r \n)

i = 6: r f (n) = k ^ C3(0) = k'6C3(0) = k6

i = 7: /7\n) = kf^Ci(O) - k_7Ci(n) = k'vCiCO) - k_7Ci(n)

= k7~k_7Ci(n)

i = 8: r ^ n) = k ^ C2(0) = k'8C2(0) = kg (3.13-6a)

From Eq.(3-4) one obtains the following kinetic equations:

r ^ — K|C^|\^2^3 — k3C^ 2^-2^4 — " —7 1 " ^7

r2 = - kiCiC2C3 - k3CiC2C4 + kg

r3 ^ — KjC 10^2^3 " 2 K | C ' | C ^ 2 ^ 3 — 2K2V^3 — ^40-3 H~ 2k3C^|C^2^4 " k^

r4 = — k3C^|CJ2^4 — ^^5^4 "^ 2k2Cx3

5 = ^^4^3

r6 = k5C4 (3.13.6b)

The first term on the right-hand side in r3 indicates consumption of A3 whereas the

second term stands for the formation of A3 according to Eq.(3.13-6) for i = 1. This

presentation is important in the determination of P33, later elaborated. The

following transition probability matrix is obtained on the basis of Eqs.(3.13-6, 6b):

Page 324: Applications of Markov Chains in Chemical Engineering

318

P =

1

2

3

4

5

6

1

Pll

0

0

0

0

0

2

0

P22

0

0

0

0

3

Pl3

P23

P33

P43

0

0

4

0

0

P34

P44

0

0

5

0

0

P35

0

1

0

6

0

0

0

P4

0

1 (3.13-6C)

The calculation of the probabilities was made by applying Eqs.(3-6), (3-9)

and (3-10) as follows:

pl l , the probability of remaining in state Ai, applies Eq.(3-6) for i = 1, 3, 7

inEq.(3.13-6). It is obtained that

PI 1 = 1 - [kiC2(n)C3(n) + k3C2(n)C4(n) + leyjAt + [k7/Ci(n)]At (3.13-6d)

The term [k7/Ci(n)]At, where k-j = k'7Ci(0) is a constant supply rate of Ai, must

be added to p n in order to comply with the integrated form of Eq.(3.13-6b) for ri.

pl3, the transition probability from Ai to A3, is calculated by Eq.(3-9) for i =

1, 3 in Eq.(3.13-6). It is obtained that

P13 = [2(l/3)kiC2(n)C3(n) + 2(l/3)k3C2(n)C4(n)]At (3.13-6e)

P22, the probability of remaining in state A2, applies Eq.(3-6) for i = 1, 3, 8

in Eq.(3.13-6). It is obtained that

P22 = 1 - [kiCi(n)C3(n) + k3Ci(n)C4(n)]At + [k8/C2(n)]At (3.13-6f)

The term [k8/C2(n)]At, where kg = k'8C2(0) is a constant supply rate of A2, must

be added to p22 in order to comply with the integrated form of Eq.(3.13-6b) for r2.

P23, the transition probability from A2 to A3, is calculated by Eq.(3-9) for i =

1, 3 in Eq.(3.13-6). It is obtained that

P23 = [2(l/3)kiCi(n)C3(n) + 2(l/3)k3Ci(n)C4(n)]At (3.13-6g)

Page 325: Applications of Markov Chains in Chemical Engineering

319

P33, the probability of remaining in state A3, applies Eq.(3-6) for i = 1, 2, 4,

6 in Eq.(3.13-6). However, the application of this equation needs some

clarification because of the following situation. According to Eq.(3.13-6) for i = 1,

A3 is consumed on the one hand, but it is also formed on the other. This fact must

be introduced in computing the probabilities and the only place is in P33 in a way

which complies with the result obtained by integration of r3 in Eq.(3.13-6b). Thus,

P33 = 1 - [kiCi(n)C2(n) + 2k2C3(n) + k4]At + 2(l/3)kiCi(n)C2(n)At

+ [k6/C3(n)]At (3.13-6h)

where the term 1 - [kiCi(n)C2(n) + 2k2C3(n) + k4]At designates the probability of

remaining in state A3; the term 2(l/3)kiCi(n)C2(n)At designates the transition

probability from A3 to 2A3 in Eq.(3.13-6) for i = 1. The latter was computed by

Eq.(3-10). The term [k6/C3(n)]At, where k6 = k'6C3(0) is a constant supply rate of

A3, must be added to P33 in order to comply with the integrated form of Eq.(3.13-

6b) for r3.

P34, the transition probability from A3 to A4, is calculated by Eq.(3-10a) for i

= 2inEq.(3.13-6). It is obtained that

P34 = 2k2C3(n)At (3.13-6i)

P35, the transition probabiUty from A3 to A5, is calculated by Eq.(3-10) for i

= 4inEq.(3.13-6). It is obtained that

P35 = k4At (3.13.6J)

P43, the transition probability from A4 to A3, is calculated by Eq.(3-10) for i

= 3 in Eq.(3.13-6). It is obtained that

P43 = 2(l/3)k3Ci(n)C2(n) (3.13-6k)

P44, the probability of remaining in state A4, applies Eq.(3-6) for i = 3, 5 in

Eq.(3.13-6). It is obtained that

P44 = 1 - [k3Ci(n)C2(n) + kslAt (3.13-61)

Finally, p46, the transition probability from A4 to A6, is calculated by Eq.(3-

10) for i = 5 in Eq.(3.13-6). It is obtained that

Page 326: Applications of Markov Chains in Chemical Engineering

320

P46 = ksAt (3.13-6m)

Transient response curves for Ci to Ce for the initial state vector C(0) = [Ci(0), C2(0), C3(0), C4(0), C5(0), C6(0)] = [3, 20, 0.01435, 0, 0, 0] , where the effect of ki = 0.02, 0.16, 0.2, 0.25 and 1 is demonstrated, are depicted in Fig.3.13-6 for the following data [69]: k2 = 1250, ks = 0.04688, k4 = 20, ks = 1.104, k6 = 0.001, k7 = 0.89, k.7 = 0.1175, kg = 0.5 and At = 0.01.

30

20 U

u

10 \-

k ,= l

h

1

p. 4

/ • .

1

/

6

. ' 5

- --. 2

i = l

3

: • 1

( a ) |

-J

H

50 100 150

Fig.3.13-6a. Cf versus t for ki = 1

Page 327: Applications of Markov Chains in Chemical Engineering

321

30

20 p,

u

10

kj = 0.25

H-\

r

1

5.'

6

. . . 2

i = l

3

1 (b)

H

-

50 100 150

Fig.3.13-6b- Ci versus t for ki = 0.25

30

20 Y-

u

10 U

50 100 150

Fig.3.13-6c. Ci versus t for ki = 0.22

Page 328: Applications of Markov Chains in Chemical Engineering

322

30

20

u

10 h

kj = 0.2

r~ '.

4^,

1

• '•'• - • '

. . - ^ . 1 - - .

5 / '

6

1

, • 1- • '

^ 1 ' > . ' •

(d)|

-j

A/ 50 100 150

Fig.3.13-6d. Ci versus t for ki = 0.2

Fig.3.13-6e. Ci versus t for ki = 0.16

Page 329: Applications of Markov Chains in Chemical Engineering

323

60

40

u

20 K

0 50 100 150

t

Fig.3.13-6f- Ci versus t for ki = 0.02

Figs.3.13-6, a to f, demonstrate the effect of ki on the transient behavior of

Ci to C6. Of particular interest is Figs.3.13-6d which is not actually chaotic but is

a complicated mixed-mode state[69].

3.14 NON-EXISTING REACTIONS WITH A BEAUTIFUL PROGRESSION ROUTE

It has been said that the essence of beauty emerges from the shape. Thus, to

conclude this chapter, two reactions are demonstrated whose routes form beautiful

shapes.

3.14-1 The Shield of David progression-route reaction The Shield of David is a Jewish national and religious symbol whose origin

goes back to the 12th century. It is comprised of two intergrated opposite triangles.

On May 24th 1949, the Parliament of the State of Israel declared that the Shield of

David will appear on the national flag and will be the identification symbol for

every Jew where ever he is. It is demonstrated below as the progression route of

the following reaction.

Page 330: Applications of Markov Chains in Chemical Engineering

324

(3.14-1)

For the above reaction scheme, the following kinetic equations are applicable.

It is assumed that a saturated vapor of gas Ai is in equilibrium with its condensed

phase containing species Ai to A12. Other assumptions are as in case 3.13-5

above, where H is the rate of supply of A1 in moles/sec from the vapor phase into

the condensed phase.

rj = H - (ki2 + kj 12)^1 2 ~ ~ vk23 + k24)C2 + ^n^i + '^i,12^12

r7 = - k7gC7 + k^-jC^

Tg = - k9 10C9 + kg9Cg

r4 = — (k45 + k45)C4 + 1 3403 + 1 2402

rg = - (k 39 + kg |o)Cg + k7gC7 + k^gC^

r o = - (k 10,11 + 1 10,12) 10 + ^,10^8 + ^9,10^9

11 - "" 1 11,12 11 "•• 1 10,11 10

ri2 - - k 12,2^12 + ki 12C1 + kji 12C11 + kio,i2Cio (3 .14- la)

The above equations yield the following transition probability matrix:

Page 331: Applications of Markov Chains in Chemical Engineering

325

P =

1

2

3

4

5

6

7

8

9

10

11

12

1 P i i

0

0

0

0

0

0

0

0

0

0

0

2 P l 2

P22

0

0

0

0

0

0

0

0

0

Pl2,2

3 0

P23

P33

0

0

0

0

0

0

0

0

0

4 0

P24

P34

P44

0

0

0

0

0

0

0

0

5 0

0

0

P45

P55

0

0

0

0

0

0

0

6 0

0

0

P46

P56

P66

0

0

0

0

0

0

7 0

0

0

0

0

P67

P77

0

0

0

0

0

8 0

0

0

0

0

P68

P78

P88

0

0

0

0

9 0

0

0

0

0

0

0

P89

P99

0

0

0

10 0

0

0

0

0

0

0

P8,10

P9,10

PlO.lO

0

0

11 0

0

0

0

0

0

0

0

0

PlO.l

P l l . l

0

12 Pi,12

0

0

0

0

0

0

0

0

lPlO,12

l P l l , 1 2

Pl2,12

(3.14-lb)

where

Pii = 1 - [ki2 + ki,i2]At + [H/Ci(n)]]At

P l2= ki2At Pi,i2 = ki,i2^t

P22 = 1 - [k23 + k24]At P23 = k23At P24 = k24At

P33= l -k34At P34 = k34At

P44 = 1 - [k45 + k46]At P45 = k45At P46 = k46At

P55= l - k s e A t p56 = k56At

P66 = 1 - [ 67 + k68]^t P67 = k67At Pgg = kggAt

P77= l - k 7 8 A t P78 = k78At

P88 ~ ^ ~ '•' 89 ••• .loJ^t P89 - ''89At Pgjo ~ 1 8,10 *

P99 = 1 - k9 joAt P9 10 = k9 joAt

Pio.io = 1 ~ tl^io.ii + ^ 10,12]' ^ pjo,ii = k]o,iiAt Pio,i2 = kjo,i2At

Pi i .u = 1 ~ kj] 12 At Pii . i2= J'li.n^t

Pl2,12 = 1 - ki2,2 ^t Pi2,2 = ki2,2At (3.14-lc)

Page 332: Applications of Markov Chains in Chemical Engineering

326

The term [H/Ci(n)]At, where H is a constant supply rate of Ai, must be added in

pil in order to comply with the integrated form of ri in Eq.(3.14-la).

The transient response of Ci to C12 is depicted in Fig.3.14-1 where the

effect of H = 0, 2 and 3 is demonstrated. Other parameters are ky = 1, Ci(0) = 1,

Ci(0) = 0 (i = 2, 3,... , 12) and At = 0.04. 0.3

0.2 \-

0.1

1

; \

^ / F / ;

/ / ' 1/ .:

1 H = 0

r^'^\ \ ' • • • - . ^ x r • • : . - . . .

\ ^y-. :-<^^::..^ \/ (^y '']y''"i' / \ y ly

il-^r:.::^^^^ 1

1:C 1

3:C =C 3 4

5:C =C 7 8

7:C 11

V • - - z : j-^"-T::-r

2:C 2

4:C =C 5 6

6:C =C 9 10

8:C 12 - j

^S^5?"^^;S*.«S5E^- "w*w

10

Fig.3.14-la. Ci versus t for H = 0

Fig.3.14-lb. Ci versus t for H = 2

Page 333: Applications of Markov Chains in Chemical Engineering

4 h

3 \-

U

1 F

1 1:C 2:C

1 2 3:C =C 4:C =C

— 3 4 5 6 5:C =C 6:C =C

7 8 9 10 7: C 8: C

11 12

r --''' ' '•'"''' '^ '

K '•-•"/^•- ,.•<--• y . .--^.--''..•••

v. - ;;-<.-- •• I ^ ..::...::-:•:..-- - i

H = 3

..-' 'J.

- ^ " .-" . .-"'

,.'''" '

1

8

; ' . - ^

..-'

1

^'"^'

^..•-•jj

•"'' - " 1 - ' , * '' -- "i

-

10

Fig.3.14.1c. Ci versus t for H = 3

327

15

3.14-2 The Benzene molecule progression-route reaction The route of this reaction is shown below

(3.14-2)

The kinetic reactions are:

r i = - ( k i 2 + ki6)Ci + k6iC6 l2 = 1 23 2 + kj2Cj + k32C3

rj = - (k34 + k32)C3 + k23C2 r4 = - k45C4 + k34C3 + k54C5

rj = - (k56 + k54)C5 + k45C4 r = - k6,C6 + kigCj + kseCs (3.14-2a)

Page 334: Applications of Markov Chains in Chemical Engineering

328

yielding the following transition probability matrix:

P =

1 l -[ki2+ki^

0

0

0

0

k^iAt

]At

2 ki2At

l-k23At

k32At

0

0

0

3

0

k23At

l-[k32+k34]At

0

0

0

4

0

0

k34At

l-k45At

k54At

0

5

0

0

0

k45At

l-[1^54+k565^t

0

6 ki^At

0

0

0

k56At

l-k^iAt

1

2

3

4

5

6

(3.14-2b) The transient response of Ci to Ce is depicted in Fig.3.14-2 where the effect

of ki2 is demonstrated for At = 0.04, ky = 1 (ij ?i 12), C](0) = C5(0) = 1, C2(0) = C4(0) = 2, C3(0) = 3 and CeCO) = 0.

u

1 H

' . . = '

y^~"*""--s ^''

. - - ^ ---

,'''

'* - - - ' ' ' i = l

l 1

- • — - -- - - - — — '

10

Fig.3.14-2a. Cf versus t for ki2 = 1

Page 335: Applications of Markov Chains in Chemical Engineering

329

Fig.3.14-2b. Ci versus t for ki2 = 0 As observed in the above Figs.3.14-1 and 2, the transient

curves look very similar to the numerous ones generated before. The question that arises then is why should not the reactions corresponding to the Shield of David and Benzene progress shape routes exist ?

Page 336: Applications of Markov Chains in Chemical Engineering

330

3>14-3 The Lorenz system, partially demonstrating chemical reactions, for creating esthetic patterns

The system of equations that Lorenz proposed in 1963 [85, p.697] are: 10

ri = IOC2 - lOCi

12 = 28Ci - C2 - C1C3

r3 = CiC2-(8/3)C3

for which Ai A2 may be written.

10

for which no reaction may be realized. 8/3

for which A3 Ai +A2 may be written.

1

As seen, two equations, may demonstrate reversible reactions. However, the

Lorenz system is in fact a model of thermal convection, which includes not only a

description of the motion of some viscous fluid or atmosphere, but also the

information about distribution of heat, the driving force of thermal convection. The

above set can be described by the following matrix, which, however, does not have

any probabiUstic significance:

1

P =

thus,

Ci(n+1) = Ci(n)[l - lOAt] + C2(n)[10At]

C2(n+1) = Ci(n)[28 - C3 (n)]At + C2(n)[l - At]

C3(n+1) = Ci(n)C2(n)At + C3 (n)[l - (8/3)At]

The transient response of Ci, C2 and C3 is depicted in Fig.3.14-3 for At =

0.01 and C(0) = [0.01, 0.01, 0.01]. The behavior in the figures clearly

demonstrates chaos characterized by:

a) The physical situation is described by non-linear differential equations.

1

2

3

1 - lOAt

lOAt

0

[28-C3(n)]At 0.5C2(n)]At

1-At 0.5Ci(n)]At

0 1 - (8/3)At

Page 337: Applications of Markov Chains in Chemical Engineering

331

b) The transient response is characterized by high sensitivity to extremely small

changes in the initial conditions as demonstrated in Fig.3.14-3 case a.

c) Numbers generated by the solution are random as shown in Fig.3.14-3 case b.

d) Order, demonstrated by esthetic patterns in Fig.3.14-4 below, can be generated

from chaos by appropriate representation of the chaotic data in Fig.3.14-3.

30 Initial conditions: 1: CCO) = € (0) = C (0) = 0.01

2: Cj(0) = CCO) = C (0) = 0.01001

(a)

Fig.3.14-3. Ci versus t for the Lorenz system demonstrating extreme

sensitivity to initial conditions in case a and chaos in case b

Page 338: Applications of Markov Chains in Chemical Engineering

332

60

50 h

40 h

U

30

20 h

10

0 -30

I ^^^^Hi

^ 1 ^ ^ 1 H |

KpDii'^Dlp^iya D D D Bq^

ir J o 1 a riT ih l l i 1 iTinih « if^ jtinn|nii;rji=-' --V'-

i l infl l 'Jj^^iJj*" i LIT if:V^j\^lj'!l(IJ-^!s^jPQiB!S

1 i ^^mmmatt^^^^^^^^.m^^m

-20 -10 0 10 20 30

Fig.3.14-4a,b. C3-C1 and C3-C2: creation of Order from Chaos

Page 339: Applications of Markov Chains in Chemical Engineering

333

30 h

20 h

10 h

V

-10

•20

^0

Fig.3.14-4c. The C2-C1 Lorenz attractor demonstrating the creation

of Order from Chaos

As observed in Figs.3.14-4, all patterns generated by the C3-C1, C3-C2 and

C2~Ci representations, remind, in one way or another, a butterfly. The latter

stands for a basic phenomenon in the chaos model known as the butterfly effect,

after the title of a paper by Edward N.Lorenz 'Can the flap of a butterfly's wing stir

up a tornado in Texas?' An additional point may be summarized as follows, i.e.,

How come that relatively simple mathematical models create very complicated

dynamic behaviors, on the one hand, and how Order, followed by esthetics

patterns, may be created by the specific representation of the transient behavior, on

the other ?

Page 340: Applications of Markov Chains in Chemical Engineering

334

Chapter 4

APPLICATION OF MARKOV CHAINS IN CHEMICAL REACTORS

The major aim of the present chapter is to demonstrate how Markov chains

can be appUed to determine the behavior of a compUcated system with respect to the

residence time of fluid elements flowing through it. In other words, to obtain the

response of the system to some tracer input, usually in the form of a pulse.

It is well-known that fluid elements entering simultaneously a continuous

reactor, do not, in general, leave together, owing to the complex flow pattern inside

the reactor. Because of this reason, there is a spread of the residence time of the

flowing elements, i.e. the time each element resides in the reactor; the latter can be

represented by the so-called age distribution function. Thus, if a chemical reaction

is carried out in a reactor, the resulting products depend on the length of time each

element of the reactants spends within the reactor, which affects the overall

conversion.

Several models have been suggested to simulate the behavior inside a reactor

[53, 71, 72]. Accordingly, homogeneous flow models, which are the subject of

this chapter, may be classified into: (1) velocity profile model, for a reactor whose

velocity profile is rather simple and describable by some mathematical expression,

(2) dispersion model, which draws analogy between mixing and diffusion

processes, and (3) compartmental model, which consists of a series of perfectly-

mixed reactors, plug-flow reactors, dead water elements as well as recycle streams,

by pass and cross flow etc., in order to describe a non-ideal flow reactor.

In the following, approach (3) above was adopted. Stimulating the resulting

flow configuration by some input, yields the residence time distribution of fluid

elements in the flow system. In the present chapter, the flow system was treated

Page 341: Applications of Markov Chains in Chemical Engineering

335

by Markov chains yielding the transition probability matrix. Each expression in the

matrix is either the probability to remain in a state (reactor) or to leave it to the next

state (reactor). Complicating the flow behavior is usually manifested by additional

terms in the matrix, which, however, does not create proportional difficulties in the

solution. Applying Eqs.(3-20), based on Eqs.(2-23) and (2-24), may yield

important characteristics of the flow behavior, viz., response to a step change or to

a pulse input. Both responses provide necessary information about the residence

time distribution of fluid elements in the flow system.

Many flow systems of interest in Chemical Engineering are presented in the

following, viz., a diagram of the flow system, the transition probability matrix and

the transient response to some input signal. Attention has been paid also to

systems employing impinging streams [73] which is an effective technique for

intensifying technological processes.

4.1 MODELING THE PROBABILITIES IN FLOW SYSTEMS

Definitions. The basic elements of Markov chains in flow systems are: the

system, the state space, the initial state vector and the one-step transition probability

matrix. The system is a fluid element. The state of the system is the concentration

of the fluid element in the reactor, assumed perfectly mixed. The state space is the

set of all states that a fluid element can occupy, where a fluid element is occupying

a state if it is in the state, i.e. at some concentration. For the flow system depicted

in Figs.(4-1) and (4-la), the state space SS, which is the set of all states a system

can occupy, is designated by

55 = [C\, C2,... ,Cz, C^]

Finally, the movement of a fluid element from state Cj to state Ck is the transition

between the states.

The initial state vector given by Eq.(2-22), corresponding to Figs.(4-1) and

(4-la), reads:

S(0) = [si(0), S2(0), S3(0), ..., Si(0), ..., Sz(0), s^(0)]

Page 342: Applications of Markov Chains in Chemical Engineering

336

where Si(0) is the probabihty of the system to occupy state i at time zero. S(0) is the initial occupation probabihty of the states [Ci, C2,..., Cz, C ] by the system.

Z+1 designates the number of states, i.e. Z perfectly mixed reactors in the flow

system as well as the tracer collector designated by ^. As shown later, the

probabilities Si(0) may be replaced by the initial concentration of the fluid elements

in each state, i.e. Ci(0) and S(0) will contain all initial concentrations of the fluid

elements. The one-step transition probability matrix is given by Eqs.(2-16) and (2-

20) whereas pjk represent the probability that a fluid element at Cj will change into

Ck in one step, pjj represent the probability that a fluid element will remain

unchanged in concentration within one step.

In the following, general expressions are derived for the transition

probabilities corresponding to two general flow configurations. The latter can be

reduced to numerous systems encountered in Chemical Engineering elaborated

below.

4.1-1. Probabilities in an interacting conflguration

Basic configurations. The models demonstrated in Figs.4-1 and 4-la

comprise of perfectly-mixed reactors, generally, of not the same volume. The

central reactor in Fig.4-1 is designated by j and also termed as reactor junction (j).

If the volume of this reactor is zero, this location designated by j in Fig.4-1 a, is

termed as point junction (j). The above situations are of practical importance and

generate a slightly different transition probability matrix. The mean residence time

of the fluid in the reactor is V/Q whereas in the case of a point it is equal to zero.

The peripheral reactors are designated by a, b,..., Z. The final reactor is the tracer

collector designated by ^. It should be noted that the letters are assigned numerical

values when a specific case is considered where ^ is assigned the highest number,

i.e. ^ = Z + 1. When the central reactor j is considered, usually, j = 1 and a = 2, b

= 3, etc. When the central reactor is not considered, usually, a = 1, b = 2, etc.

The total number of reactors is the number of states. If reactor j is considered, the

total number of states equals ^ -h 1 = Z + 2.

As observed, there are various interacting flows in Figs.4-1 and 4-la as

follows: a) Flows between the central reactor or point j , and the peripheral ones,

and vice versa, Qji and Qy, respectively, as well as between the central reactor, or

point, and collector ^, i.e. qj^. b) Flows between every peripheral reactor and each

Page 343: Applications of Markov Chains in Chemical Engineering

337

of the others, Qik, where i, k = a, b, ..., Z, k ?t i. It should be noted that among

these stresims are also the so-called recycle streams, c) Flows between every

peripheral reactor and the collector ^, q^ , where k = a, b,..., Z. d) External flows

into the reactors, Qr, r = j , a, b, ..., Z. For example, in Fig.4-1 we consider

reactor p. Qp indicates an external flow into reactor P. Qkp refers to the flows

from all reactors into reactor p, where k = j , a, b,..., Z, k t p. Qpj demonstrates

all flows from reactor p to all other reactors k, i.e. k = j , a, b, ..., Z, k 9fc p.

Finally, qp^ indicates the flow from reactor p to the collector ^.

Stimulating the system by a tracer input introduced into reactor j or reactors i,

causes a change of the concentration in the entire flow system due to interaction

between the reactors. The following situations my be possible with regard to the

tracer. If its concentration C' = 0 at the exit of reactor ^, the tracer is completely

accumulated in reactor ^; thus, the reactor is considered as "total collector" or "dead

state" or "absorbing state" for the tracer. In other words p^^ = 1, and once the

tracer enters this state, it stays there for ever. If, however, the concentration C^ of

the tracer in the reactor is equal to its concentration at the exit, i.e. C^ = C' , there

is no accumulation of the tracer in reactor ^, If 0 < C' < C , the tracer is partially

accumulated in reactor ^, which is considered as "partial collector". Generally, the

fluid is always at steady state flow. In the case of a closed circulating system, i.e.

Qj. = 0, r = j , a, b,. . . , Z, the tracer is eventually distributed uniformly between all

reactors.

Finally, it should be noted that the schemes depicted in Figs.4-1 and 4-la

cover numerous flow configurations encountered in Chemical Engineering where a

specific configuration is determined by appropriate selection of the interacting

flows and the number of reactors. Numerous examples of high significance will be

treated in the following.

Page 344: Applications of Markov Chains in Chemical Engineering

338

a) Overall scheme QkaQak^p

b) Tracer collector

k k

Fig.4-1. A scheme for an interacting flow system with a reactor junction (j)

Page 345: Applications of Markov Chains in Chemical Engineering

339

a) Overall scheme

b) Tracer collector

Fig.4-la. An interacting flow system with a point junction (j)

Page 346: Applications of Markov Chains in Chemical Engineering

340

Derivation of the probabilities from mass balances

Reactor j . A mass balance on the tracer in the reactor junction (j) in Fig.4-

1 for k = a, b,... , Z where k^] and ^ reads:

dC V j - d f ^ ^ Q k A - Uj Cj + XQjkCj (4-1)

It should be noted that for a specific configuration, j , k and ^ are assigned

numerical values where ^, the collector, should be assigned the highest value. Cj

and Ck are, respectively, the concentrations of the tracer in reactor j and reactors k;

Qkj and Qjk are the interacting flows between reactor k and j and vice versa,

whereas flow qj^ is from the central reactor to the collector reactor ^. Vj is the

volume of the fluid in reactor j , assumed to remain unchanged.

In the case of a point junction (j) in Fig.4-la, Eq.(4-1) for Vj = 0 yields that

the concentration at this point is:

^ Q k j ^ k ^ ^ k j ^ k

Cj = -^ = -^ (4-la)

k k

The following quantities are defined for the flows in Fig.4-1 where k = a, b,

..., Z, k:^i,% gives:

««=^ %=t fe=f («) Qi

Hj = # (4-3)

where |LIJ (1/sec) is a measure for the transition rate of the system (fluid element)

between consecutive states (reactors). Qj is the flow rate into reactor j where Qkj

and Qjk are the flows from reactor k to j and j to k, respectively. In the case of a

closed recirculation configuration, i.e., Qj = Qk = 0 (k = a, b,. . . , Z), one of the

Page 347: Applications of Markov Chains in Chemical Engineering

341

internal streams should be selected as reference flow instead of Qj in Eqs.(4-2) and

(4-3) in order to perform the above non-dimensionalization.

Integration of Eq.(4-1) between the times t and t+At, or step n to n+1, while

considering the above definitions, yields for k = a, b,..., Z where k T j , 4. that

Cj(n+l) = Cj(n) 1 - h + X"jk h^^ + X^k(n)[akjtijAt] (4-4)

In terms of probabilities, the above equation reads:

Cj(n+1) = Cj(n)pjj + 2,Ck(n)Pkj (4-4a)

Pjj is the probability to remain in reactor j and pkj are the transition probabilities

from reactors k to j . The definition of the probabilities is obtained from Eq.(4-4).

Reactor i. A mass balance on the tracer in reactor i in Figs.4-1 and 4-la

for i, k = a, b,. . . , Z where k ?i i, j , ^ gives:

dCj ^ Vi-3^ = QjiCj + 2 ^ Q k i C , - qi + Qij + Z Q i k R

k / (4-5)

In the case of the point junction (j) in Fig.4-la, the concentration Cj is given

by Eq.(4-la). Defining the following quantities with respect to the reactor junction

(;)inFig.4-l:

aki = Qki

Qj

Qik Qji Qij ^ i ^ ttik--^ " j i - - ^ " ' j " " ^ ^ ' ^ " " ^

Qj

(4-6)

(4-7)

and expressing Eq.(4-5) in a finite difference form, yields for i, k = a, b, ..., Z

where k9 t i , j , ^ that:

Page 348: Applications of Markov Chains in Chemical Engineering

342

Ci(n+1) = Cj(n)[aji|LiiAt] + QCii)

+ XCk(n)[aki^iAt] (4-8) k

An alternative form of the above equation in terms of transition probabilities,

reads:

Ci(n+1) = Cj(n)pji + Ci(n)pii +X^k(n)Pki (4-8a) k

where Cj(n) is the concentration in reactor j . pji is the transition probability from

state j to i, pa is the probability to remain in reactor i and pki are the transition

probabilities from reactors k to i. The definition of the probabilities is obtained

from Eq.(4-8). In the case of the point junction (j) in Fig.4-la, the concentration

Cj is given by Eq.(4-la). In the case of a closed recirculation configuration, i.e.,

Qj = Qk = 0 (k = a, b,.. . , Z), or if Qj = 0 and Qk ^ 0, one of the internal streams

or one of the Qk's should be selected as reference flow instead of Qj in order to

perform the above non-dimensionalization.

Reactor ^. A mass balance on the tracer in reactor ^ in Fig.4-1 for k = a,

b, . . . , Z where k^],^ reads:

^ • ^ = 2<q>c Ck + qj^q (4-9)

yielding:

C^(n+1) = Cj(n)[pj^^^At] +X^k(n)[p^^^At] + C^(n) (4-10) k

In terms of transition probabilities the above equation reads:

C^(n+1) = Cj(n)pj^ +X^k(n)Pk^ + C^(n)p^^ (4-lOa)

Page 349: Applications of Markov Chains in Chemical Engineering

343

Pjt is the transition probability from reactor j to ^, p^t are the transition

probabilities from reactors k to ^ whereas p^t = 1 is the probability to remain in

reactor ^ since this reactor is considered as a collector for the tracer. The definition

of the probabilities is obtained from Eq.(4-10). For the case depicted in Fig.4-la,

i.e. iht point junction (j) scheme, Cj(n) is given by Eq.(4-la), where:

H = T^ h = ^ >•=•''•' (4-11)

Finally, the following relationships resulting from mass balances on the

flows, must be satisfied simultaneously. They serve to determine the quantities

otk, cxij, ocji, akj, ttjk, aik, aki as well as pj^ and pk^.

An overall mass balance on the flow system in Figs.4-1 and 4-la, i.e. on

reactors j and k , k = a, b,.. . , Z where k^} and , gives:

Qj + X^k = Qj + X^k^ y''^^^^ 1 + S^k = Pj + XPk^ (4-12a)

ttk is given by Eq.(4-12d) and pj^, pk^ by Eq.(4-11).

A mass balance on the flows of reactor j or point j in Figs.4-1 and 4-la, respectively, for k = a, b,.. . , Z where k ;t j and , yields:

Qj + XQkj = qj^+XQjk yi^i^s i + X^kj = Pj^+X"jk (4-i2b) k k k k

where ttkj and ajk is given by Eq.(4-6). If Qj = 0, one of the in flows Qi where i

= a, b, ..., Z or one of the internal flows, should be taken as a reference flow in

Eqs.(4-2) and (4-6) instead of Qj. In addition, Eq.(4-12a) should be ignored and

in Eq.(4-12b), the figure 1 should be omitted.

A mass balance on reactors i in Figs.4-1 and 4-la for i , k = a, b, ..., Z

where k ?t i, j and ^, reads:

Page 350: Applications of Markov Chains in Chemical Engineering

344

Qi + Qji + X ^ w = qi + Qij + X ^ i k yields k k

tti + ttji + ^ a k i = pi + ay + Y^o-m (4-12C)

where aji, ttki, ttik and Pi are defined in Eq.(4-6). In the case of a closed

system, i.e., Qj = Qk = 0 (k = a, b,.... Z), one of the internal streams should be

selected as reference flow in order to perform the non-dimensionalization of the

quantities designated by a. The coefficient ai is defined by:

Qi (4-12d)

where Qj is the flow rate into reactor j in Figs.4-1 or to junction j in 4-la.

For the reactor junction j scheme in Fig.4-1, Eqs.(4-4), (4-4a), (4-8), (4-8a)

(4-10) and (4-lOa) may be expressed by the following matrix:

P =

j

a

i-l

i i+1

Z

^

C j -j

Pjj Paj

Pi-l,j

Pij

Pi+l.j

Pzj

1 0

Ca = a

Pja

Paa

Pi-l,a

Pia

Pi+l,a

Pza

0

i-l

Pj,i-1

Pa,i-1

i

Pji Pai

Q+i = i+1

Pj,i+1

Pa,i+1

Pi-l,i-l Pi-l,i Pi-l,i+l

Pi,i-1 Pii Pi,i+1

Pi+l,i-l Pi+l,i Pi+l,i+l

Pz,i-1 Pzi Pz,i+1 0 0 0 0

Pjz Paz

m Pa4

Pi-l,z Pi-14

Piz Pi

Pi+l,z Pi+1,

Pzz Pz

0 1 (4-13)

The calculation of the new concentration vector C(n+1) is performed by

Eq.(3-20), i.e.:

C(n+1) = C(n)P

where

Page 351: Applications of Markov Chains in Chemical Engineering

C(n) = [Cj(n), Ca(n), Cb(n),..., Cz(n), q(n)]

345

(4-13a)

For the point junction j scheme depicted in Fig.4-la, Eq.(4-la) for Cj(n), and

Eqs.(4-8), (4-8a), (4-10) and (4-lOa) for Ci(n) and C^(n), may be expressed by

the following matrix:

P =

j a

i-l

i

i+l

Z

^

j

0

Paj

Pi-l,J

P'ij •*• P i + l j

P'zj

0

a

Pja

Paa

Pi-l,a

Pia

Pi+l,a

Pza

0

i-l

•• Pj,i-l

Pa,i-1

•• Pi-l,i-l

Pi,i-1

•. Pi+l,i-l

Pz,i-1

0 0

i

Pji

Pai

Pi-l,i

Pii

Pi+l,i

Pzi

0

Q+1 = i+l

Pj,i+1

Pa,i+1

Pi-l,i+l •

Pi,i+1

Pi+l,i+l •

Pz,i+1 0

z Pjz

Paz

Pi-l,z

Piz

Pi+l,z

Pzz

0

Pj^

Pa

Pi-1,^

Pi^

Pi+1,^

Pz^

1

(4-13b)

where

Pkj OCkj

pj^+X«jk

k = a, b, ..., Z where k T j , ^ (4.13c)

It should be noted that the p jks are not exactly probabilities but merely

quantities which enable one to present the junction concentration Cj(n), due to

mixing of various streams at this point, given by Eq.(4-la) in the above matrix and

to compute it from C(n+1) = C(n)P. The above matrix differs from Eq.(4-13a) by

the following: pjj = 0 as well as some of the expressions for computing the

probabilities are different, as detailed below.

Summary of probabilities

These quantities resuh in from the above mass balances which yield

equations (4-4), (4-4a), (4-8), (4-8a) (4-10) and (4-lOa).

Page 352: Applications of Markov Chains in Chemical Engineering

346

The probability pjj of remaining in state j (reactor j) in a single

step (single time interval At) for k = a, b, ..., Z where k 9 j , ^,

reads:

P j j = l - pj +X%Vj ^ ^^'"^^"^

Pjj stems from Eq.(4-4) and is applicable for the reactor junction j scheme in Fig.4-

1. Pjj = 0 for the case depicted in Fig.4-la.

The transition probability pji (i = a, b, •••, Z) in a single step

(single time interval At), from state j (reactor j) or junction j , to state

i (reactor i) where i 9 j , ^, reads:

Pji = ajiiLiiAt (4-15)

pji stems from Eq.(4-8) and is applicable for the scheme depicted in Figs.4-1 and

4-la. For the latter case Cj is obtained from Eq.4-la.

The transition probability pj^ in a single step (single time

interval At), from state j (reactor j) or junction j , to state ^ (reactor ^)

where j ^ ^, reads:

Pj^ = Pj^^^At (4-15a)

which stems from Eq.(4-10) corresponding to Figs.4-1 and 4-la.

The probability pii of remaining in state i (reactor i) in a single

step (single time interval At) for k, i = a, ..., Z where k ? i, j and ^,

reads:

r Pii = 1 - Pi + oCij + X^ikh^^ (4-16)

V

pii stems from Eq.(4-8) and corresponds to Figs.4-1 and 4-la.

The transition probabilities py and pi in a single step (single

time interval At) from state i (reactor i) to state j or ^ (reactor j or ^)

for i = a, b, ..., Z where i 9t j and I,, reads:

Page 353: Applications of Markov Chains in Chemical Engineering

347

Pij = ttij^jAt (4-17a)

which stems from Eq.(4-4) and corresponds to Fig.4-1. From Eq.(4-10) follows

that

Pi^=Pi^^^At (4-17b)

corresponding to Figs.4-1 and 4-la.

The transition probabilities between the peripheral reactors, corresponding to

Figs.4-1 and 4-la, are:

The transition probability pki in a single step (single time

interval At) from state k (reactor k) to state i (reactor i) for k = a, b,

••*, Z where k ? i, j and ^, reads:

Pki = akiHiAt (4-18)

which stems from Eq.(4-8). It follows, for example, from Eq.(4-18) that:

Pi,i+i = ai,i+iM'i+iAt (4-18a)

Pi+l,i = ai+l,iWAt (4-18b)

The transition probability pk^ in a single step (single time interval At) from state k to state ^ (reactor ^) for k = a, ..., Z where k ? i, j and , reads:

Pk^=Pk^|Li^At (4-19)

stemming from Eq.(4-10).

Inspection the probabilities defined in Eqs.(4-14) to (4-19) and the transition

probability matrix given by Eqs.(4-13) and (4-13a), leads to the following

conclusions:

a) In general, for every row:

2jPik '^ 1 1 = a,..., Z (4-20a)

Page 354: Applications of Markov Chains in Chemical Engineering

348

b) However, if all reactors are of an identical volume, i.e. |Lii = |ij = |LI according to

Eqs.(4-3) and (4-7), then for each row:

XPjk = 1 (4-20b)

and the matrix given by Eq.(4-13) is time-homogeneous or stationary.

Additional expressions of transition probabilities, without a mathematical

proof, have been suggested [75], viz.:

Pjj ~ ^ Pjk "• ^ ^ (4-21)

These expressions reduce to the above ones if the first term in the Taylor series

expansion is taken, which is justified for short At. At is the time a molecule in

vessel j can either remain where it is or move on to vessel k.

4.1-2. 'Dead state' (absorbing) element. Such an element depicted in Fig.4-2

"cbi rebi

Fig.4-2.'Dead state' element

is characterized by the following transition probabilities:

Pii = 1 Pij = 0 (4-22)

4.1-3. Plug flow element. Such an element depicted in Fig.4-3 is

characterized by the following transition probabilities for a pulse input introduced at

t = 0: pjj = l pji = 0

0 < t < t p : pjj = 0 Pii = 0

t = tp: Pjj = 0 Pii = 1

t>tp: pjj = 0 pii = 0 (4-23)

Page 355: Applications of Markov Chains in Chemical Engineering

349

tp = V/Q is residence time in the reactor which is identical for all fluid elements.

Fig.4-3.Plug-flow element

4.2 APPLICATION OF THE MODELING AND GENERAL GUIDELINES

The above modeling is applied to numerous flow configurations which have

appeared in various Chemical Engineering textbooks as well as additional ones of

particular interest, i.e. impinging-stream reactors [73]. In general, any flow

configurations under consideration will consist of a series of perfectly-mixed

reactors, plug-flow reactors, 'dead water' elements as well as recycle streams, by

pass and cross flow etc., or part of the above.

Our major concern is to study the transient behavior of the configuration by

introducing an ideal pulse input at the inlet. The resulting response curve provides

the RTD of fluid elements leaving the system. As shown below, a certain element

in Eq.(3-20), which is the Markov chain key equation, yields the response to the

ideal pulse input, while another element gives the response to a step change input.

According to [38] the RTD is designated by E where:

Edt (4-24)

represents the fraction of the exit stream of age (the time spent by the element in the

reactor) between t and t+dt. Thus:

fEdt=l Jo

whereas

J" Jo

Edt

Page 356: Applications of Markov Chains in Chemical Engineering

350

is the fraction of the exit stream younger than age ti. If C(t) designates the

response of a state (reactor) in its exit to a deha function or impulse introduced into

the reactor, then:

E(t)= ^^^^ (4-25)

f C(t)dt Jo

where the integral represents the area under the response curve. An important

quantity frequently applied in the following, which stems from the response curve,

is the mean residence time of fluid elements in the reactor, tm. This quantity is

defined by:

oo

fc(t)tdt

f C(t)dt Jo

t„,=-^ (4-26)

In treating a certain configuration, the first step is to define the states that the

system can occupy. By a state is meant, the concentration Ci in a perfectly mixed

reactor i or at the inlet or the exit of a plug-flow reactor, that the system (fluid

element) can occupy. The states will be designated by Ci, C2,... whereas the state

space SS, will read:

SS = [Ci, C2,... , Cz, q ] = [1, 2, ..., Z, ^]

The next step is to 'break' the complicated flow configuration into basic

elements which were described above in sections 4.1-1 to 4.1-3; thus, the

probabilities of remaining in a state, pjj, or of moving to a new state, pjk, can be

deduced. This yields the transition probabiUty matrix P.

A further step is to specify the initial concentration of each state, i.e. the

initial state vector S(0). This is defined by:

S(0) = C(0) = [Ci(0), C2(0),..., Cz(0), q (0) ]

Page 357: Applications of Markov Chains in Chemical Engineering

351

In the present case, the initial concentration is that of the pulse input introduced into

the reactor. The concentration at some time t = nAt or step n, in the various

reactors, i.e., the states of the system, is defined by the following state vector:

S(n) = C(n) = [Ci(n), C2(n),..., Cz(n), q(n)]

whereas the relation between the above quantities, which are connected by the

transition probability matrix P, is given by:

Ck(n+1) = ]^Cj(n)pjk j = 1, 2,..., Z, ^ j

C(n+1) = C(n)P (3-20)

The above equations are based on Eqs.(2-23) and (2-24). The justification

for applying the above equations to flow systems under consideration lies in the

complete agreement obtained by the Euler integration of the linear equations, Eq.(4-

1), (4-5) and (4-9), and expressing their difference presentation, i.e., Eq.(4-4), (4-

4a) (4-8), (4-8a) (4-10) and (4-lOa) in the form of Eq.(3-20) above. Thus, flow

system can easily be treated by Markov chains where the matrix P becomes

'automatic' to construct, once gaining enough experience. In addition, flow

systems are presented in unified description via state vector and a one-step

transition probability matrix.

Detailed demonstrations of the above guidehnes will be made in typical cases

out of numerous ones presented below. The presentation of the examples is made

according to the following categories:

1) Perfectly-mixed reactor systems (chapter 4.3).

2) Plug flow-perfectly mixed reactor systems (chapter 4.4).

3) Impinging-stream systems (chapter 4.5).

In each case are presented a diagram of the flow configuration, the transition

probability matrix and the transient response to a pulse input. It should be

emphasized that the nomenclature in the text is specific to each case. The

magnitude of C(0) are the quantities on the Ci axis of the response curve

corresponding to t = 0. An important parameter in the computations is the

magnitude of the interval At. This parameter has been chosen recalling that pjj and

Page 358: Applications of Markov Chains in Chemical Engineering

352

Pjk should satisfy 0 < Pjj and Pjj < 1 on the one hand, and that Cj versus nAt

should remain unchanged under a certain magnitude of At, on the other. In

addition, a comparison with the exact solution has been conducted in many cases,

which made it possible to evaluate the accuracy of the solution obtained by Markov

chains. The quantities reported in the comparison are the maximum deviation,

Dniax» ai d the mean deviation, Dmean- On the basis of these comparisons, a

representative value of At = 0.01 is recommended, which is the parameter of

Markov chains solution.

Page 359: Applications of Markov Chains in Chemical Engineering

353

4.3 PERFECTLY MIXED REACTOR SYSTEMS Perfectly mixed reactors are the key element for conducting chemical

processes and in simulation of complex flow systems. Other synonyms are mixed

reactor, back mix reactor, an ideal stirred reactor and the CFSTR (constant flow

stirred tank reactor). As the name implies, it is a reactor in which the contents are

well stirred and uniform throughout. Thus, the exit stream from this reactor has the

same composition as the fluid within the reactor.

In the following a variety of configurations will be treated, which find

importance in practice and in simulation, as well as elaborate the application of the

model in chapter 4.1. In some cases the derivations are also detailed.

4,3-1 The flow configuration comprising of two perfectly-mixed reactors of

volumes Vi and V2 is demonstrated in Fig.4.3-1. A tracer in a form of a pulse

input is introduced into reactor 1 and is transferred by the flow Qi into reactor 2

where it is assumed to accumulate. Thus, this reactor is a "dead" or "absorbing"

state for the tracer, i.e. C' = 0 in Fig.4-1.

Fig.4.3-1. Two perfectly-mixed reactors

There are two states for which the state space is:

SS =[Ci ,C2] = [ l ,2] (4.3.1a)

where

C(n)= [Ci(n), C2(n)] (4.3-lb)

and according to Eq.(3-20)

C(n+1) = C(n)P

The initial state vector reads:

C(0) = [Ci(0), C2(0)] = [1, 0] (4.3-lc)

i.e. it has been assumed that the initial concentration of the tracer in reactor 1 is

unity.

Page 360: Applications of Markov Chains in Chemical Engineering

354

Referring to Fig.4-1, yields for the configuration in Fig.4.3-1 that: Qj = Qi (i

= 2, 3, ...) = Qij = Qji = 0 or ttj = ai (i = 2, 3, ...) = ay = aji = 0, i.e., vessel j is

not considered, thus, a = 1 and ^ = 2. From Eq.(4-12a) follows that ai = P12 = 1

while taking Qi as reference flow. Applying Eqs.(4-16) and (4-17b), considering

the above information, yields pn = 1 - jiiAt pi2 = m^t where jiii = QiA^i, |Ll2 =

QlA^2 according to Eq.(4-7). p22 = 1 according to Eq.(4-22), since the tracer does

not leave reactor 2, thus, being a "dead state" with respect to the tracer. P2i = 0

since the system, i.e. a fluid element of the tracer, can not return from reactor 2 to

1. The above probabilities yield the following transition matrix:

P =

C i = l 1 - fXiAt

0

C2 = 2 ^2At

1 (4.3-ld)

Thus,

Ci(n+1) = Ci(n)[l - mAt ] C2(n+1) = Ci(n)|Li2At + C2(n)

In the numerical solution it was assumed that the reactors are of an identical

volume, thus, |ii = |i2 = M- The transient response of Ci and C2 is depicted in

Fig.4.3-la where the effect of |l = 1/tm is demonstrated. As seen, increasing |X (or

decreasing the mean residence time in the reactor) brings the reactors faster to

steady state.

Page 361: Applications of Markov Chains in Chemical Engineering

355

2

1.5

"1 r

H = 2 , n ^ = l -

1 I 1 r

Fig.4.3-la. Ci versus t demonstrating the effect of |LII and ^2

For At = 0.005, the agreement between the Markov chain solution and the

exact solution for Ci = Ci(0)exp(-|Liit) is Dmax = 2.5% and Dmean = 1-2% with

respect to |Xi = 2. The agreement is better for C2.

4.3-2 The flow configuration comprising of two perfectly-mixed reactors of

volumes Vi and V2 is demonstrated in Fig.4.3-2. A tracer in a form of a pulse

input is introduced into reactor 1 and is transferred by the flow Qi into reactor 2;

Q - Ql is the by-pass stream. The tracer is assumed to accumulate in reactor 2,

while flow Qi is leaving the reactor. This configuration simulates a situation

demonstrated in ref.[81], designated as "short-circuit". It should be noted that the

behavior in reactor 1 is independent of what happens in reactor 2.

Q,, *

Q-Qi

Fig.4.3-2. Perfectly-mixed reactors with a by-pass

Eqs.(4.3-la) to (4.3-lc) in the aforementioned case are applicable also in the

present case. The following definitions were made establishing the three

parameters of the system, v, \i and q:

Page 362: Applications of Markov Chains in Chemical Engineering

356

v = v - ^ l = v - q = - ^ ( l < q < o o )

Thus, Qi ^

H,= V 7 = q ^ 2 = -q

(4.3-2a)

(4.3-2b)

The transition probability matrix is identical to Eq.(4.3-ld) yielding: Ci(n+l) = Ci(n)[l-mAt] C2(n+1) = Ci(n)^2At + C2(n) (4.3-2c) where |i.i and |l2 are given be Eq.(4.3-2b). The pulse input is introduced into

reactor 1 bringing its concentration to unity. The transient response of Ci and C2

is depicted in Fig.4.3-2a where the effect of q is demonstrated. As seen, increasing

q, i.e. decreasing the flow rate Qi into reactor 1, reduces changes in the initial

concentration of the tracer in the reactor, whereas the concentration of the tracer in

reactor 2 remains almost unchanged, i.e. zero.

1.5

U^l

0.5

n

„_

1 1

/ l

1 1

q = 0.5 -1

J

0 0.5

u

1

0.8

0.6

0.4

0.2

0

1.5 2 0

~

I

h-

1 1

1 1

.--'•""" ^ = J

1 0 0.5 1.5 2 0

Fig.4.3-2a. Cf versus t demonstrating the effect of the by-pass ratio q for |i = 1, V = 2 and At = 0.002

Page 363: Applications of Markov Chains in Chemical Engineering

357

4>3-3

Fig.4.3-3 demonstrates two perfectly-mixed reactors. Reactor 1 contains a

"dead water" element of volume Vd [21, p.296] where the other part of volume Vi

is perfectly mixed. The total volume of the reactor is Vi+Vd and the volume of the

second reactor is V2. A tracer in a form of a pulse input is introduced into reactor 1

and is transferred by the flow Qi into reactor 2 where it is accumulating.

Fig.4.3-3. Perfectly-mixed reactors with a "dead water" element in

reactor 1

Eqs.(4.3-la) to (4.3-lc) in case 4.3-1 are applicable also in the present case

as well as the transition matrix given by Eq.(4.3-lf). The following definitions

were made: Vi + Vd ,^ ^ ^ Q,

V =

^ll = -rT- = V|I

( l < V < o o ) |X =

Qi

v,+v,

The parameters of the present configuration are: v, |LI and |X2- The transient

response of Ci and C2, computed by Eq.(4.3-2c), and E(t) given by Eq.(4-25), is

presented in Fig.4.3-3a for a pulse input into reactor 1. The effect of increasing v,

the magnitude of the "dead water" element, is clearly demonstrated for values of

0.5,1 and 2. The integration to obtain E(t) was performed numerically.

Page 364: Applications of Markov Chains in Chemical Engineering

358

2

1.5

U" 1

0.5

0

[-/'

I 1 _ . J

/ C 2

\ C andE(t)

1 r ——f

V = 0.5-i

-

-~ 1_

-| r

v = H

2 3 t

3k

U 1\-

0

\ 1

\ ~\

^E(t) \

1 \ L \ fe \

\

1

_:!- 1

1

C 2

^ ... 1

1

v = 2-J

-

1 0 0.5 2.5 1 1.5 2

t Fig.4.3-3a. Ci and E(t) versus t demonstrating the effect of v

f or |X = |i2 = 2

For At = 0.0025, the agreement between the Markov chain solution and the

exact solution for E(t) = |Llvexp(-^vt) [21, p.296] is Dmax = 4.4% and Dmean =

2.5% with respect to v = 2. The agreement is better for the smaller v.

Cases 4.3-4 to 4.3-7 in the following, demonstrate examples modeling long­

time scale behavior of real stirred reactors [21, p.269].

4.3-4 Increasing the residence time in the system may be achieved by adding reactor

2 in parallel to reactor 1. The pulse input is introduced into reactor 1 and is finally

accumulated in reactor 3.

Page 365: Applications of Markov Chains in Chemical Engineering

359

Fig.4.3-4. Configuration demonstrating long time-scale behavior

There are three states here, i.e.:

5S=[Ci ,C2 , C3]

where

C(n)= [Ci(n), C2(n), C3(n)]

and

(4.3-4a)

(4.3.4b)

(3-20) C(n+1) = C(n)P

The initial state vector reads:

C(0) = [Ci(0), C2(0), C3(0)] = [1.25, 0, 0]

j = l, a = 2, ^ = 3 where Q2 = a2 = 0. Eq.(4-12a) gives P13 = 1, i.e. Qi = qn .

Eq.(4-12b) yield that 1 + a2i = a n + a ^ ; thus a2i = ai2, i.e. Q12 = Q21. Eq.(4-

12c) gives for i = a = 2 that ai2 = 0C2b noting that a2 = 0, that the sum on the left-

hand side does not exist and that P23 = 0- ^^ addition:

Vi=fV V2 = ( l-f)V where V = Vi+V2 and 0 < f < l

Vi and V2 are the volumes of reactors 1 and 2 in Fig.4.3-4. From Eq.(4-7):

|Lii = QiA^l = ji/f |Li2 = QlA^2 = H/(l-f) ^3 = Ql/V3 where ^ = QiA^

From Eq.(4-14) for j = 1, i = a = 2, ^ = 3 and the above information:

Pll = 1 - (Pl3 + 0Li2)\i\At = 1 - (1+ ai2)(Myf)At

From Eq.(4-16) for i = 2, considering the above information:

P22 = 1 - cx2iM'2At = 1 - ai2|ii2At = 1 - a\2(\xJ(l - f))At

From Eq.(4-15) for j = 1 and i = 2:

P12 = oci2^2At = ai2(M/(l - f))At

FromEq.(4-15b)forj = l and ^ = 3:

P13 = CXlB^At = l|l3At

From Eq.(4-17a) for i = 2 and j = 1:

P21 = 0C2imAt = ai2(n/f)At

From Eq.(4-17b) for i = 2 and ^ = 3:

Page 366: Applications of Markov Chains in Chemical Engineering

360

P23 = p23^3At = 0|l3At = 0

The above probabilities are summarized in the following transition probability matrix with four parameters f = Vi A , b = ai2, |LI and ILI3:

P =

C i = l 1 - (l+b)(M/f)At

b(|i/f)At

0

C2 = 2 b(n/(l-f))At

1 - b(n/(l-f))At

0

C2 = 3 UsAt

0

1

From Eq.(3-20) above it follows that:

Ci(n+1) = Ci(n)[l - (1 + b)(|i/f)At] + C2(n)[b(n/f)At]

C2(n+1) = Ci(n)[b(My(l - f))At] + C2(n)[l - b(^/(l - f))At]

C3(n+1) = Ci(n)[^3At] + C3(n) (4.3-4c)

For 13 = |Lli and a pulse input of C\{0) = 1.25 introduced into reactor 1, the

transient response of Ci, C2 and C3 computed by Eq.(4.3-4c), is presented in Fig.4.3-4a demonstrating the effect of f = ViA^ and b = ai2.

Page 367: Applications of Markov Chains in Chemical Engineering

361

u

1.2

1

0.8

0.6

0.4

0.2

0

f X y W

rf\ r " 1

^ ^ —

1 3_

2

J _. — - - j

- 1

f = 0.2, b = 0.25

-

-

0.05 t

0.1 0

f = 0.2,b = 5

1 3 .-'

^

0.05 t

0.1

Fig.4.3-4a. Ci versus t demonstrating the effect of f and b for |LI = 20

For At = 0.0002, the agreement between the Markov chain solution and the

exact solution for Ci [21, p.303] is Dmax = 0.78% and Dmean = 0.48%. The exact

solution is restricted to Ci(0) = 1/f.

4>3-5 The configuration depicted in Fig.4.3-5 comprises of two interacting reactors

forming a closed recirculation system. The pulse introduced into reactor 1 is

distributed at steady state between reactors 1 and 2.

*

i ^ 2 1 = Q l 2

Fig.4.3-5. A closed recirculation system

There are two states, i.e.:

Page 368: Applications of Markov Chains in Chemical Engineering

362

SS= [Ci,C2]

where

C(n)= [Ci(n),C2(n)]

and

C(n+1) = C(n)P

The initial state vector reads:

C(0) = [Ci(0), C2(0)] = [1.25, 0]

Referring to Fig.4-1, yields for the configuration in Fig.4.3-5 that a (= i) = 1

and b = 2. As seen, this is a closed recirculation system, for which Eq.(4-12c)

gives that Q12 = Q21 or au = a2l- If Q12 is selected as a reference flow, Eq.(4-6)

gives a i2 = a2i = 1 and from Eq.(4-7) m = Q12/V1 = |LL/f, |i2 = Q12/V2 = M (l - 0

where 0 < f < 1 and |i = Q^A^ noting that V = Vi + V2 and V1/V2 = f/(l-f) =

|i2/|iil. pii and P22 are computed from Eq.(4-16), pi2 and p2i from Eq.(4-18),

yielding the following transition matrix:

P =

Thus,

C i = l

1 - iLiiAt

jilAt

C2 = 2

^2At

1 - |Li2At

Ci(n+1) = Ci(n)[l - mAt ] + C2(n)[mAt ]

C2(n+1) = Ci(n)[H2At] + C2(n)[l - ^2At ] (4.3-5a)

with two parameters |X and f = ViA^. The transient response of Ci and C2

computed by Eq.(4.3-5a), is presented in Fig.4.3-5a demonstrating the effect of f.

u

1.2

1

0.8

0.6

0.4

0.2

0

h

1 1

1

2..-' '"

1

1 1 f=0.8

1 1

1 J

-H

1

r ' --"''

V 1

1 1 f=0.5

- ""*

1 1

1 J

-

1 0.01 0.02 0.03 0.04 0.05 0

t 0.01 0.02 0.03 0.04 0.05

t

Page 369: Applications of Markov Chains in Chemical Engineering

363

0.02 0.03 t

0.05

Fig.4.3-5a. Ci versus t demonstrating the effect of f for |X = 20

For At = 0.0002, the agreement between the Markov chain solution and the exact solution for C2 [21, p.303] is Dmax = 1-3% and Dmean = 0.6%. The exact solution is restricted to Ci(0) = 1/f.

4>3-6 This case is an extension of the previous case for an open system. Reactor 3

was added as a collector for the tracer introduced into reactor 1.

*

•fcct" ' ^ 2 3 '

Fig.4.3-6. An open recirculation system

There are three states in this case for which Eqs.(4.3-4a) and (4.3-4b) are relevant. The initial state vector reads:

C(0) = [Ci(0), C2(0), C3(0)] = [1.25, 0, 0] The system depicted in Fig.4.3-6 is deduced from the general scheme in

Fig.4-1 in the following way. Designating j = 1, a = 2 and ^ = 3 while noting that reactor 2 is the only one belonging to the peripheral reactors i in Fig.4-1. In addition, P13 = a2 = 0. Eq.(4-12a) gives that P23 = 1. Eqs.(4-12b) and (4-12c), noting the above results, give both that a i 2 = l + a 2 i . Vi=fV, V2 = ( l - f)V

Page 370: Applications of Markov Chains in Chemical Engineering

364

where V = Vi + V2, yield m = QiA^i = | f, 112 = Q1/V2 = ^i/(l - f). ^3 = QlA^3

where 0 < f < 1, l = QiA^ and ViA^2 = f/(l-f) = m'm-

On the basis of the above results, the probabilities are obtained from the

following equations: pn from Eq.(4-14), pi2 from Eq.(4-I5), p2i from Eq.(4-

17a), p22 from Eq.(4-16) and P23 from Eq.(4-17b), yielding the following

transition matrix with four parameters f =V\/W, b = ai2, |l and 1x3:

P =

1

2

3

Ci = l 1 - b(nyf)At

(b-l)(n/f)At

0

C2 = 2 b(n/(l-f))At

l-b(n/(l-f))At

0

C3 = 3 0 1

^3At

1

Thus,

Ci(n+1) = Ci(n)[l - h(\i/f)m + C2(n)[(b-l)(|Li/f)At]

C2(n+1) = Ci(n)[b(^/(1 - f))At] + C2(n)[l - b(|Li/(l - f))At]

C3(n+1) = C2(n)[|ii3At] + €3(11) (4.3-6a)

Taking 1X3 = ^1, yields the transient response of Ci, C2 and C3 computed by

Eq.(4.3-6a), which is presented in Fig.4.3-6a demonstrating the effect of f and b.

As seen, by increasing b = ai2, and after some time, reactors 1 and 2 acquire an

identical concentration of the tracer, which is eventually vanishing and

accumulating in reactor 3.

Z \ L 0.02 0.04 0.06 0.08 0.1 0 0.02 0.04 0.06 0.08 0.1

t t

Page 371: Applications of Markov Chains in Chemical Engineering

365

1

1 r 11

i/ 1/ 1

1 1 1 _

?"'" f = 0.2,b = 5i

0 0.02 0.04 0.06 0.08 0.1 t

0.02 0.04 0.06 0.08 0.1 0 t

Fig.4.3-6a. Cf versus t demonstrating the effect of f and b for |Li = 20

For At = 0.0002, the agreement between the Markov chain solution and the

exact solution for C2 [21, p.303] is Dmax = 6.4% and Dmean = 3.4%. The exact

solution is restricted to Ci(0) = 1/f.

4,3-7 The fluid flow is divided into flows Qi and Q2 as shown in Fig.4.3-7. The

tracer, in a form of a pulse input, is introduced into reactor 1 whereas reactor 3 is a

collector for the tracer.

*

'13

\ 2 \ ^23

Q1+Q2

Fig.4.3-7. An open recirculation system with divided flow

There are three states in this case for which Eqs.(4.3-4a) and (4.3-4b) are

relevant, where C(0) = [Ci(0), C2(0), CsCO)] = [1, 0, 0]

The system depicted in Fig.4.3-7 can be deduced from the general scheme in

Fig.4-1 in the following way. Designate a = 1, b = 2 and ^ = 3 where reactor j is

not considered here. Taking Qi + Q2 as a reference flow rate and defining:

Page 372: Applications of Markov Chains in Chemical Engineering

366

a i = (4.3-7a) Q1+Q2

resulted in by applying Eq.(4-12a), while noting that Oj = 0, that:

a, + a2 = 1 = Pi3 + P23 (4.3-7b)

Eq.(4-12c) for i =1, 2, respectively, yields:

aj + a2i = aj2 + P13 and a2 + ai2 = 0,21 + P23 (4.3-7c)

It should be noted that the expression on the RHS is obtained from the one on the

LHS by applying Eq.(4.3-7b).

Designating V = Vi + V2, f = VjA^ where f - 1 = V2A/ , yields by considering

Eqs.(4-5) and (4-7), that ^ = (Qi + Q2)A i = fi/f, ^2 = (Ql + Q2)A 2 = H/(l - 0,

^3 = (Ql + Q2)A 3 0 < f < 1 where ^ = (Qi + Q2)A .

On the basis of the above results, probabilities are obtained by applying the followin;

eters f, b = ai2, q = P13, a i , \i and fi3. Thus, from Eq.(4.3-7c) a2i = b + q - a i ,

P23 = 1- q- Note that for a i = q, a2i = ai2 = b.

P =

Thus,

C i = l l-(q+b)(n/f)At

C2 = 2 b(M/(l-f))At

C2 = 3

qR3At

a2l(H/f)At l-[a2i+(l-q)](M/(l-f))At (l-q)|l3At

0 0 1

Ci(n+l) = Ci(n)pii+C2(n)p2i

C2(n+1) = Ci(n)pi2+ C2(n)p22

C3(n+1) = Ci(n)pi3 + C2(n)p23 + C3(n) (4.3-7d)

Taking 13 = -1. yields the transient response of Ci, C2 and C3 computed by

Eq.(4.3-7d), which is presented in Fig.4.3-7a. demonstrating the effect of q = tti

given by Eq.(4.3-7a).

Page 373: Applications of Markov Chains in Chemical Engineering

367

1

L .^ \1 3/

1 / - \ /

pK \ ^\---—

. — -r — ""

H

q = 0.5

-

1 0.1 0.2 0.3 0 0.1 0.2

t t

Fig.4.3-7a. Cf versus t demonstrating the effect of q for |i = 20, b = 1, f = 0.5 and At = 0.0005

0.3

4.3-8 The closed recirculation system shown below comprises of Z perfectly-mixed

reactors of not the same volume. If a tracer is introduced in a form of a pulse input

into the first reactor, the recorder will measure the tracer as it flows the first time,

the second time, and so on. In fact, it measures a tracer which passed through Z

reactors, 2Z reactors and so on, i.e. the superposition of all these signals.

[ 1

Q 2 \ i-1

dbn Q r ^ i+i

o^ z h

Fig.4.3-8. A closed recirculation system

Referring to Fig.4-1 gives the following information for the system in

Fig.4.3-8, i.e. a = 1, b = 2, etc. In addition, since the system closed and reactor j

is not included, ai = pi^ = ay = Oji = 0. Therefore, only Eq.(4-12c) is applicable,

yielding, by considering Eq.(4-8), that ai-i^i = ai^i+i = 1 for i = 2,..., Z-1. From

Eqs.(4-16) and (4-19a), respectively, one obtains that pjj = 1 - jUjAt and Pi,i+i =

|Lli+lAt, i = 1, ..., Z-1; pz,i = |LliAt. The parameter JLLI is given by Eq.(4-7), i = 1,

2,..., Z where Qj is replaced by Q in Fig.4.3-8. The above information yields the

following transition matrix:

Page 374: Applications of Markov Chains in Chemical Engineering

368

1

2

3

= i-1

i

Z-1

Z

1 l-^ijAt

0

0

0

0

0

^lAt

2 ^,2^1

l-|i2At

0

0

0

0

0

3

0

l3At

1-^3 At

0

0

0

0

. . .

0

0

i-1

0

0

0

1-h-iAt

0

0

0

i

0

0

0

ijAt

1-^At

0

0

. . .

0

0

z-1 0

0

0

0

0

1-l^z-iAt

0

z 0

0

0

0

0

^zAt

l-^zAt

Thus,

Ci(n+1) = Ci(n)[l - mAt] + Cz(n)[|iiiAt]

Ci(n+1) = Ci.i(n)[mAt] + Ci(n)[l - mAt] i = 2, ..., Z-1

Cz(n+1) = Cz-i(n)[^zAt] + Cz(n)[l - M t ]

A particular solution is obtained for Z = 5 and a constant JLII, i.e. all reactors

have the same volume. For the pulse input C(0) = [Ci(0), C2(0), C3(0), C4(0),

^5(0)] = [1, 0, 0, 0,0] the response curve for Ci to C5 is depicted in Fig.4.3-8a.

Fig.4.3-8a. Ci versus t for |LI = 20

For At = 0.0005, the agreement between the Markov chain solution and the

exact solution for C5 [21, p.295] is as follows: for C5,exact = 0.001, D = 8.9%;

Page 375: Applications of Markov Chains in Chemical Engineering

369

C5,exact = 0.01, D = 3.3%; for Cs.exact = 0.1, D = 0.074% and for Cs.exact = 0.2,

D = 0.012%.

4.3-9 The following scheme is an open recirculating system with a recycle of

magnitude Qzi-

^1+^21 Qi+Qzi Qi+Qzi

|q5L_»Jro| ^ Jro]—Jrol—Jrol V « z i

^

-1.i ^l,i+1

^ I zrnz+i I

^ Z 1

Fig.4.3-9. An open recirculation system with recycle

The above system can be deduced from Fig.4-1 as follows: j = 1, a = 2, etc.

where ^ = Z+1; in addition Q2 = Q3 =,..., Qz = 0 or a2 = as = ,..., oCz = 0.

Eq.(4-12a) gives 1 = Pz,z+l-

Eq.(4-12b) gives 1+ ttzi = OL12. If we define recycle by R = Qzi/Qi and

consider Eq.(4-2) for kj = zl , it follows that azi = R and that also ai2 = 1 + R.

Eq.(4-12c) gives for i = a = 2that ai2 = cx23,

for i = 3, 4. ..., Z-1, it gives that ai.i j = ai,i+] = 1 + R,

and for i = Z it is obtained that az-i,z = Pz,z+l(=l) + ^z\ = R + 1.

The following probabilities were obtained:

Eq.(4-14) gives pn = 1- ai2|iiAt = 1- (1 + R)fiiAt

Eq.(4-16) gives pii = 1 - ai,i+i|XiAt = 1- (1 + R)mAt i = 1,..., Z-1

where Pzz = 1 - (Pz,z+1 + cXzOMt = 1 - (1 + R)|LLzAt .

Eq.(4-15) gives pi2 = ai2H2At = (1 + R)|i2At

Eq.(4-18a) gives pi,i+i = ai,i+i|ii+iAt = (1 + R)|ii+iAt i = 1,..., Z - 1

Eq.(4-17b) gives pz,z+l = Pz,z+mz+lAt = l|iz+lAt i = Z

Eq.(4-17a) gives pzi = oCzlHlAt = R|LIIAt

For Z = 5, while assuming that ^1 = 11 and considering the above probabilities,

gives that:

Page 376: Applications of Markov Chains in Chemical Engineering

370

P =

1

2

3

4

5

6

1

1- (l+R)M.At

0

0

0

RuAt

0

2

(l+R)jiAt

1- (l+R)^At

0

0

0

0

3 4

0 0

(l+R)nAt 0

1- (l+R)nAt (l+R)(iAt

0 1- (l+R)|iAt

0 0

0 0

5

0

0

0

(l+R)nAt

1- (l+R)nAt

0

6

0

0

0

0

|iAt

1

For the pulse input C(0) = [1, 0, 0, 0,0, 0], the response curve for Ci to C6, computed from C(n-i-l) = C(n)P, is depicted in Fig.4.3-9a.

0.5 1 1.5 0 0.5 1 t t

Fig.4.3-9a. Cf versus t demonstrating the effect of R for ^ -10 and At = 0.01

4,3-10

1 W^

2 — ^ '

3 — ^

4 - ^

^

5

^56 ^- 6

Fig.4.3-10. Perfectly mixed reactors with baclc flow

The equations for the above configuration [21, p.298] can be obtained from Fig.4-1 by designating j = 1, a = 2, b = 3,..., Z = 5 and | = 6. In addition, Qji = Qij = 0 or aij = aji = 0,i = 3,...,Z; Qi = ai = 0,i = 2,...,Z.

Page 377: Applications of Markov Chains in Chemical Engineering

371

Eq.(4-12a) gives 1 = ^55; Eq.(4-12b) gives 1 + aii = an', Eq.(4-12c) for i = a = 2 gives ai2 + OC32 = 0 21 + 0 23, hence 1 + a32 = CX23; for i = 3, a23 + a43 = a32 + a34, hence 1 + a43 = a34; for i = 4, a34 + a54 = a43 + a45, hence, 1 + OC54 = a45; for i = 5, a45 = p56 + ^54.

The following probabilities were obtained considering the above results: pll = 1 - ai2|iiAt from Eq.(4-14). Pii = 1 - (ttij + ai,i.i + ai,i+i)^iAt from Eq.(4-16), i = 2, ..., Z - 1, i ^ j , ^.

For i > 2, ay = 0. In addition, Pzz = 1 - (Pz + 0Cz,z.i)|XzAt. P12 = 1 - 0Ci2|Lt2At from Eq.(4-15). Pi,i+1 = 0 i,i+lW+lAt from Eq.(4-18a), i = 2, ..., Z - 1 where from Eq.(4-

17b) pz = Pz l At. Finally, Pi+l,i = ai+i,i|liAt from Eq.(4-18b), i = 1,..., Z - 1. Assuming that ^i = \i, a2i = a32 = a43 = a54 = R, thus, ai2 = a23 = a34 =

OC45 = 1 + R where the rest values of a are zero. Considering the above, yields the following transition probability matrix which can be easily extended to a higher number of states:

P =

1

2

3

4

5

6

1

1- (l+R)^lAt

R^At 1

0

0

0

0

2

(l+R)|iAt

- (l+2R) lAt

R^At

0

0

0

3

0

(l+R)^At

1- (l+2R)MAt

RuAt

0

0

4

0

0

(l+R)MAt

1- (l+2R)nAt

R lAt

0

5

0

0

0

(l+R)^At

1- (l+R)nAt

0

6

0

0

0

0

\iAt

1

For the pulse input C(0) = [1, 0, 0, 0, 0, 0] the response curve for Ci to €5, computed from C(n+1) = C(n)P according to above matrix, is depicted in Fig.4.3-10a. It may be observed that for large value of the back flow R, the concentration of the tracer becomes uniform after some time, i.e. all reactors act as a single reactor.

Page 378: Applications of Markov Chains in Chemical Engineering

372

u

1

0.8

0.6

0.4

0.2

^0 0.5 1 1.5 0 0.5 1 1.5 t t

Fig.4.3-10a. Ci versus t demonstrating the effect of R for |LI = 10 and

At = 0.0005

r-

i = l

1

6,

'T' ^——

1

R=10

\

4,3-11 The following configuration simulates a flow pattern in a tubular reactor [21,

p.334, case d] in the presence of side-leaving streams.

^ Q i

'15

^23 d o ! 3 4 1 ^

25 *35

45 doi_2.

u Fig.4.3-11. Perfectly mixed reactors with side-leaving streams

The equations for this configuration can be obtained from Fig.4-1 as

follows, designating j = l , a = 2, b = 3,...,Z = 4 and ^ = 5 while noting that ai =

0, Qii = 0 for i = 2,..., 4.

Eq.(4-12a) gives: 1 = P15 + p25 + P35 + P45. Eq.(4-12b) gives: 1 = P15 +

ai2. Eq.(4-12c) gives: for i = a = 2, an = P25 + OC23; for i = 3, a23 = p35 + a34;

for i = 4, a34 = P45. Thus, if three values of p are given, the rest of the

coefficients are known.

The following probabilities were obtained considering the above results:

Pll = 1 - (Pl5 + cxi2)mAt from Eq.(4-14)

Pii = 1 - (Pi5 + 0Ci,i+i)|iiAt from Eq.(4-16), i = 2, 3 where

P44 = 1 - P45M'4At

In addition:

P12 = 0Ci2|Li2At from Eq.(4-15),

Page 379: Applications of Markov Chains in Chemical Engineering

373

Pi,i+1 = «i, i+lR+l^t from Eq.(4-18a), i = 2, 3

Pl5 = PiaM-S' t from Eq.(4-15b), and

Pi5 = Pis^sAt from Eq.(4-19), i = 2, 3, 4

Defining |Xi = QiA^i, = QA^ where V = Vi + V2 +V3 +V4 and Vi = mV, V2 =

nV, V3 = pV, V4 = (1 - m - n - p)V for 0 < m + n + p < 1, while considering the

above results, gives the following transition probability matrix:

P =

1

2

3

4

5

1

Pll

0

0

0

0

2

P12

P22

0

0

0

3

0

P23

P33

0

0

4

0

0

P34

P44

0

5

P15

P25

P35

P45

1

A specific example was obtained for V5 = Vi, i.e. (15 = |ii = \Um. Other

parameters were: P15 = 0.06 and 0.6, P25 = 0-2, P35 = 0.1; m = 0.5, n = 0.3, p =

0.1, and |X = 2. For the pulse input C(0) = [1, 0, 0, 0, 0], the response curve for

Ci to C5, computed from C(n+1) = C(n)P, is depicted in Fig.4.3-lla for At =

0.005.

1

0.8

0.6 r 0.4

0.2

0

\ I 1, 1

\ i = 1

1

0 = 0.06 IS

1

\ 1 1 ,J^ \ 5

V - \ •

A?

1 1 H

a =0.6 15 J

" • - .

0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 t t

Fig.4.3-lla. Ci versus t demonstrating the effect of a i 5

2.5

Page 380: Applications of Markov Chains in Chemical Engineering

374

4,3-12 The following configuration is in some aspects similar to the previous one. It

simulates a flow pattern in a tubular reactor [21, p.334, case b] in the presence of

side feedings.

h 12

m ''23

QN JO^ Ji^ 34 45

Fig,4.3-12. Perfectly mixed reactors with side feeding

The equations for this configuration are obtained from Fig.4-1 by

designating a = 1, b = 2,..., Z = 4 and ^ = 5 where reactor j is not active and the

reference flow is Q = Qi + Q2 + Q3 + Q4. As seen, the state space comprises of

five reactors. Eq.(4-12a) gives: a i + a2 + as + a4 = 1 = P45; Eqs.(4-12c) gives:

a i = a i2 for i = 1, a2 + an = a23 for i = 2, as + a23 = a34 for i = 3 and a4 +

a34 = p45. From the above equations it may be concluded that if a i , a2 and as

are specified, the rest of the coefficients are known. Thus, the following

probabilities are obtained from Eq.(4-16):

pii = 1 - ai2mAt P22 = 1 - a2sH2At ps3 = 1 - as4|i3At

P44 = 1 - p45R4At

The following probabilities are obtained from Eq.(4-18a):

P12 = ai2^2At P23 = a23|X3At P34 = as4H4At and from Eq.(4-19)

P45 = P45l 2At

From Eq.(4-7) |ii = QA i where \i = QA^ and Q = Qi + Q2 + Qs + Q4. In addition,

V = Vi + V2 +V3 +V4 where Vi = mV, V2 = nV, Vs = pV and V4 = (1 - m - n -

p)V for 0 < m + n + p < 1. Thus, |Xi = p,/m, |i2 = |Li/n, ILL3 = M-Zp and 114 = |i/(l-

m-n-p). Considering the above results, gives the following transition probability

matrix:

Page 381: Applications of Markov Chains in Chemical Engineering

375

P =

1

2

3

4

5

1

Pll

0

0

0

0

2

P12

P22

0

0

0

3 0

P23

P33

0

0

4 0

0

P34

P44

0

5 0

0

0

P45

1

A specific example was obtained for V5 = Vi, i.e. 115 = p,i = |i/m. Other

parameters were: a i = 0.2 and 0.6, a2 = 0.2 and as = 0.1; ai is defined by

Eq.(4-12d) where Qj = Qi + Q2 + Q3 + Q4. m = 0.5, n = 0.3, p = 0.1, and |Li = 2.

For the pulse input C(0) = [1, 0, 0, 0, 0], the response curve for Ci to C5,

computed from C(n+1) = C(n)P, is depicted in Fig.4.3-12a for At = 0.005.

tti = 0.2 ttj = 0.6

I v

- A . .•V-rS

r" r

1 . • - 1 - 1

5 /

-1

\ '* --»~.

4 0

Fig.4.3-12a. Cf versus t demonstrating the effect of a^

4.3-13 The following configuration demonstrates an interacting system whereby it is

possible to increase the mean residence time by applying operating different

streams in the flow system. Reactor 5 is the tracer collector.

Page 382: Applications of Markov Chains in Chemical Engineering

376

^ 1 f ^

'12

^

13

Q

24

35 ^

4 A

'43

oloL,242jd& A 4

''45

25

Fig.4.3-13. A model for demonstrating the increasing of the

mean residence time

The above configuration is obtained from the general scheme in Fig.4-1 by

designating j = l ,a = 2, b = 3,... . Z = 4 and ^ = 5 which is the 5th state.

Eq.(4-12a) gives: 1 = p25 + p35 + p45

Eq.(4-12b) gives: 1+ aj i + 0.31 = an + ai3

Eq.(4-12c) gives: an + 042 = P25 + «2l + OC24 for i = 2,

a i3 + a43 = P35 + a3i + a34 for i = 3 and a24 + 034 = P45 + 042 + 043

fori = 4. (4.3-13a)

The above five equations contain eleven coefficients; thus, six of which must be

prescribed in order to solve for the others. The following probabilities of

remaining in the state are obtained from Eqs.(4-14) and (4-16): pii = l - ( a i 2 + oi3)|XiAt

P22 = 1 - (P25 + 021 + a24)^2At

P33 = 1 - (P35 + 031 + 034)^3At

p44 = 1 - (P45 + 042 + a43)|X4At (4.3-13b)

The following probabilities of leaving the state are obtained fi-om Eqs.(4-15):

P12 = ai2|i2At pi3 = ai3|X3At

From Eq.(4-18b):

P21 = 02imAt

FromEq.(4-18a):

P24 = 024mAt

From Eq.(4-17b):

P25 = p25^5At

P31 = 03i|XiAt

p34 = 034|X4At

P42 = a42fl2At P43 = a43li3At

P35 = p35^5At P45 = p45R5At (4.3-13C)

The above probabiUties are arranged in the following matrix:

Page 383: Applications of Markov Chains in Chemical Engineering

377

P =

1 2 3 4 5

Pll P12 P13 0 0

P21 P22 0 P24 P25

P31 0 P33 P34 P35

0 P42 P43 P44 P45

0 0 0 0 1 (4.3-13d)

The above results are applied for the two cases below, 4.3-13(1) and 4.3-13(2).

4.3-13(1) The following assumption were made in the present case referring to

Fig.4.3-13: Q12 = Q21 = Q34 = Q43, Ql3 = q35 where Q31 = Q24 = Q42 = q25 =

q45 = 0. Thus, ai3 = P35 = 1, ai2 = Oiil = OC34 = a43 = a where the rest of the

coefficients are zero. The above scheme reduces to the following on:

*

12

'13 ^

Q2I ^34

^

»35 *

MB

^

Also, all reactors are of the same volume, i.e. Vj = Vj = V, thus, it follows from

Eqs.(4-3) and (4-7) that ^j = p,i = ^ = QiA . The matrix given by Eq.(4.3-13d) is

reduced to the following one where Eqs.(4.3-13a) to (4.3-13c) above, are

applicable, considering the coefficients which are zero.

P =

1

2

3

4

5

1 l-(l+a)^At

a|iAt

0

0

0

2 a|jAt

l-a^At

0

0

0

3 HAt

0

l-(l+a)|iAt

anAt

0

4

0

0

a|xAt

l-a|xAt

0

5

0

0

^At

0

1

Page 384: Applications of Markov Chains in Chemical Engineering

378

For pulse inputs C(0) = [1, 0, 0, 0, 0] and [0, 1, 0, 0, 0], the response

curves for Ci to C5 were computed from C(n+1) = C(n)P. A specific example

was obtained for p, = 1 where the effects of a = 0.5 and 5 as well as Ci(0) and

C2(0) are demonstrated in Fig.4.3-13(l) for At = 0.005. As seen in cases (a) and

(b) for the same Ci(0) = 1, by increasing a, i.e. the fluid exchange rate between

the reactors, the difference in the concentration between reactors 1-2 and 3-4,

diminishes faster. A similar behavior is observed in cases (c) and (d) for the same

C2(0) = 1. The effect of Ci(0), i.e. the initial location of the pulse, is demonstrated

in cases (a) and (c) for a = 0.5 as well as (b) and (d) for a = 5. As seen, for a =

0.5, the concentration profiles are different, whereas for a = 5 they are identical

with respect to reactors 3 and 4 where rector 1 replaces 2 because of the initial

location of the pulse.

The mean residence time tm in the flow configuration demonstrated above

was computed by Eq.(4-26) for the response in reactor 3. Theeffect of a = ai2

= a2i = a34 = 043 on tm is as follows. For a = 0, only reactors 1 and 3 are active

and tm = 2; for a > 1, i.e. all four reactors are active and tm = 4, demonstrating the

interaction between the vessels on tm-

1

0.8

0.6

0.4

0.2

0

, ' ' ' 1 Ua)

r\'"^ •' ^ 3 A •

t — 1 — • ( • • " 1

C^(0)=l,a=0.5

-J 1 r-

(b)

10 0

n 1 r-

C ( 0 ) = l , a = 5

10

Page 385: Applications of Markov Chains in Chemical Engineering

379

u

1

0.8

0.6

0.4

0.2

0

r 1 1

\(C)

2*-.

1 1 .•' ''

1 1 1 1

^ C (0 = 1, a = 0.5 2

'' ''''' "r - ^

n 1 r (d)

•.2 C (0) = 1, a = 5

10 0 10

Fig.4.3-13(l). Ci versus t demonstrating the effect of Ci(0), C2(0) and a for |LI = 1

4>3-13(2) Referring to Fig.4.3-13, the following assumption were made in the present

case: Q12 = Q34 = q25 = q45. Thus, an = OC34 = P25 = P45 = a where the rest of the coefficients are zero. The above scheme reduces to the following one:

[4 2

13 rdoi — \ ^ 1

4 r

i

^45

5J I r

Also, all reactors are of the same volume, i.e. Vj = Vi = V, thus, it follows from Eqs.(4-3) and (4-7) that |Xj = ^i = ji = QiA . The matrix given by Eq.(4.3-13d) in case 4.3-13 above reduces to the following one where also Eqs.(4.3-13a) to (4.3-13c) there are applicable, considering the coefficients which are zero.

Page 386: Applications of Markov Chains in Chemical Engineering

380

p =

1

2

3

4

5

1 1-HAt

0

0

0

0

2 a|iAt

l-a|iAt

0

0

0

3 (l-a)iiAt

0

l-(l-a)|jAt

0

0

4 5

0 0

0 a|iAt

ajiAt (l-2a)|iAt

l-ajiAt apAt

0 1

For the pulse input C(0) = [1, 0, 0, 0, 0], the response curve for Ci to C5 is

computed from C(n+1) = C(n)P. A specific example was obtained for |x = 1

where the effects of a = 0, 0.4 and 1 is demonstrated in Fig.4.3-13(2) for At =

0.02. As seen in cases a and c, a = 0 and 1, the concentration profiles are

identical whereas rector 2 replaces 3.

The mean residence time tm in the reactors' configuration demonstrated above

was computed by Eq.(4-26) for the mean concentration of streams q25, qss and

q45. The effect of a = an = OC34 = p25 = p45 on tm is as follows. For a = 0,

only reactors 1 and 3 are active and tm = 2; for 0 < a < 1, all four reactors are

active and tm = 4 whereas for a = 1, again two reactors are active, 1 and 2,

demonstrating the flow interaction effect between the vessels on tm-

a = 0

1.5 0

a = 0.4

Page 387: Applications of Markov Chains in Chemical Engineering

381

Fig.4.3-13(2). Ci versus t demonstrating the effect a for |i = 1

4,3-14 The following configuration is a generalized scheme for demonstrating non

ideal mixing vessel [77].

Qa'^^a^Q/Qs

Fig.4.3-14. A generalized model for demonstrating non ideal mixing vessel

The above configuration is obtained from the general scheme in Fig.4-1 by designating j = l ,a = 2, b = 3, ..., Z = 5 and the collector ^ = 6 which is the 6th state. Eq.(4-12a) gives: a2 + as +, 04 + as = p56 Eq.(4-12b) gives: a2i + asi + a4i + asi = an + a n + ai4 + OL\S

Page 388: Applications of Markov Chains in Chemical Engineering

382

Eq.(4-12c) gives: tti + ai2 + 0.32 + 0152 = 01,21 + 01.23 + 0025 for i = 2,

as + ai3 + a23 + 043 = 0131 + 032 + a34 for i = 3,

04 + ai4 + a34 + a54 = 041 + a43 + 045 for i = 4,

as + ai5 + a25 + "45 = asi + a52 + 054 + P56 for i = 5

(4.3-14a)

The six equations contain twenty one coefficients; thus, fifteen of which must be

prescribed in order to solve for the others.

The following probabilities of remaining in the state are obtained from

Eqs.(4-14) and (4-16):

Pii = 1 - (ai2 + ai3 + ai4 + ai5)mAt

P22 = 1 - (a2i + a23 + a25)|A2At

P33 = 1 - (asi + a32 + a34)^3At

P44 = 1 - (041 + a43 + a45)M4At

P55 = 1 - (asi + a52 + a54 + p56)^5At (4.3- 14b)

The following probabiUties of leaving the state are obtained:

FromEqs.(4-15): P12 = ai2^2At P13 = ai3|X3At P14 = ai4|l4At P15 = ciis^sAt

(4.3-14c)

From Eqs.(4-19a, 19b):

P21 = a2imAt

P31 = asmiAt

P41 = a4imAt

P5i = a5imAt

P56 = p56^6At

P23 = a23|A3 At

P32 = a32H2At

P43 = a43|X3At

P52 = a52^2At

P25 = a25 t5At

P34 = a34|X4At

P45 = a45^5At

P54 = a54|X4At and from Eqs.(4-20)

(4.3-14d)

The above probabilities are arranged in the following matrix:

P =

1

2

3

4

5

6

1

Pll

P21

P31

P41

P51

0

2

P12

P22

P32

0

P52

0

3

P13

P23

P33

P43

0

0

4

P14

0

P34

P44

P54

0

5

P15

P25

0

P45

P55

0

6

0

0

0

0

P56

1 (4.3-14e)

Page 389: Applications of Markov Chains in Chemical Engineering

383

A simplified case appears in ref.[77], treated below in 4.3-14(1).

4,3-14(1) The following assumption were made in the present case while referring to

Fig.4.3-14: Q23 = Q12 = Qsi = Q34 = Q43 = Q54 = Q25 = Q52 = 0, or ais = an =

asi = a34 = a43 = a54 = ais = OC52 = 0- Also, Q2 = Q3 = Q5 = 0, or a2 = a3 =

as = 0 as well as pi6 = P26 = p36 = P46 = 0- However, a4 = 1 since Q4 is the

reference flow rate noting that Qj = Qi = 0. Additional assumptions made were:

ai4 = a4i = a; a is = asi = y; a n = a32 = a2i = p. From Eq.(4.3-14a) for i = 4

and 5, it follows that a45 = P56 = 1. AH reactors are of the same volume, i.e. Vj =

Vi = V, thus, it follows from Eqs.(4-3) and (4-7) that |LIJ = p,i = |i = Q4/V.

Consequently, the above scheme reduces to the following one.

The matrix given by Eq.(4.3-14e) in case 4.3-14 above is reduced to the

following one where also Eqs.(4.3-14a) to (4.3-13d) there are applicable,

considering the coefficients which are zero.

P =

1 l-(a+p-py)|xAt

PnAt

0

a|iAt

7|iAt

1 ^

2 0

1-pHAt

PjlAt

0

0

0

3 p^At

0 l-P|aAt

0

0

0

4 a^At

0

0

l-(l+a)^At

0

0

5 6 THAt 0

0 0

0 0

lAt 0

l-(l+Y)HAt ^At

0 1

Page 390: Applications of Markov Chains in Chemical Engineering

384

For a pulse input creating a unit concentration in the reactor, which is introduced into reactors 1 or 3 or 4 or 5, the response curves given below for Ci to C6 is computed from C(n+1) = C(n)P.

1

u

n 1 1 r- 1 I r~

0.8 p. i = 1

0.61

r 0.4

0.2

0

C j ( 0 ) = l

6

- : ' l " • ^ '

V 1 1 1 1 1 1

C,(0)=1

^ \

- - 3 '^

1 6^ r -" """ 1 , -' ^ 1 ^... K. . i . . - -1-; - -. I -r :-'r-''-'r-"-'"i-" -

'

5 ^

' 1

2

\---^/-10 0 10

^4(^1= 1

1 1 1 r

10 0

Fig.4.3-14(l). Cf versus t demonstrating the effect of the introduction location of the tracer

Fig.4.3-14(l) demonstrates the effect of the introduction location of the pulse on the response curves for a = ai4 = a4i = 0.05, P = a n = a32 = OL21 = 0.1, y = 0C15 = asi, i = 1 and At = 0.004.

4.3-15 The following configuration is a generalization the example in [78, p.74]

which aimed at describing various conditions of mixing.

Page 391: Applications of Markov Chains in Chemical Engineering

Fig.4.3-15. Parallel reactors model

The above configuration is obtained from the general scheme in Fig.4-1 by

designating a = 1, b = 2,..., Z and the collector ^ which is the Z + 1 state,

Eq.(4-12a) gives:

a i + a2 = 1 = pz-l,^ + Pz, (4.3-15a)

where the reference flow is Qi + Q2 and

ai = Qi/(Ql + Q2) i = l , 2 (4.3.15b)

Eq.(4-12c) gives:

cxi + OL2\ + as i = a i2 + a n for i = 1

0 2 + OC12 + a32 + a42 = ^21 + ^23 + ^24 for i = 2

a i3 + a23 + OC43 + OC53 = a3i + a32 + a34 + OC35 for i = 3

0 24 + 0 34 + a54 + a64 = a42 + a43 + a45 + a46 for i = 4

CX35 + OC45 + a65 + OC75 = CX53 + CX54 + OC56 + OC57 for i = 5

0 46 + OC56 + ^76 + OC86 = a64 + CX65 + a67+ OC68 for i = 6

ai-2,i + ai-i,i + ai+i,i + ai+2,i = ai,i.2 + ai,i.i + ai,i+i + ai,i+2 3 < i < Z-2

az-4,z-2 + 0Cz-l,z.2 + CXz,z-2 = az.2,z-4 + CXz-2,z-l + 0Cz-2,z for i = Z-2

0Cz-3,z-l + CXz-2,z-l + az,z-l = 0Cz-l,z-3 + az-l,z-2 + az-l,z + Pz-l,^ for i = Z-l

0Cz-2,z + az-i,z = az,z-2 + cxz,z-i + Pz, for i = Z

(4.3-15C)

where Z = 5, 7, 9,... and ^ = 6, 8, 9,..., respectively.

The following probabilities of remaining in the state are obtained from Eq.(4-16):

pil = l - ( a i 2 + ai3)mAt

P22 = 1 - (a21 + OC23 + a24)|Ll2At

P33 = 1 - (a3i + a32 + a34 + a35)|i3At

P44 = 1 - (CX42 + CX43 + a45 + a46)l^4At

Page 392: Applications of Markov Chains in Chemical Engineering

386

Pii = 1 - (ai,i-2 + ai,i.i + ai,i+i + ai,i+2)mAt 3 i < Z-2

Pz-l,z-l = 1 - (az-l,z-3 + az-l,z-2 + Otz-Lz + Pz-l, )^z-lAt

Pzz = 1 - (az,z-2 + CCz,z-l + Pz,4)Mt

The following probabilities of leaving the state are obtained:

From Eqs.(4-17b):

Pz-14 = Pz-l, fi At pz% = Pz i At

From Eqs.(4-18) for k, i = 1, 2,..., Z-2; k vt i

Pki = cCkiRAt

The above probabilities are arranged in the following matrix:

P =

1

2

3

4

5

Z-2

Z-1

z

1 2 3 4 5 6

Pll P12 P13 0 0 0

P21 P22 P23 P24 0 0

P31 P32 P33 P34 P35 0

0 P42 P43 P44 P45 P46

0 0 P53 P54 P55 P56

(4.3-15d)

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

owing

i ...

0 ...

0 ...

0 ...

0 ...

0 ..

0 ..

Pii •• 0 ..

0 ..

0 ..

0 ..

0 ..

matrix:

z-2 0

0

0

0

0

0

0

0

Z-1

0

0

0

0

0

0

0

0

Pz-2,z-2 Pz-2,z-l

• Pz-l,z-2 Pz-l,z-l

• Pz,z-2 . 0

Pz,z-1 0

(4.3-15e)

(4.3-15f)

Z

0

0

0

0

0

0

0

0

Pz-2,z

Pz-l,z

Pzz 0

^ 0

0

0

0

0

0

0

0

0

Pz-1,4

Pz4 1

4.3-15(1) The above model was demonstrated graphically for the following

configuration consisting of six states, i.e. Z = 5 and = 6.

Page 393: Applications of Markov Chains in Chemical Engineering

387

Q.+Qo

56

Considering Eqs.(4.3-15a) and (4.3-15c) above gives:

1 = P46+P56 oci + a2i + asi = ai2 + ai3 for i = 1

0C2 + Otl2 + ^32 + a42 = OC21 + a23 + a24 for i = 2

CX13 + CX23 + CX43 + a53 = 0631 + OC32 + a34 + a35 for i = 3

CX24 + CX34 + a54 = a42 + a43 + a45 + P46 for i = 4

a35 + a45 = OC53 + a54 + P56 for i = 5

As observed, there are six equations and eighteen unknowns. It has been

assumed that:

CX12 = a2i = b a23 = a32 = d a43 = a34 = e a45 = a54 = f

a42 = a53 = a3i = R (4.3-15(l)a)

Designating ai and a2, defined in Eq.(4.3-15b), by:

ai = a a 2 = l - a (4.3-15(l)b)

where from the above equations it follows that:

a i = p56 CL2 = P46

a24 = R + 1 - a ai3 = a35 = R + a

Thus, the solution is reduced to six parameter: a, b, d, e, f, R and m = \i. The

following probabilities are obtained from Eqs.(4.3-15d, e, f) and the above

parameters:

pi 1 = 1 - (a + b + R)|iAt P12 = b|LiAt P13 = (a + R)|iAt

P 2 2 = l - ( b + d + R + l - a)^At p2i = bfiAt p23 = d|LlAt

P24 = (R + 1 - a )|LiAt

P33 = 1 - (a + d + e + 2R)^At p3i = R|iAt P32 = d|LiAt P34 = e|iAt

P35 = (a + R)|iAt

P 4 4 = l - ( l - a + e + f + R)|iAt p42 = R|iAt P43 = epAt P45 = f|iAt

P46 = (l-a)^At

P55 = 1 - (a + f + R)|iAt P53 = R|iAt P54 = f|iAt p56 = a|iAt

The above probabilities are presented in the following transition matrix:

Page 394: Applications of Markov Chains in Chemical Engineering

388

P =

1

2

3

4

5

6

1 2 3 4 5 6

Pll P12 P13 0 0 0

P21 P22 P23 P24 0 0

P31 P32 P33 P34 P35 0

0 P42 P43 P44 P45 P46

0 0 P53 P54 P55 P56

0 0 0 0 0 1

For the pulse input C(0) = [0, 1, 0, 0, 0, 0], the response curve for Ci to C6

was obtained from C(n+1) = C(n)P. A specific example was computed for |ii = 5,

a = 0.5, b = d = e = f=0.1,At = 0.001 where the effect of the recycle R = 0, 5 is

demonstrated in Fig.4.3-15(l). As observed, by increasing the recycle, reactors 2,

4 and 1, 3, 5 acquire sooner the same concentration.

1

0.8

0.6

0.4

0.2

0

{ 1

h\i = 2

[_ \

1 V. "^

R = 0

y^

> '

. 4 5

,,''•

: • • . . • « _ . . . - - .

1

-\

0 0.5

Fig.4.3-15(l). Ci versus t demonstrating the effect of the recycle R

The following configurations, 4.3-16 and 4.3-16(1), are multiloop circulation

models [74-76] for fitting experimental residence time distribution data of

continuous stirred vessels.

4.3-16 The following scheme is a three loop model consisting of six perfectly mixer

reactors which simulates a mixer [75].

Page 395: Applications of Markov Chains in Chemical Engineering

389

Fig.4.3-16. Three loop model with inflow to impeller

The above configuration is obtained from the general scheme in Fig.4-la by

designating a = l , b = 2, ...,Z = 6 and ^ = 7 which is the 7th state; the junction is

j . The concentration at this point, Cj, is given by Eq.(4-la).

Eq.(4-12a) gives:

1 = P57 where the reference flow is Qj in Fig.4.3-16.

Eq.(4-12b) gives:

1 + a2j + a4j + a6j = aji + aj3 + ajs (4.3-16a)

Eq.(4-12c) gives:

ocji = 0 12 for i = 1; ai2 = a2j for i = 2; OJB = a34 for i = 3;

a34 = a4j for i = 4; Ojs = P57 + a56 = 1 + a56 for i = 5;

0 56 = 0C6j for i = 6 (4.3-16b)

The following probabilities of remaining in the state are obtained from Eq.(4-

16) where m = QjA i:

Pii = 1 - ai2mAt P22 = 1 - a2j^2At

P33 = 1 - a34|X3At P44 = 1 - a4j|l4At

P55 = 1 - (P57 + CX56)|Li5At P66 = 1 " a6j^6At (4.3-16c)

The following probabiUties of leaving the state are obtained:

From Eq.(4-13c): pkj = ockj/ZkOCjk k = 2,4, 6

From Eq.(4-15): pji = OjiiLiiAt k = 1, 3, 5

From Eq.(4-17b): P57 = p57|i7At

where pi2, P34 and p56 are obtained From Eq.(4-18). The above probabilities are

arranged in a transition matrix given below which is of the kind demonstrated by

Eq.(4-13) with pjj = 0, noting those probabilities among pki and pkj which are zero

according to the scheme in Fig.4.3-16.

Page 396: Applications of Markov Chains in Chemical Engineering

390

P =

j

1

2

3

4

5

6

7

J 0

0

P'2j

0

P'4j

0

P'6j

0

1

pjl

Pll

0

0

0

0

0

0

2

0

P12

P22

0

0

0

0

0

3

Pj3

0

0

P33

0

0

0

0

4

0

0

0

P34

P44

0

0

0

5 6 7

Pj5 0 0

0 0 0

0 0 0

0 0 0

0 0 0

P55 P56 P57

0 P66 0

0 0 1 (4.3-16d)

Eqs.(4.3-16a,b) is a set of seven equations with nine unknowns, two of

which must be specified. Thus, it has been assumed that:

Ql2 = Q34 = r» or ai2 = a34 = r/Qj = R, i.e. the circulatory flow in the loop

1-2-j. Therefore, Eqs.(4.3-16a,b) yield:

otji = o 2j = 0Cj3 = a4j = R and a6j = a56 and aj5 = 1 + a6j

where it is further assumed that:

0C6j = OC56 = R thus, aj5 = 1 + R

If all reactors have the same volume V, i.e. ILLJ = |LI = QjA , the probabilities in

Eqs.(4.3-16c,d) satisfying the above matrix, read: Pii = 1 - R^At i = 1, 2, 3, 4, 6 P55 = 1 - (1 + R)jiAt

P12 = P34 = P56 = R^At

Pjl = Pj3 = [(R/(3R + l)] iAt pj5 = [(R + 1)/(3R + l)]jiAt

P'2j = P'4j = P'6j = 1/3 As seen, the solution is a function of the parameters |Li, R and At. For the

pulse input given by Eq.(4-13a), i.e. C(0) = [Cj(0), 1, 0, 0, 0, 0, 0, 0], where

Cj(0) is given by Eq.(4-la), the response curve for Ci to C7 was obtained from

C(n+1) = C(n)P. A specific example was computed for |i = 1, At = 0.005 where

the effect of the circulation R = 0.2, 1 and 5 is demonstrated in Fig.4.3-16a. As

observed, by increasing R, reactors 1-6 acquire sooner an identical concentration

and behave as a single reactor.

Page 397: Applications of Markov Chains in Chemical Engineering

391

R=l

1, '

4 0

;• v . .

Fig.4.3-16a. Ct versus t demonstrating the effect of the recycle R

4 . 3 - 1 6 ( 1 )

The following scheme is a simplified version of case 4.3-16 for Qj = 0, i.e. a

closed three loop system.

Assuming all reactors are of the same volume, that the reference flow is one

of the internal flows, and that all flows in the loops are identical, thus all aij = 1,

yields the following transition matrix:

Page 398: Applications of Markov Chains in Chemical Engineering

392

P =

j

1

2

3

4

5

6

J 0

0

1/3

0

1/3

0

1/3

1 |j.At

l-|iAt

0

0

0

0

0

2

0

|lAt

l-|iAt

0

0

0

0

3 MAt

0

0

l-\lAt

0

0

0

4

0

0

0

pAt

l-pAt

0

0

5 loAt

0

0

0

0

1-MAt

0

6 0

0

0

0

0

^At

l-HAt

For the pulse input C(0) = [Cj(0), 1, 0, 0, 0, 0, 0], where Cj(0) is given by Eq.(4-la), the response curve for Ci to €5 was obtained from C(n+1) = C(n)P. A specific example was computed for At = 0.01 where the effect of |i = 0.1, 1 is demonstrated in Fig.4.3-16(l). As observed, by increasing (i, reactors 1-6 acquire sooner an identical concentration which is equal to 1/6.

2 4 6 8 1 0 0 2 4 6 8 10 t t

Fig.4.3-16(l). Ci versus t demonstrating the effect of |X

4>3-17 This is another configuration simulating a continuous mixer with an equally

divided flow between the upper two circulation loops [75].

Page 399: Applications of Markov Chains in Chemical Engineering

'^^^LA

393

Fig.4.3-17. Three loop model with inflow divided

The derivation of the probabiUties is quite similar to the former case. For the

transition matrix given by Eq.(4.3-16d), assuming that all reactors are of the same

volume and that the reference flow is Q in Fig.4.3-17, the following expressions

were obtained:

Pii = 1 - (1/2 + R)|iiAt i = 1, 2, 3, 4

P66 = 1 - R^At

pjl = pj3 = R|lAt

P2j = P4j = (l/2 + R)/(l+3R)

Pl2 = P34 = (l/2 + R)|llAt

where the rest of the probabilities are zero. The solution is a function of the

parameters \i, R and At, and for the pulse input C(0) = [Cj(0), 1, 0, 0, 0, 0, 0, 0],

where Cj(n) is given by Eq.(4-la), the response curve for Ci to C7 was obtained by

C(n+1) = C(n)P. A specific example was computed for p, = 2, At = 0.01 where

the effect of the circulation R = 0, 1 and 5 is demonstrated in Fig.4.3-17a. As

observed, by increasing R, reactors 1-6 acquire sooner an identical concentration

and behave as a single reactor.

P55=l- ( l+R)^At

P77= 1 Pj5=(l+R)HAt

p'6j = ( l + R ) / ( l + 3 R )

P56 = RflAt P57 = |lAt

u

1

0.8

0.6

0.4

0.2

n

y 1

\ i = 1 - \

" Ao /j<'.

1

1 ^ " '""T——

1

J

A -

, ^

2 3 t

5 0

Page 400: Applications of Markov Chains in Chemical Engineering

394

o

1

0.8

0.6

0.4

0.2

0

ll

[\\2

r

1 1 R = 5

7

1 ' n^

1

1

H H

-

2 3 t

Fig.4.3-17a. Ci versus t demonstrating the effect of the recycle R

4,3-18 Another configuration simulating a continuous mixer with non divided inflow

[74] is depicted below.

1^

Fig.4.3-18. Three loop model with non divided inflow

Assuming that all reactors are of the same volume, that the reference flow in

Fig.4.3-18 is Q2, yields the following results by applying Eqs.(4-12a) to (4-12c):

Oji = Ojs = asj = asj = a i2 = R where R = Q5J/Q2 = Qjk/Q2 k = 2, 3

a2j = ocj4 = a45 = R+ 1.

The transition matrix reads:

Page 401: Applications of Markov Chains in Chemical Engineering

395

P =

j 1

2

3

4

5

6

J 0

0

P'2j

P'3j

0

P'5J 0

1

pjl

Pll

0

0

0

0

0

2 0

P12

P22

0

0

0

0

3

Pj3

0

0

P33

0

0

0

4

Pj4

0

0

0

P44

0

0

5 0

0

0

0

P45

P55

0

6 0

0

0

0

0

P56

1

where the probabilities obtained from Eqs.(4-14) to (4-19) are: pll = P33 = l-R^At pii = 1 - (1+R)nAt i = 2,4,5

Pj 1= pj3 = R^At Pj4 = (1 +R)^At

p'2j = (1 + R)/(l + 3R) p'3j = p'5j = R/(l + 3R) P12 = R lAt P45 = (1 + R)|ilAt P56 = M.At

Cases a and b in Fig.4.3-18a, demonstrate the effect of the recycle R for jl =

1 and At = 0.004. Cases b, c and d demonstrate the effect of the introduction

location of the tracer, reactors 1, 3 or 5, on the response curves for p, = 1 and R =

5.

u

0.8

0.6

0.4

0.2

n

-

z.

1

i = l

3--^ -5

' ' (a)

R= 1, Tracer in 1 H

H

^ - - ^ . _ ^ -

pj^-^T"" 0.5 1.5

Page 402: Applications of Markov Chains in Chemical Engineering

396

1

0.8

0.6

•"0.4

0.2

0

U.

1

r \ 1 \

s 1 m I * ' III! — — I I

[^tl.

1

" * " l - ^ 1

. _ _ _ J

' (0

R = 5. Tracer in3-^

A -^^"1

I 1 0.5 1.5

R = 5, Tracer in 5—1

2 0

hl .3 ,4>:

0.5 JL J .

1.5

Fig.4.3-18a. Q versus t demonstrating the effect of the recycle R

and the introduction location of the tracer in reactors 1, 3 and 5

4,3-18fl) The following configuration is a simplified version of case 4.3-18 where Qj

0, i.e. a closed three loop model.

Assuming that all reactors are of the same volume, that the reference flow is

one of the internal flows, and that all flows in the loops are identical, thus all ay =

1, yields the following transition matrix:

Page 403: Applications of Markov Chains in Chemical Engineering

397

P =

j

1

2

3

4

5

J 0

0

1/3

1/3

0

1/3

1

M,At

l-fiAt

0

0

0

0

2

0

M.At

l -^At

0

0

0

3

(iAt

0

0

l -nAt

0

0

4

MAt

0

0

0

l-M,At

0

5

0

0

0

0

^At

1-nAt

Cases a and b in F ig .4 .3-18( l ) , demonstrate the effect of fx « 0.1 and 1 for

At = 0 .01 . Cases b , c and d demonstrate the effect of the introduction location of

the tracer, reactors 1, 3 o r 5 , on the response curves for (i = 1. A s observed, by

increasing \i, reactors 1-5 acquire sooner an identical concentration which is equal

to 1/5.

u

u

1

0.8

0.6

0.4

0.2

0

1

0.8

0.6

0.4

0.2

0

L ^ ' - L - 1 . ^ ^ —I — — r—V.I

1 1 r (b)]

\i - 1,Tracer at IH

10 0 10

(d) |i • 1, Tracer at 5 -

1 . 3 ^ ' ' - l , . - •

C^^ I \ L 4 6

t 10

Fig.4.3-18(l). Ci versus t demonstrating the effect of \JL

and the introduction location of the tracer in reactors 1, 3 and 5

Page 404: Applications of Markov Chains in Chemical Engineering

398

4 .3 -19 An extension of case 4.3-18 where the central loop contains a varying

number of reactors designated by 3, 4,..., Z-2, Z > 5, is depicted in Fig.4.3-19.

iHi*"^^-dii

Fig.4.3-19. Three loop system with the central loop of variable number of reactors

From Eq.(4-12a) it is obtained that Pz = 1 where the reference flow is Q2.

From Eq.(4-12b) it follows that:

a2j + ocz-2,j + oczj = ocji + aj3 + aj,z-i where from Eq.(4-12c) it is obtained that:

For i = 1: Oji = ai2; i = 2: 1 + ai2 = aij; i = 3: Ojs = a34; i = 4: a34 = a45; ... i = i: ai-i,i = ai,i+i; ... i = Z-2: az.3,z-2 = otz-2,j; i = Z-1: aj,z-i = az-i,z; i = Z: az-i,z = azj + 1

The above set yields Z (Z > 5) independent equations and Z+3 unknowns; thus three unknowns must be fixed. For example, if Z = 6 and ^ = 7, the process is as follows: Fix an = R, thus, oji = R and a2j = 1 + R;

Fix aj3 = R, thus, a34 = a4j = R; Fix a56 = R + 1, thus, aj5 = R + 1 and a6j = R;

For the general case, the following transition matrix is applicable:

Page 405: Applications of Markov Chains in Chemical Engineering

399

j

1

2

3

4

P = i-1

i

i+1

Zr2

Z-1

Z

^

J 0

0

P'2j

0

0

0

0

0

P'z-2,j

0

P'zj

0

1

Pjl

Pll

0

0

0

0

0

0

0

0

0

0

2

0

P12

P22

0

0

0

0

0

0

0

0

0

3

Pj3

0

0

P33

0

0

0

0

0

0

0

0

4 ...

0 ...

0 ...

0 ...

P34 ...

P44 ..

0 ..

0 ..

0 ..

0 ..

0 ..

0 ..

0 ..

i-1

0

0

0

0

0

Pi-l,i

0

0

0

0

0

0

i

0

0

0

0

0

-1 Pi-1,

Pii

0

0

0

0

0

i+1 .

0

0

0

0

0

i 0

Pi,i+1 .

Pi+l,i+l .

0

0

0

0

Z-2

0

0

0

0

0

0

0

0

.. Pz-2,z-

0

0

0

Z-l

Pj,z-1

0

0

0

0

0

0

0

21 0

Pz-l.z-

0

0

z 0

0

0

0

0

0

0

0

0

lPz-l,z

Pzz

0

^

0

0

0

0

0

0

0

0

0

0

Pz^

1

specific expressions for the probabilities, obtained from Eqs.(4-15) to (4-19) are:

Pll = 1 - ai2M.lAt = 1- RmAt P22 = 1 - a2jH2At = 1- (l+R)|j,2At

P33 = 1 - a34|a3At = 1- R|X3At P44 = 1 - a45|i4At = 1- R^At

Pii = 1 - ai,i+miAt = 1- RmAt pz.2,z-2 = 1 - az-2,jHz-2At = 1- (l/3)^z.2At

Pz-l,z-l = 1 - az-i,zM z-lAt = 1- (l+R)|J,z-lAt

Pzz= 1 - (l+azj)^zAt = 1- (l+R)^lzAt

Pjl = ajimAt = RmAt

Pj,z-1 = aj,z-mz-lAt = (l+R)^iz.iAt

P12 = ai2^2At = R|X2At

P45 = a45^5At = R|X5At

Pz-l,z = Oz-Lz^z-lAt = (l+R)^z.lAt

p'2j = a2j/(cxji+aj3+aj,z-i) = (1+R)/(1+3R)

p'z-2o = az-2,j /(aji+aj3+aj,z-i) = R/(1+3R)

p'zj = ttzj /(aji+aj3+aj,z.i) = R/(1+3R)

Particular solutions were obtained for a total number of states of 7,9 and 13, in the

following. The increase in the number of states was in the central loop.

For a total number of states of 7, where the central loop contains the two

states 3 and 4, Fig 4.3-19 is reduced to:

Pj3 = aj3^3At = R^3At

P34 = a34|X4At = R|X4At

Pi,i +1 = ai,i+i|li+iAt = R|Xi+iAt

Pz^=Pz^Mt = |At

Page 406: Applications of Markov Chains in Chemical Engineering

400

The following matrix is obtained for reactors of an identical volume V, i.e. (ii =|i. =

Q2A :

j

1

2

= 3

4

5

6

7

J 0

0 1+R 1+3R

0 R

l+BR

0 R

1+3R

0

1

RM,At

1-R^At

0 1

0

0

0

0

0

2

0

R iAt

- (l+R)^At

0

0

0

0

0

3

R^At

0

0

1-R^At

0

0

0

0

4

0

0

0

R lAt

1-R^At

0

0

0

5

(l+R)^At

0

0

0

0

1- (l+R)^At

0

0

6

0

0

0

0

0

(l+R)M.At

1- (l+R)^At

0

7

0

0

0

0

0

0

M.At

1

For a total number of states of 9, where the central loop contains the four

states 3 to 6, Fig 4.3-19 is reduced to:

The following matrix is obtained:

fej^

Page 407: Applications of Markov Chains in Chemical Engineering

401

j

1

2

3

4

P = 5

6

7

8

9

J 0

0 l+R 1+3R

0

0

0 R

1+3R

0

1+3R

0

1

R iAt

1-R^At

0

0

0

0

0

0

0

0

2

0

RM-At

1-l+R)^lAt

0

0

0

0

0

0

0

3

R^lAt

0

0

l-R^iAt

0

0

0

0

0

0

4

0

0

0

RM,At

l-R|iAt

0

0

0

0

0

5

0

0

0

0

R^At

l-RjiAt

0

0

0

0

6

0

0

0

0

0

R^iAt

l-R^iAt

0

0

0

7

(l+R)^At

0

0

0

0

0

0

1-l+R)^iAt

0

0

8

0

0

0

0

0

0

0

(l+R)^At

1-l+R)^iAt

0

9

0

0

0

0

0

0

0

n

1

For a total number of states of 13, where the central loop contains the eight

states 3 to 10, Fig 4.3-19 is reduced to

irni >^[db|;^|db| ^idbl >JoDl

The following matrix is obtained while designating p = RjiAt, q = 1 - R|LiAt,

r = (l+R)M,At and s = 1 - (l+R)^At.

Page 408: Applications of Markov Chains in Chemical Engineering

402

j 1

2

3

4

5

P = 6

7

8

9

10

11

12

13

J 0

0 l+R 1+3R

0

0

0

0

0

0

0 R

1+3R

0 R

1+3R

0

1

P

q 0

0

0

0

0

0

0

0

0

0

0

0

2

0

P s

0

0

0

0

0

0

0

0

0

0

0

3

P 0

0

q 0

0

0

0

0

0

0

0

0

0

4

0

0

0

p

q 0

0

0

0

0

0

0

0

0

5

0

0

0

0

p

q 0

0

0

0

0

0

0

0

6

0

0

0

0

0

p

q 0

0

0

0

0

0

0

7

0

0

0

0

0

0

p

q 0

0

0

0

0

0

8

0

0

0

0

0

0

0

p

q 0

0

0

0

0

9

0

0

0

0

0

0

0

0

p

q 0

0

0

0

10

0

0

0

0

0

0

0

0

0

p

q

0

0

0

11

r

0

0

0

0

0

0

0

0

0

0

s

0

0

12

0

0

0

0

0

0

0

0

0

0

0

r

s

0

13

0

0

0

0

0

0

0

0

0

0

0

0 HAt

1

Fig.4-3.19 demonstrates response curves of various states (reactors) for a

pulse introduced in state 1 raising its concentration to unity. The parameters of the

graphs are the recycle rate R (= ai2 = Oji = Ojs = a34 = a4j), 0.5 and 5, and the

number of states in the central loop, 2,4 and 8 corresponding to a total number of

states 7, 9 and 13, respectively. Common data were |i = 1 and At = 0.01. The

general trends observed were that the approach towards equilibrium becomes

slower by increasing the number of states and that the streams attain faster a

uniform concentration by increasing R. The effect of the number of states is

reflected by curves 4 (case a), 6 (case c) and 10 (case e) corresponding to the exit

reactors of the central loop, as well as the overall effect reflected in curves 7 (case

b), 9 (case d) and 13 (case f) corresponding to the final collector of the tracer. The

effect of R is demonstrated in cases a and b, c and d as well as e and f.

Page 409: Applications of Markov Chains in Chemical Engineering

403

u

0

u

1

0.8

0.6

0.4

0.2

0

1

0.8

0.6

0.4

0.2

0

1

0.8

0.6

0.4

0.2

%

Fig.4

\

p-

k

k

1 1 1 1 Number of states = 2, R = 0.5

^ ,

^ \ . 7 2 .

(a)J

-]

r-

, 1 1 1 1 1 Number of states = 2, R = 5

i t\ - - ' ' ' ' h, - '::. . ^

m^

-

-

\j • ' \ \ 1 1 2 3

t 5 0 1 2 3

t

1 1 1 1 1 \ Number of states = 4, R = 0.5 (c)

; ^ "

\-

L 2

V . r t ' - ' '

-

" --. ?' ~ . _ _ _ , . ' " " " " • - . . , -

^,1 _ -1 I ^ ~

'

0 2 3 t

n 1 i \ Number of states = 4, R = 5 (d)

/V

5 0 1 I I 1 I

2 3 t

V i l l i \ Number of states = 8, R = 0,5

r ^ M

1 " "--- 13 1 2 > ^ '

[/ , - t - ' - 1 ^10 1 1

(e)

J

A _

1 \ 1 \ Number of states = 8, R = 5 (f)

12>

13.

10 / .. ' , : . j " • - 7

I -•-' I 1 2 3 4 5 0 1 2 3 4 5

t t

3-19. Ci versus t demonstrating the effect of the number of states in the central loop and the recycle R

Page 410: Applications of Markov Chains in Chemical Engineering

404

4.3-19(1) The following configuration is a simplified version of case 4.3-19 where Q2

= 0, i.e. a closed three loop model.

lilHi*-ii*-^

Assuming that all reactors are of the same volume, that the reference flow is

one of the internal flows, and that all flows in the loops are identical, i.e. all aij =

1, yields the following transition matrix for p = pAt and q = 1 - nAt:

j

1

2

3

4

P = i-1

i

i+1

7/1

Zrl

z

j

0

0

1/3

0

0

0

0

0

1/3

0

1 1/3

1

P q 0

0

0

0

0

0

0

0

0

2

0

P q 0

0

0

0

0

0

0

0

3

P 0

0

q 0

0

0

0

0

0

0

4 ..

0 ..

0 ..

0 ..

p ..

q ..

0 ..

0 .

0 .

0 .

0 .

0 .

. i-1

0

0

0

0

0

. q 0

0

0

0

.. 0

i

0

0

0

0

0

P q 0

0

0

0

i+1 ..

0 ..

0 ..

0 ..

0 ..

0 ..

0 ..

P ..

q .

0 .

0 .

0 .

Z-2

0

0

0

0

0

0

0

0

q 0

0

Z-1

p 0

0

0

0

0

0

0

0

q 0

z 0

0

0

0

0

0

0

0

0

p q

Fig.4-3.19(1) demonstrates response curves of various states (reactors) in the

closed system, computed from C(n+1) = C(n)P for a pulse introduced in state 1

raising its concentration to unity. The parameter of the graphs is the number of

Page 411: Applications of Markov Chains in Chemical Engineering

405

states in the central loop, 2, 4 and 8 corresponding to a total number of states 6, 8 and 12, respectively. Common data were \i = 1 and At = 0.01. The general trend observed is that the approach towards equilibrium, i.e. concentration of l/(number of states) becomes slower by increasing the number of states. The effect is reflected by curves 4 (case a), 6 (case b) and 10 (case c) corresponding to the exit reactors of the central loop. In addition, the approach versus time towards equilibrium in case c is slower than in case a where case b is intermediate.

u

1

0.8

0.6

0.4

0.2

0

u "1 1 1 Number of states = 2 (a)| Number of states = 4 (b)

10 0 10

U

1

0.8

0.6

0.4

0.2

0

1

\ — I

A - X^ il' c.

-^127

1 1 1 Number of states = 8

^"""--^ 10

1..- \--^"\'^

{0)1

4

-

-

4 6 t

10

Fig.4.3-19(l). Ci versus t demonstrating the effect of the number of states in the central loop

Page 412: Applications of Markov Chains in Chemical Engineering

406

4.4 PLUG FLOW-PERFECTLY MIXED REACTOR SYSTEMS

Plug flow reactors are widely used in industry. In simulation of chemical

processes, such a reactor represents, many times, a certain time delay element in the

process between two stages. A plug flow reactor is characterized by the fact that

the flow of fluid through the reactor is orderly with no element of fluid overtaking

or mixing with any other element ahead or behind. The necessary and sufficient

condition for plug flow is for the residence time in the reactor to be the same for all

elements of fluid. In the following, several flow configurations comprising of plug

flow reactors will be treated.

The treatment of the plug flow-perfectly mixed reactor systems, generally,

comprises the following steps: a) Defining the stages for the transfer process of the

tracer between the states (reactors) according to the residence times, tp, of the plug

flow reactors in the flow system, b) Establishing the transition probability matrix

for each stage, c) Determination of the distribution coefficients aij, Pij from mass

balances given by Eqs.(4-12a), (4-12b) and (4-12c). d) Determination of transition

probabilities by Eqs.(4-14) to (4-19). e) Computation of the response curves

according to C(n+1) = C(n)P. At each step ^ C j = 1 must hold for all states for a

unit concentration pulse.

In the following, a variety of configurations will be treated, which find

importance in practice and in simulation, as well as elaborate the application of the

model in chapter 4.1.

4>4-l The basic scheme of a single plug flow reactor with recycle is shown below

in Fig.4.4-1. Such a reactor may be described also as a combination of perfectly

mixed reactors. Other simplified configurations are described in the following

cases, 4.4-1(1) and 4.4-1(2). The states are the concentration of the tracer in the

perfectly mixed reactors, i.e. SS = [Ci, C2, ..., C^ ]. The residence time in the

plug flow reactor is tp = V/Qi where V is the volume of the reactor. The recycle

stream is Qz2-

Page 413: Applications of Markov Chains in Chemical Engineering

407

ifrsiitzitti I z-ll z L 3 VZ2

Fig.4.4-1. Basic scheme of a plug flow reactor composed of perfectly mixed reactors with recycle

Referring to Fig.4-1, yields for the above configuration that: Oj = ai (i = 2,

3,...) = ay = ttji = 0, i.e., reactor j is not considered; hence, a = 1, b = 2,... and ^

= Z+1.

From Eq.(4-12a) follows that pz^ = 1

From Eq.(4-12c) follows: for i = 1 ai = an = 1

fori = 2 ai2 + az2 = 0C23 or alternatively

1 + R = a23 where az2 = Qz2/Ql = R for i = 3 a23 = a34 = 1 + R for i = i ai-i,i = ai,i+i = 1 + R

fori = Z az.i,z = Pz + azi= 1+R

The corresponding probabilities are obtained from Eqs.(4-16), (4-18), (4-18a) and

(4-19) yielding the following probability matrix:

1

2

3

P = i

i+1

Z-1

Z

%

where

P22 =

Pii =

1 l-^ijAt

0

0

0

0

0

0

1 0

2 ijAt

P22

0

0

0

0

R ljAt

0

1 - (l+R)^2At

1 - (l+R)mAt

3

0

P23 P33

0

0

0

0

0

. . .

. . .

. . .

0

0

. . .

P23

i

0

0

0

Pii

0

0

0

0

i+1

0

0

0

0

Pi,i+1

Pi+l,i+l

0

0

0

0

= (l+R)|Ll3At

Pi,i+1 = (1+R)W+

. . .

. . .

. . .

lAt

z-1 0

0

0

0

0

Pz-l,z-l

0

0

z 0

0

0

0

0

Pz-l.z

Pzz

0

^

0

0

0

0

0

0

Pz

1

Page 414: Applications of Markov Chains in Chemical Engineering

408

Pz-i,z-i = 1 - (l+R)^z-iAt pz-1,2 = (l+R)Mt Pzz = 1 - (l+R)|LizAt pz = iLi At

The parameters of the solution are: i = QiA i (i = 1,.., Z+1) and R. If all reactors

have the same volume, |ii = (Z-l)|i = (Z-l)/tp where [i = QiA^p; Vp is the volume

of the plug flow reactor and tp is the mean residence time of the fluid in the reactor.

In the numerical solution, the plug flow reactor was divided into ten perfectly

mixed reactors of identical volumes, i.e. Z = 11, ^ = 12 and i = \i. The transient

response of Q (i = 1, 2, 5, 8, 11 and 12) for C(0) = [1, ..., 0] is depicted in

Fig.4.4-1, cases a to d. For cases e and f, C(0) = [0, 1,..., 0], i.e. the tracer was

introduced into reactor 2. The effect of the following parameters was explored for

cases a to d: R = 0 and 10, ^ = 10,50 and 100 where At = 0.0005 for the large m

and 0.004 for the smaller ones. The quantities R = 0, 5, |Lii = 500 and At =

0.00004 were applied in cases e and f.

The effect of an identical value of m = 10 and 50 is demonstrated in cases a

and b. As seen the curves attain faster the steady state for |LLi = 50. The effect of

the recycle R is demonstrated in cases c and d, noting that for R = 10 all transient

responses for reactors i = 2,..., 11 are almost identical. This is because the plug

flow reactor behaves as a single perfectly mixed reactor. Note that in the above

cases the tracer was introduced into reactor 1. In cases e and f the tracer was

introduced into reactor 2, i.e. directly into the entrance of the plug flow reactor.

The response at the exit of the reactor is that of reactor 11, as shown. For the case

of R = 0, a maximum in the response is observed for reactor 11 after about 0.02

time units. Indeed, this figure is the mean residence time of the plug flow reactor

noting that tp = (number of perfectly mixed reactors)(l/p,i) = 10(1/500) = 0.02.

When the recycle is R = 5, as seen in case f, the behavior of the plug flow reactor

becomes perfectly mixed.

Page 415: Applications of Markov Chains in Chemical Engineering

409 1

1 - 1

I

8

1 1

1 2 / '

- - -f 1 (b)

^.= 50 a = i....,i2) R = 0 H

' - - I 2 0 0.1 0.2 0.3

t 0.4 0.5

U

1

0.8

0.6

0.4

0.2

0 I

1

0.8

0.6

*0.4

0.2

0

\ . 1

f\2

t K\\

b ; / ' •

1

11

• •

- 1 V:

1

/""

1 12.

^ i

(i = R =

>. ' * ^ - ^

. . . I . . .

= ^^12=

= 100 = 2, ..., :0

* * - L : : ^

m

10 -

1 1 ) -

-

—•»>'»-..:„m

\ . '

t • I 1

1

^ - 1 1

1

1 1 12. (d)

\i. = 100 (i = 2, ..., l l ) -R=10

0.1 0.2 0.3 t

0.4 0.5 0

[

f V -r 1 1

(e)

1.= 500 ~i

(i = 2, ..., 11)J R = 0

1 0.01 0.02

t

0 0.1

L

h

1 1

0.2 0.3 0.4 0 t

(Ol

\^r 5 0 1 0 = 2, ...,11) J R = 5

1 1 0.03 0.04 0 0.01 0.02

t 0.03 0.04

Fig.4.4-1. Ci versus t demonstrating the effect of the recycle R and

4>4-l(l) A simplified approach for treating a plug flow reactor by comparison to case

4.4-1 above, which avoids the division of the reactor into perfectly mixed reactors, is demonstrated in Fig.4.4-l(l). The states are the concentration of the tracer in the two perfectly mixed reactors 1 and 2, at the inlet and exit of the plug flow reactor. The residence time in the plug flow reactor is tp = Vp/Qi where Vp is the volume of

Page 416: Applications of Markov Chains in Chemical Engineering

410

the reactor. This case simulates the situation demonstrated in ref.[81], designated

as 'partial mixing and piston flow'.

3i^ 12 f o b ^ = 2|

Fig.4.4-l(l). A simplified scheme of a plug flow reactor

In establishing the probability matrix, distinguish is made between two stages

in the transfer process of the tracer from one state to the other, i.e., 0 < t < tp and

t > tp. For 0 < t < tp, Ci = Ci(0) and C2 = C2(0) whereas for t > tp, the following

matrix is apphcable:

P = 1 2

1 2

Pll P12

0 1

The probabilities are determined as follows ignoring the presence of the

plug flow reactor. Referring to Fig.4-1, yields for the configuration in Fig.4.4-

1(1) that: Qj = Qi (i = 2, 3,...) = Qy = Qji = Oor aj = ai (i = 2, 3,...) = ay = aji

= 0, i.e., vessel j is not considered, hence, a = 1 and ^ = 2. From Eq.(4-12a)

follows that a i = Qi/Qi = P12 = qi2/Ql = 1- Applying Eqs.(4-16) and (4-17b),

noting the above results, yields pn = 1 - [XiAt pi2 = |X2At where m = QiA^i, |Li2

= Ql A^2 according to Eq.(4-7). The parameters of the solution are m , |L12 and tp.

In the numerical solution it has been assumed that the reactors are of the same

volume, thus, m = |i2 = |Li. The transient response of Ci and C2, for C(0) = [1,0]

while C(n+1) = C(n)P is depicted in Fig.4.4-l(la) where the effect of i = QiA^i =

10,20 and 200 is demonstrated for tp = 0.1 and At = 0.001. As seen, increasing

|i (or decreasing the mean residence time in the reactor) brings the reactors faster to

a steady state. The results for \i = 200 are interesting showing an immediate

response of the reactors. For constant Qi, a large |LI indicates that the volimie of the

reactor is very small, i.e. the mean residence time of the fluid in it is extremely

short. Thus, the case for |Li = 200 simulates a situation where reactors 1 and 2 are

Page 417: Applications of Markov Chains in Chemical Engineering

411 reduced to 'points', usually for sampling, or the location of some measurement

device.

u

1

0.8

0.6

0.4

0.2

0

-

-

-

1 i = l

2 _ 1

1

/

1

1 1 ^l= loH

-

V "^

-

1

0.05 0.1 t

0.15 0.2 0

U

1

0.8

0.6

0.4

0.2

0

1 1 i = l !

-

-

-2 1

/ /

i 1 I \

I

1 J

-J J

i = 200

-

1

0.05 0.1 t

0.15 0.2

Fig«4.4-l(la). Ci versus t demonstrating the effect of |Li

4 . 4 - 1 ( 2 )

An extension of case 4.4-1(1) is demonstrated in Fig.4.4-1(2). The scheme

comprises of a plug flow reactor, a feeding reactor 1 and a collector 4, as before.

In addition, measurement points of the concentration, simulated by 'small'

perfectly mixed reactors 2 and 3, were added. The residence time of the fluid in the

plug flow reactor is tp. These reactors make it possible to include the recycle

stream Q32, impossible to add in case 4.4-1(1). Reactors 2 and 3 simulate also

reactors 2 and 11 in example 4.4-1.

Page 418: Applications of Markov Chains in Chemical Engineering

412

*

32

Fig.4.4-1(2). Plug flow reactor with concentration measurement 'points* 2 and 3

The two stages in the transfer process of a tracer between the states are

expressed in the following transition matrices:

0 < t <tp: t > t n

P =

P =

1

2

3

4

1

Pll 0

0

0

2

P12

P22

P32 0

3

0

P23

P33 0

1 2 1 2 3 4

1 I Pll P12 1 Pll P12 0 0 2 1 0 1 1 2 I 0 P22 P23 0

P34

1

(4.4-1(2))

Referring to Fig.4-1, yields for the above configuration that: Oj = ai (i = 2,

3,...) = ay = ttji = 0, i.e., reactor j is not considered; hence, a = 1, b = 2,... and ^

= 4.

From Eq.(4-12a) follows that P34 = 1

From Eq.(4-12c) follows:

for i = 1 a i = ai2 = 1

fori = 2 ai2 + (X32 = a23 or altematively

1 + R = a23 where a32 = Q32/Q1 = R is the recycle

for i = 3 a23 = P34 + a32 = 1 + R

The probabilities corresponding to the matrix given by Eq.(4.4-1(2)) are

obtamed from Eqs.(4-16), (4-17b), and (4-18a); they read:

pil = l - | I lAt pi2 = |Ll2At

P22 = 1 - (l+R)|Ll2At P23 = (l+R)|l3At

P33 = 1 - (l+R)|l3At P32 = R|l2At P34 = |l4At

Page 419: Applications of Markov Chains in Chemical Engineering

413

|Lii = Qi/Vi, i.e. the reactors are of different volumes. The parameters of the

solution are: |Xi (i = 1,..., 4), the recycle R and the residence time tp in the plug

flow reactor the effect of which is depicted in Fig4.4-1(2) for C(0) = [1,0, 0, 0]

and At = 0.004.

Cases a and b demonstrate the effect of the residence time tp in the plug

flow reactor, 0.2 and 1. As seen, in case a tp was too short for the pulse

introduced in reactor 1 to reach its maximum value of unity. The effect of the

recycle, R = 0 and 10 is demonstrated in cases a and c. As observed, the

concentration of the tracer in reactors 2 and 3 becomes identical at tp = 0.2; the

concentration are different in the absence of a recycle. The effect of |Lli is

demonstrated in cases a, d and e. In case a and d all |ii = Qi A i = are equal; in case

a |li = 10 (i = 1, ..., 4) and in d |ii = 100 (i = 1, ..., 4). Note that a large |ii

indicates a smaller volume of reactor for a constant Qi. Thus, it is clearly

demonstrated that the system attains faster its steady state values for m = 100.

Case e demonstrates the effect of large ^i =100 for reactors 2 and 3, by

comparison to the behavior of reactors 1 and 4 for which m = 10. It is observed

that the response of reactors 2, 3 is faster than the response of reactors 1 and 4. It

should be noted that if the tracer is introduced into reactor 2, the behavior is

similar, however the concentration in reactor 2 remains constant and begins to

change only after tp time units have passed.

1.2,

1

0.8 I

0.4

0.2

0

IA u I' 1'/--

4 , ' /

/ /

/ /

— 1 ^ r ^

a = 1,... R = 0 t =0.2

p

{•-::=:;....-T4 .

(a)|

.4)-j

(b)|

\ ^

I \ --^1

\ / (i=l,...,4)J

\ ; R=o J A/\ t =1 '

3 A\ P

/, ^ s , 0.2 0.4 0.6

t 0.8 0.5 1.5

Page 420: Applications of Markov Chains in Chemical Engineering

414

Q / f

i

1 v^ 1

I 4 ^ 1 / 1 1 \ 1 1 . 1 / 11

i li V

(d)l

^.= 100 "i

(i = l 4)-J

R = 0 J t =0.2 1 p

- 1 0.2 0.4 0.6 O.J

t 10,

1 0 0.1 0.2 t

0.3 0.4

(e)

^ = 1= 10 ^^=^^=100-^ R = 0

Fig.4

0.1 0.2 0.3 0.4 t

4-1(2) . Ci versus t demonstrating the effect of R, tp and m

4,4-1(3) A simplification of case 4.4-1(2) is demonstrated in Fig.4.4-1(3) simulating

the configuration in ref.[80, p.761]. Reactor 2 was added to take into account the

recycle stream Q12. The response of this reactor can be controlled by the

magnitude of ^2- If reactor 2 is a measurement 'point' of the concentration, i.e. a

very small reactor, |LL2 should be assigned a relatively large value.

Fig.4.4-1(3). A plug flow reactor with a closed loop

Page 421: Applications of Markov Chains in Chemical Engineering

415 The following transition matrices are applicable:

0 < t <tp:

P =

1

Pll 0

3

P13

1

t > t „

P = 1

2

3

1

PU

P21 0

2

Pl2

P22 0

3

P13 0

1

Applying Eqs.(4-12a) to (4-12c) gives that an = (X2h Pl3 = 1- Eqs.(4-14)

to (4-17a) yield the following probabilities:

For 0 < t < tp: pn = 1 - |LliAt pis = iiisAt

For t > tp: Pll = 1 - (l+ai2)|XiAt pi2 = ai2^2At pi3 = isAt

P22 = 1 - ai2H2At P21 = ai2mAt

The parameters of the solution are: |ii (i = 1,..., 3), the recycle R = ai2 and

the residence time tp in the plug flow reactor. The effect of R = 0 and 5 is depicted

in Fig4.4-l(3a), cases a and b, for C(0) = [1, 0, 0, 0], tp = 0.5 and At = 0.005. m

= 2 has been assumed for all reactors. The effect of tp = 0.5 and 2, is demonstrated

in cases b and c. It is observed that the concentration in reactor 1 diminishes before

the tracer has reached reactor 2 due to the long residence time in the plug flow

reactor.

1

0.8

0.6

0.4

0.2

0

- • . i = l

' ^

, '3

.

(a)

R = 0,t =0.5 p J

1 i i 1 0.5

-M

^

1 '^ /•—..

1

' 3

1

- - - • • ' * ' (b)^

R = 5,t =0.5 p J

-

1 1 1 1.5

t 2.5 0 0.5 1 1.5

t 2.5

Page 422: Applications of Markov Chains in Chemical Engineering

416

u

1

0.8

0.6

0.4

0.2

0

- M

h -' ''

2

^ ' 3

' .,

ic)\

R = 5,t = 2 p J

1 1 i 1 1 0 0.5 1 1.5 2 2.5

t Fig.4.4-l(3a). Cf versus t demonstrating the effect of R and tp

4,4-2 Fig.4.4-2 demonstrates a plug flow reactor containing a "dead water" element

of volume Vd where the active part is of volume Vp. The system contains also two

perfectly mixed reactors 1 and 2. A tracer in a form of a pulse is introduced into

reactor 1 and is transferred by the flow Qi into reactor 2 where it accumulates.

SuJ* ^^

iliii:-'::!-. ^ .: —n ^ r

Fig.4.4-2. Plug flow reactor with a **dead water" element

The following definitions were made:

V = — % (l<V<c<.) 11 = Vp + Vd

yielding that

V Qi ~ Jiv , = ^ = ^

This case is similar to case 4.4-1(1). For 0 < t < tp, Ci = Ci(0) and C2

C2(0) whereas for t > tp, the following matrix holds:

Page 423: Applications of Markov Chains in Chemical Engineering

417

P =

1 1 - |liAt

0

2

1

where |Xi = QiA^i, i = 1,2. The parameters of the solution are: |ii, \i and v where

C(0) = [1, 0].

Fig.4.4-2a demonstrates results for v = 1 and 10 corresponding to tp = 1

and 0.1, receptively, while |X = 1. Other data are: (li = fi2 = 10 and At = 0.01 for v

= 1 and 0.001 for v = 10. It should be noted that by increasing v, the effective

volume of the reactor is decreased for a constant value of Qi, hence tp is decreased.

This is clearly reflected in the figure below.

u

1

0.8

0.6

0.4

0.2

0

_

-

i = l

/ 1 1

x"""" 2 / ^

\l J

k " •,

1

-

-

\ /

i 1

1 1

2 ^^

"--..,

1

^- ^

v= 10

-

1 0.4 0.5 0 1 2 0 0.1 0.2 0.3

t t Fig4.4-2a. Cf versus t demonstrating the effect of v

4,4-3 The flow configuration in Fig.4.4-3 comprises of two perfectly-mixed

reactors of volumes Vi and V2 and a plug flow reactor of volume Vp. The tracer is introduced into reactor 1 and is transferred by the flow Qi into reactor 2; Q - Qi is the by-pass stream. The tracer is accumulated in reactor 2, while flow Qi is leaving the reactor.

^Hi Q-Qi

Fig.4.4-3. Plug flow reactor reactors with a by-pass

Page 424: Applications of Markov Chains in Chemical Engineering

418

The following definitions were made:

q = - ^ ( l<q<oo) 1^ = —

yielding that

This case is similar to case 4.4-1(1) yielding the following matrix for t > tp:

2

1 2 1 - |LiiAt |i2At

0 1

where for 0 < t < tp, Ci = Ci(0) and C2 = C2(0). |ii = Qi/Vi, i = 1, 2. The

parameters of the solution are: m, \i and q where C(0) = [1,0].

Cases a and b in Fig.4.4-3a demonstrate the effect of q. By increasing q,

i.e. the by-pass stream, the mean residence time in the tubular reactor tp is

increased from 0.1 to 0.5 time units. Case a and c demonstrate the effect of \i\ =

|X2. By increasing this quantity from 10 to 500, the mean residence time in the

perfectly mixed reactor is decreased, and the response becomes instantaneous. In

the computations, At = 0.002 for q = 1 and 0.01 for q = 5.

1

0.8

0.6

0.4

0.2

0

1

-

0

i = l \

'\

/

/ / 1

0.1

^

'\

^'.^

1 1 0.2 0.3

t

2 _^ M (a)!

n = n=ioH

q=l H

-

! 0.4 0 0.5 0

k

H

1 /^

'. /

f •

/ \ /

^ ' (b)]

u = u= 10 H ^ = 10 q = 5 i

1 1 1 0.5 1.5

Page 425: Applications of Markov Chains in Chemical Engineering

419

u

1

0.8

0.6

0.4

0.2

0

i = l :

-

-

2 J

|X = H = 500 H

' ^ = 1 0

-

1 1 1 1 0.1 0.2 0.3 0.4 0.5

t

Fig4.4-3a. Cf versus t demonstrating the effect of q, |LII and ^2

4,4-4 The following system comprises two plug flow reactors, perfectly mixed

reactors and recycle streams Q45, Q53 and Q52. Reactors 2, 4 and 5 may be

considered also as measurement points of the concentration, and in this case their

|ii's are assigned a large value, say, 500.

ribuj^ 'PI *

Q, '52

'P2 lobuJ^ %=i

h %

Fig.4.4-4. Two plug flow reactors in series with recycle

Noting that the residence time in each plug flow reactor is tpi and tp2 yields the

following matrices:

0 < t <tpi:

P =

1 2

Pll P12 0 1

tpl<t < tp2: 1

1

P = 2

3

Pll 0

0

2

P12

P22 0

3

0

P23 1

Page 426: Applications of Markov Chains in Chemical Engineering

420

tp2^t:

P =

1

2

3

4

5

6

1

Pll 0

0

0

0

0

2

P12

P22 0

0

P52 0

3

0

P23

P33 0

P53 0

4

0

0

P34

P44 0

0

5

0

0

0

P45

P55 0

6

0

0

0

P46 0

1

Referring to Fig.4-1, yields for the configuration in Fig.4.4-4 the following

probabilities by considering reactor j (hence j = 1, a = 2,... and ^ = 6) as well as

Eqs.(4-14) to (4-19) and taking Qi as reference flow:

P12 = ai2H2At

P23 = a23H3At

P34 = a34|X4At

P45 = a45^5At

P52 = a52 l2At

pii = l-ai2|i.iAt

P22 = 1 - a23^2At

P33 = 1 - a34^3At

P44 = 1 - (p46 + a45)|X4At

P55 = 1 - (a52 + a53)|X5At

The balances given by Eqs.(4-12a) to (4-12c) yield: ai2 = p46 = 1 a52 + 1 = a23 for i = 2 a23 + 0153 = 0034 for i = 3 a34 = 045 + 1 for i = 4 045 = 052 + a53 for i = 5

Case a: a53 = 0, yields the following matrices designating 0145 = a^i = R:

P46= P46^6At

P53 = a53^3At

0 < t <tpi: tpi<t <tp2:

P =

1 2 l-JljAt 2^t

0 1 P =

1

2

1 2 3 l-H,At H2^t 0

0 1. (l+R)H3At

(l+R)H2At

1 (4.4-4)

Page 427: Applications of Markov Chains in Chemical Engineering

421

tp2^t:

P =

1

2

3

4

6

1 l"^iAt

0

0

0

1 0

2 3 M 2 t 0

1. (l+R)^i3At

(l+R)Mt

0 1-(l+R)|ii3At

R^ljAt Q

0 0

4

0

0

(l+R)|LL4At

1-(l+R)Mt

0

6

0

0

0

^ 6

1

The parameters of the solution are: |ii, |L12» 3» 4* l 6i tpi and tp2, R and the

introduction location of the pulse. In cases a, b, c the pulse is introduced into

reactor 1; in cases d and e it is introduced into reactor 4. The effect of the above

parameters is demonstrated in Fig.4.4-4a. Cases a and b as well as d and e

demonstrate the effect of the recycle R = 0 and 10. As observed, increasing of R

causes the overall system to behave as a single perfectly mixed reactor. In case d,

where the pulse is introduced in reactor 4, reactors 2 and 3 are initially inactive;

however, in the presence of the recycle they become active. The effect of |i is

depicted in case c where |ii = 1x5 = 1 and |Li2 = ^3 = ^4 = 25. The effect of tp, i.e.

tpi = 1, for which At = 0.005, and tp2 = 2, for which At = 0.01, is demonstrated

in all cases by the beginning and termination of each response curve.

C(0) = [1,0,0,0,0] C(0) = [1,0,0,0,0]

Page 428: Applications of Markov Chains in Chemical Engineering

422

25

20 U

C(0) = [1.0.0.0.0]

(c) _

3.

M ' I '

1.2,4,6 , ^,

^ - |Ll « 1

li « li » u - 25 ^2 'a U R = 0

1-4

U

1

0.8

0.6

0.4

0.2

0

i = 4

2.3

C(0) =

1

\ • \ .'

' 1

L

[0.0.0.1.0]

(d)

* i . i^

6

R =

- iH = 0 J

A

i - - J - 1 — 1 1 — 1 1 1 • 1 1

r ^ h-

p

h-

2,3 1 1

(

2,3

i 1

::(0) = [0,0,0.1.0]

(e)

6. - '

, , '

1 •^^*'"*-' ...^.23^

I 1 - i 1 —

H

-\

^ - i H R=loJ

^ 1 5 t

10 0 5 t

10

Fig4.4-4a. Ci versus t demonstrating the effect of R, i and the

introduction location of the pulse for tpi = 1 and tp2 = 2

Case b: 052 = R, yields the following matrix while the matrices given by

Eq.(4.4^) are applicable also here:

Page 429: Applications of Markov Chains in Chemical Engineering

423

tpi^t:

P =

1

2

3

4

5

6

1 1-jiiAt

0

0

0

0

0

2 Ji2At

1-(l+R)JA2^t

0

0

Rpi2^t

0

3

0 (l+R)|i3At

1-

0

RjAsAt

0

4

0

0

(l+2R)M'4At

1-(l+2R)|i4At

0

0

5

0

0

0

2R^5At

1- 2R^5At

0

6

0

0

0

*6

0

1

where the parameters of the solution were spelled above.

The effect of the parameters is demonstrated in Fig.4.4-4b for C(0) =

[0,0,0,0,1,0], i.e., the pulse was introduced into reactor 5. tpi = 1, for which At

= 0.005, and tp2 = 4, for which At = 0.02. Cases a, b and c demonstrate the

effect of the recycle R = 0,0.5 and 10. In case a, all reactors are inactive since R =

0. However, increasing of R causes the entire system to ^proach the behavior of

a single perfectly mixed reactor. The effect of \i is demonstrated in case d, which

can be compared to case b, for m = |J16 = 1 and pi2 = ^3 = M4 = M5 = 10- As

observed, the approach of the system towards equilibrium is much faster.

u

1

0.8

0,6

0.4

0.2

0

^ _ .(a) 1 i = 5

1,23,4,6 1 "'" "l

~J

R = oJ

-J

——^~\

10 15 0

Page 430: Applications of Markov Chains in Chemical Engineering

424

1

0.8 I

0.6

'0.4

0.2

0

u.

r 5

U

k

h" 1 1.2,3^4,6 _ 1

(c)

•'*'''**,,^2^.4,5

1

^ "1

^-l1 R=10^

-j "^ - -=\

r 5

k

L [ 1,2,3.4,6

1

1 1

1

1 1

^ ( 4 ) , 6 1

j i . - 1 0 1 (1=2.3.4.5) J

R = 0.5 1

1,2.3.4,5 L ... 1 .,., J 1

0 5 10 15 0 5 10 15 t t

Fig4.4-4b. Q versus t demonstrating the effect of R and (li for

tpi = 1 and tp2 = 4

Page 431: Applications of Markov Chains in Chemical Engineering

425

4.4-5 The following system comprises two plug flow reactors and two perfectly

mixed reactors designated by 1 and N+1. The latter may be considered also as measurement points of the concentration. This system has been used to simulate an airlift reactor [79].

TITITI |ob|cb|cb| N-11 N I

fe - ^ N+1

cb|cb|cb| Z |cb|cb|cb| Z irhirhi^ I z I Z-llz-2l i i+li t I i-1 I lN+3N+2r^

Fig.4.4-5. Two plug flow reactors in a closed loop

The need for conservation of mass of the tracer at each step in the calculation of the closed loop, prohibited the use of the simplified approach applied in cases 4.4-1(1) to 4.4-4, i.e. distinguishing between the states according to the residence time in the plug flow reactors, tpi and tp2. The latter approach is useful in open systems. Thus, each of the plug flow reactors was divided to perfectly mixed reactors of equal or unequal volumes. The scheme in Fig.4.4-5 can be deduced from the general model demonstrated in Fig.4-1 by ignoring reactor j , noting that ai (i = 2, 3, ..., Z) = 0 where ay = aji = 0 and ai^i+i = 1 while taking Q as reference flow. Thus, the designation of the reactors in Fig.4.4-5 is a = 1, b = 2, etc. and the following matrix is applicable for the process:

Page 432: Applications of Markov Chains in Chemical Engineering

426

1 2

i

i+1

N

N+1

P = N+2

N+3

J

J+1

Z-1

z

1 Pll

0

0

0

0

0

0

0

0

0

0

Pzl

2 3 Pl2 0 P22 P23

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

i+1 0 0

ii Pi, i+1

0 Pi+l, i+1

N 0 0

0

0

0 PNN

0

0

0

0

0

0 0

0

N+1 N+2 N+3 .. 0 0

0

0

0 PN, N+1

P N + 1 , N+1

0

0

0

0

0 0

0

0 0

0

0

0 0

P N + 1 N+2

P N + 2 N+2

0

0

0

0 0

0

0 .. 0 ..

0 ..

0 .

0 . 0 .

0 .

P N + 2 , . N+3

P N + 3 , . N+3

0 .

0 .

0 . 0 .

0 .

• j . 0 . 0

. 0

. 0

.. 0 . 0

. 0

.. 0

.. 0

:: pjj

. . 0

.. 0

. . 0

.. 0

j+1 . 0 . 0 .

0 .

0 .

0 . 0 .

0 .

0 .

0 .

j+1 pj+1 , . j+1

0 . 0 .

0 .

.. z-1

. . 0

.. 0

.. 0

. . 0

.. 0 .. 0

.. 0

.. 0

.. 0

.. 0

.. 0

', ..Pz-1

z-1 .. 0

z 0 0

0

0

0 0

0

0

0

0

0

Pz-1 z

Pzz

0 0

0 0

0 0 0 0

0 0

The corresponding probabilities are obtained from Eqs.(4-16) and (4-17a)

which read:

Pi i=l -mAt ( i = l , 2 , ...,Z) Pi,i+i = li+iAt (i = 1, 2, ..., Z-1)

where |ii = QA i. For prescribed values of tpi, tp2 and n, the number of the

perfectly mixed reactors comprising the plug flow reactor. At = tpi/n.

A numerical solution was obtained by dividing each plug flow reactor into ten

perfectly mixed reactors, i.e. Z = 22 in Fig.4.4-5. It was also assumed that tpi =

tp2 = 0.01, |ii = |i = 80 and 800 for all reactors where At = tpi/10 = 0.001. Thus,

the above probabilities are reduced to:

Pii = 1 - ^At i = 1, 22 pi,i+i = lAt (i = 1,..., 21) p22a = ^At

Fig.4.4-5a demonstrates the effect of |LL on the transient response of C2 and Cn,

i.e. the first and the last perfectly mixed reactors in the upper plug flow reactor in

Fig.4.4-5. The tracer was introduced into reactor 2. It is obsereved that by

increasing |X from 80 to 800, the number of oscillations for reaching the steady state

concentration 1/22 in the system is increased. It should also be noted that for i =

800, the distance between two successive peaks corresponding to C2 and Cn is

Page 433: Applications of Markov Chains in Chemical Engineering

427

approximately equal to tp = 0.01. It will approach tp by increasing the number of reactors comprising the plug flow reactor.

u

1

0.8

0.6

0.4

0.2

0

-

i ii = 2

h- \

'-^^'-^ 1

11

1 1

1 = 80

n - ' '

1 1

A

-

-

> - ^ * - ™ .

0.1 0.2 0.3 0.4 0.5

Fig4.4-5a. Ci versus t demonstrating the effect of \i

Fig.4.4-5b shows the relationship between C2 and Cn at identical times. The graph demonstrates the approach towards the ultimate concentration 1/22 in all reactors which is clearly observed for |LI = 800.

Page 434: Applications of Markov Chains in Chemical Engineering

428

0.12 I-

0.08

U 0.04 k

u

0.15

:: 0.1

0.05

0

1 = 800

^

•f- / - ' " • ' • ' - .

^•-•••^^S^^te^^..

H

-J

H

1 1 1 0.05 0.1 0.15

Fig4.4-5b. Cii versus C2 demonstrating the effect of |x

4,4-5(1) A simpler solution for the configuration in Fig.4.4-5, avoiding the division of

the plug flow reactor to perfectly mixed reactors, is given in the following for the

scheme in Fig.4.4-5(1). Q

^

pi

Tb ip2

Fig.4.4-5(1). Two plug flow reactors in a closed loop

Page 435: Applications of Markov Chains in Chemical Engineering

429

Three cases will be treated below, depending on the magnitudes of tpi and tp2.

Case a: tpi = tp2 = tp

The stages in the transfer process of the tracer between the states, reactors 1

and 2, are expressed by the following transition matrices for which pii = 1 - flAt i

= 1, 2; pi2 = P21 = M-At where M- = Hi = (12:

0 < t <tp:

P =

1

1

0

2

0

1

tp<t <2tp:

P =

1 2

Pll P12 0 1

2tp<t <3tp:

P =

1 2

1 0

P21 P22

3tp<t <4tp:

1

P = Pll P12 0 1

4tp<t <5tp:

1

P =

2

1 0

P21 P22

5tp<t <6tp:

1

P = Pll P12 0 1

6tp<t <7tp:

1

P = Pll 0

P12 1

7tp<t <8tp:

1 2

1 0

P21 P22

The parameters of the solution are: tp, (ii and Ji2- A numerical solution was

obtained for |X = QA' = m = |i2. iO, 20, 100 and 1000, i.e. for reactors of an

identical volume V. The residence time in the reactor tp = 0.01, 0.1 and 1

corresponding to At = 0.0001, 0.001 and 0.01, respectively. The tracer was

introduced into reactor 1. Fig.4.4-5(la) demonstrates in cases a to d the effect of H

and in cases c and d the effect of tp. Note in cases c and d that an identical behavior

was obtained for different combinations of |i and tp.

Page 436: Applications of Markov Chains in Chemical Engineering

430

1

0.8

0.6

0.4

0.2

0

-

-

sj = l

'''i' 1

^ = 10 t = 0.01 p

, • - •'" '

1 1

•(a)]

-H

\ - - ^

. • --^

-

-

0.05 0.1 t

0.15 0.2 0

^1=100 t =0.01 i = 1000 t = 0.01

10 1

u U

n

•.:i;

n

y

'•.n

y

m

mm iU 0.2 0 0.05 0.1

t 0.15 0.2

Fig4.4-5(la). Ci versus t demonstrating the effect of jii and tp

Case a: tpi > tp2

The following transition matrices apply for which ph = 1 - |iAt i = 1, 2; pi2

= p2i = |xAt where |X = |ii = |X2-

0 < t <tpi:

P =

1 2

1 0

0 1

tp l<t < tpl+tpi: 1 2

1

1 ^= 2 Pll P12 0 1

tpi+tp2<t <2tpi+tp2:

1 2

1

P = 2

1 0

P21 P22 1

2tpi+tp2<t <2tpi+2tp2: 2tpi+2tp2<t <2tpi+3tp2: 2tpi+3tp2<t <3tpi+3tp2:

1 2 1 2 1 2

1 P =

Pll P12 0 1

1 P =

1 0

P21 P22

1 P =

Pll P12 0 1

Page 437: Applications of Markov Chains in Chemical Engineering

431

3tpi+3tp2 , < t < 3tpi+4tp2:

1 2

1

P = 2 Pll P12 0 1

3tpi+4tp2 < t <4tpi+4tp2

1 2

1

P = 2

1 0

P21 P22

The parameters of the solution are: tpi, tp2, |Xi and |Li2- A numerical solution

was obtained for |x = m = |i2, 1, 10 and 10, where |Li = QA , i.e. for reactors of an

identical volume V. In addition |Xi = 1, 2, 100 and |X2 = 1, 2, 5,10. The residence

time in the reactors was tpi = 1 and tp2 = 0, 0.05, 0.5 and 1 corresponding to At =

0.0005, 0.005 and 0.01, respectively. The tracer was introduced into reactor 1.

The following behaviors are demonstrated in Fig.4.4-5(lb):the effect of tp2 is

demonstrated in cases a to d; the effect of |LI in cases b, e and f; the effect of |Li2 in

cases c, g and h; the effect of |ii in cases c, i, j and k.

t = 1 t = 0 ^1=1 _El P2.

t = 1 t = 0.05 II = 1 pi P2_

8 10

Page 438: Applications of Markov Chains in Chemical Engineering

432

1

0.8

0.6

0.4

0.2

0

i = l

t = pi

1 t I

= 0.05 >2

l = 10

2. _ _. .. . , ^ M

1

I L I I,, } »

1 1 I ,

n 1 V

V

1

-

t pi

= 1 t = P2

D.05 H = = 100

2

1

(f)

i 1 1 1

Page 439: Applications of Markov Chains in Chemical Engineering

433

u

1

0.8

0.6

0.4

0.2

0

i =

_2

t = pi

{

J 1

1 t = p2

0.5 H

/

/ ..--.-_.-.J-.-

= IOC 1 h- 1

(kj

H

H

/

J 1

/

1 d

0 2 4 6 8 10 t

Fig4.4-5(lb). Ci versus t demonstrating the effect of |Xi and tpi

4.4-6 The fluid flow Qi is divided into flows Q12 and Q13 as shown in Fig.4.4-6.

The tracer, in a form of a pulse input, is introduced into reactor 1; reactor 4 is the

collector of it.

Fig.4.4-6. An open system with divided flow between two plug flow reactors

Noting the residence times in each plug flow reactor, i.e. tpi and tp2, and assuming that tpi > tp2, yields the following matrices:

0< t <tp2: tp2<t <tpi:

p =

1

2

3

1

Pll 0

0

2

P12

1

0

3

P13

0

1

1

2

P = 3

4

1

Pll 0

0

0

2

P12

P22

0

0

3

P13

0

1

0

4

0

P24

0

1

Page 440: Applications of Markov Chains in Chemical Engineering

tpi < t:

1

2

P= 3

4

1

Pll 0

0

0

2

P12

P22

0

0

3

P13 0

P33

0

4

0

P24

P34

1

Referring to Fig.4-1, yields for the configuration in Fig.4.4-6 the following

probabilities considering reactor j (hence j = 1, a = 2,... and ^ = 4) as well as

Eqs.(4-12a) to (4-12c) and (4-14) to (4-17b) and Qi as reference flow:

PI 1 = 1 - m At P12 = ai2|i2At P13 = ais^isAt

where an + a n = 1 P24 = oci2 p34 = OL\3

P22 = 1 - ai2^2At P24 = ai2|^4At

P33 = 1 - aiB^lsAt P34 = (1 - ai2)|l4At

As seen, the parameters of the solutions are: |ii = QiA^i (i = 1,..., 4), ai2 =

Ql2/Qb tpi and tp2. In the numerical solution it has been was assumed that tpi =

0.1, tp2 = 0.05, At = 0.001 as well as C(0) = [1, 0, 0, 0], i.e. the pulse was

introduced into reactor 1 in Fig.4.4-6. The effect of an = Qi2/Ql» 0 and 0.5, is

demonstrated in cases a and c in Fig.4.4-6a, as well as in cases a to d and e, f, for

different values of |ii. The effect of m is demonstrated in cases a to c, c to e and d

to f. In case d, the sudden change of C4 at tp2 = 0.05 and tpi = 0.1, is clearly

observed. In cases e and f, C2 and C3 are greater than unity because the volume of

reactors 2 and 3 are reduced by increasing ^i (for a constant flow rate), i.e. p,2 = ^3

= 100 by comparison to |ii = 14 = 1.

Page 441: Applications of Markov Chains in Chemical Engineering

435

u

1

0.8

0.6

0.4

0.2

0

\,^^ 1

1 ^

r " f

.^

1 1 1 |i = 1 a = 0

i 12

^ .,4, - ' '

' - - ' ' ' 2

1 i 1

(a)

-""-'

0 0.2 0.4 0.6 t

0.8 1 0

U

1

0.8

0.6

0.4

0.2

0

-

-

= 1

1 ' \ "

/

(\

1 \

\ / '

l /^ /

1' \ 1

1

• \ 1,2,4 1 1,2 ^

1

(c)l

~1

^1.= 100--

a = 0 12 —

1-3 1

1 ! 1

— i ^

i 1 /

~ 1 3 '

/ 4

V

_m

^ . = 1 0 0 -

a = 0.5 12 —

^ 1-3 1 1 1" 1 1

0 0.05 0.1 0.15 0.2 0 0.05 0.1 0.15 0.2

10 i = 3/

: / :

- ;

- ^

fi 1

K

— ^

K 1 ^ = ^ = 100 (e^ 2 3

a = 0 12

-

4

1 1

-13

l\

" / ^ 1 1 1

K - \

1

= 1 i = M = 100 (ri 2 3

a =0.5 1 12

- , _ _ _ 4 -

1 1 0 0.2 0.4 0.6

t 0.8 1 0 0.2 1 0.4 0.6 0.8 1

Fig4.4-6a. Ci versus t demonstrating the effect of |Xi and a i2

4.4-6(1) A simplification of case 4.4-6 is demonstrated in Fig.4.4-6(1) simulating the

configuration in ref.[78, p.88].

Page 442: Applications of Markov Chains in Chemical Engineering

436

Fig.4.4-6(1). An open system with divided flow

The following transition matries are applicable:

0 < t <tp2:

P =

tp2^t :

1

2

3

4

1

PU 0

0

0

2

P12 1

0

0

3

P13 0

P33 0

4

0

0

P34 1

1

2

P= 3

4

1

Pll 0

0

0

2

P12

P22

0

0

3

P13 0

P33 0

4

0 1 P24

P34 1

Eqs.(4-12a) to (4.12-c) give that P24 = ^12. p34 = CX13 where a i2+a i3 =1.

Eqs.(4-14) to (4.17b) yield the following probabilities:

Pll = 1 - mAt P12 = ai2|li2At P13 = ai3|l3At

P22 = 1 for 0 < t < tp2 and P22 = 1 - ai2^2At for t > tp2 P24 = oci2^4At

P33 = 1 - ai3^3At P34 = ai3^4At

As seen, the parameters of the solutions are: [i[ = Qi/Vi (i = 1, ..., 4), a i2 =

Q12/Q1, tpi and tp2. In the numerical solution it was assumed that tpi = 2, tp2 = 1,

At = 0.01 as well as C(0) = [1, 0,0,0], i.e. the pulse was introduced into reactor 1

in Fig.4.4-6(1). The effect of a i2 = Qi2/Ql» 0» 0-5 and 1, is demonstrated in

Fig.4.4-6(la); cases a to c correspond to |Xi = 10 and cases d to e for |ii = 100.

The effect of |ii, 10 and 100, is demonstrated in cases a and d, b and e and c and f

for various values of ai2.

Page 443: Applications of Markov Chains in Chemical Engineering

437

1

0.8

0.6

0.4

0.2

0

V

/ .

LL 1

1

/

— 2 — = : ~ ^ 1 1

- —

4

, .1.

(a^^

a = 0 H 12

1

i _ i

1

4 (b)

/ I /

/ / a = 0.5 "1

^.= 10 ^

\ -

1 1 1 1 1 0 0.2 0.4 0.6 0.8 1 0

t 1

0.8

0.6

0.4

0.2

0

0.5

4 J

\ : -\

\i a = 1 -

, n,= io -

1-3 1 i i 1 1

0.5 1.5

1

0.8

0.6

0.4

0.2

0

\ i = l '

A / \ /

" / \

-/ \

L _ 2 . \

\^ / " "

V / \

/ \ \

1 1

4

a = 12

^ i =

1-3 1 1

(d)

0 -j 100 J

-

0 0.01 0.02 0.03 0.04 0.05 0 t

1

0.8

0.6

r 0.4

0.2

0

-

\-

LJ

2

1.3,4

1

2,4

4 J

(f)^

a = I H 12

i = 100-^

1-3 1

1.5

0.5 1.5

Fig4.4-6(la). Ci versus t demonstrating the effect of ^i and ai2

Page 444: Applications of Markov Chains in Chemical Engineering

438

4,4-6(2) This case simulates the situation demonstrated in ref.[81], designated as

'partial mixing with piston flow and short-circuit' in continuous flow systems. It is

an extension of case 4.4-6(1) which is demonstrated in Fig.4.4-6(2) below.

•p2

^

Fig.4.4-6(2). An open system with plug flow and short-circuit

The following transition matrices are applicable:

0 < t <tp2: tp2^t :

p =

1 2

3

4

5

Pll 0

0

0

0

P12 1

0

0

0

P13 0

P33 0

0

P14 0

0

P44 0

0 0

P35

P45 1

P =

1

2

3

4

5

1

Pll 0

0

0

0

2

P12

P22

0

0

0

3

P13 0

P33 0

0

4

P14 0

0

P44 0

5

0

P25

P35

P45 1

Similar to case 4.4-6(1), the following probabilities are obtained:

Pll = 1 - ILliAt P12 = ai2^2At P13 = ai3|l3At P14 =

ai4|i4At

P22 = 1 for 0 < t < tp2 and P22 = 1 - CXi2|i2At for t > tp2 p25 = CXi2|l5At

P33 = 1 - OCislIsAt P35 = ai3|l5At

P44 = 1 - ai4|X4At P45 = ai4|X5At

where a u = 1- (ai2 + a n ) . The parameters of the solution are: tp, m, a i2 and

a i3 and the pulse was introduced into reactor 1 in Fig.4.4-6(2), i.e. C(0) = [1,0,

0, 0, 0]. In the numerical solution it was assumed that |ii = ji2 = 1x4 = 115 = 30,

i.e. the response of the reactors is relatively fast by comparison to the response of

Page 445: Applications of Markov Chains in Chemical Engineering

439

reactor 3 for which ^3 = 2. These conditions, approximately, simulate the model in

ref.[81] where reactors 1, 2, 4 and 5 may be considered as measurement 'points'

(very small reactors) of the concentration of the pulse. Cases a, b and c in

Fig.4.4-6(2a) demonstrate the effect of tp2 = 0.03, 0.3 and 3 for a 12 = a 13 = a 14

= 1/3. Cases c and d demonstrate the effect of ai2 and 013. In the later case ai2 =

0.1 and ai3 = 0.9 for tp2 = 3, i.e. Q14 = 0. In the solution. At = 0.0003, 0.003

and 0.03 for tp2 = 0.03, 0.3 and 3, respectively.

u

1

0.8

0.6

0.4

0.2

0

-Aj = i

> " ^ :

' t =0.03 ^ J p2 n

a - a - a - 1/3 (a) 12 13 14 J

H

_5. . -" ' H

1 3 1 H I 0.05

1 \ r t =0.3 p2

a - a - a - 1/3 12 13 14 . . . -

(b)

J I I L 0.1 0.15

U

1

0.8

0.6

0.4

0.2

0

. ^ -

1,

Lv—i^

t

« t = 3 « ' P2 , .

a - a - a - 1/3 12 13 14

. = 1 . . . , ' 5

- " ' 2 ———

> \

"(c^

-A J

H

i . 1 1 „.„. 1

0 0.2

1 1

-

L 1 ' '

u^

0 4

1

t

- 2

0.6

1

t = p2

a -0 .1 12

0.8 1

1 J

(d)J 3

a - 0.9 13 1

-

1 1 1 1 l-

Fig4.4-6(2a). Ci versus t demonstrating the effect of tp2andaij

4.4-7 This example extends the previous one and simulates a contacting pattern of

fluid elements of different ages [21, p.334]. The fluid flow Qi is divided into

flows Q12, Qi3 and Q14 as shown in Fig.4.4-7. The tracer, in a form of a pulse

input, is introduced into reactor 1 whereas reactor 5 is the collector of the tracer.

Page 446: Applications of Markov Chains in Chemical Engineering

440

Fig.4*4-7. An open system with divided flow between three plug flow reactors

Noting that the residence time in each plug flow reactor is tpi, tp2 and tp3 and

assuming that tpi > tp2 > tp3, yields the following matrices:

0 < t <tp3:

1

1 1 2

**= 3

4

PU 0

0

0

tp2<t <tpi:

1

1

2

P = 3

4

5

Pll 0

0

0

1 0

2

P12

1

0

0

2

P12

P22

0

0

0

3

P13

0

1

0

3

P13

0

P33

0

0

4

Pl4

0

0

1

4

P14

0

0

1

0

5

0

P25

P35

0

1

tp3<t < tp2:

1

1

2

P = 3

4

5

Pll 0

0

0

0

tpi < t:

1

1

2

P = 3

4

5

Pll 0

0

0

1 0

2

P12

P22

0

0

0

2

P12

P22

0

0

0

3

P13

0

1

0

0

3

P13

0

P33

0

0

4

P14

0

0

1

0

4

P14

0

0

P44

0

5 1 0

P25

0

0

1 1

5 1 0

P25

P35

P45

1

Referring to Fig.4-1, yields for the configuration in Fig.4.4-7 the following probabilities by considering reactor j (hence j = 1, a = 2,... and ^ = 5) and taking Qi as reference flow. From Eqs.(4-12a) to (4-12c) one obtains that ai2 = P25» OC13 = P35 and ai4 = P45 where ajk and pi are defined in Eq.(4-2). In addition ai2 + a\3 +ai4 = 1 where Eqs.(4-14) to (4-17b) yield:

Page 447: Applications of Markov Chains in Chemical Engineering

441

pil = 1 - (ai2 + ai3 +ai4)mAt pn = ai2Ji2At pu = aisjisAt

pi4 = (l-ai2-ai3)M4At

P22 = 1 - ai2M'2At P25 = ct^wAt

P33 = 1 - aisfisAt P35 = aisjisAt

P44= l - ( l -a i2-a i3)wAt P45= (1-ai2 - ai3)^5At

As seen, the parameters of the solutions are: Mi = Ql/Vi (i = 1, ..., 5), a 12, a 13,

tpi, tp2 and tp3. In the numerical solution it was assumed that tpi = 0.4, tp2 = 0.2

and tp3 = 0.1; m = 200, At = 0.001 as well as C(0) = [1, 0, 0, 0, 0], i.e. the pulse

was introduced into reactor 1 in Fig.4.4-7. The effect of a 12 = Q12/Q1 = 1.

ai3 = Q13/Q1 = 1» ai4 = QlVQl = 1 and ai2 = 0.2 ai3 = 0.3 and ai4 = 0.5, is

demonstrated in Fig.4.4-7a.

u

1 h

0.8 h

0.6

0.4 [-

0.2 h

0

-0.1

u

-0.1

i = l / 2 : •

L 3-5

0.1 0.2 t

0.1 0.2 t

1-4

0.3

a - 1 12 «J

J I L 0.4 0.5

1

0.8

0.6

0.4

0.2

0

-

i = l (

1

\ 1 1

3

2,4,5 1

1; 1: I 1 ,1 •1

.1 1

5

1-4 1

- j J

a -1 13 J

J

[ , „.

0.3 0.4 0.5

Page 448: Applications of Markov Chains in Chemical Engineering

442

u

1

0.8

0.6

0.4

0.2

0

-

-

i = 11

i 1

1

i

:i \ .

4

1,2,3.5 i 1 i

[; 5 1

1 CL m l \

1 ^ - J

•» H ' V l - 4 1

-0.1

u

1

0.8

0.6 h

0.4

0.2

0

-0.1

i = l

0.1 0.2 0.3 0.4 0.5 t

a - 0.2 a - 0.3 a - 0.5 12 13 14

4,5

'5 -J

JLS \ 1-4 J I I I L 0 0.1 0.2 0.3 0.4 0.5

t

Fig4.4-7a. Q versus t demonstrating the effect of aij

4.4-8 The following configuration of two interacting plug flow reactors was applied

elsewhere [21, p.298] for describing deviation from plug flow and long tails. Due to the interaction between the reactors, it is necessary to divide the reactor into perfectly mixed reactors. In the following example, the reactor was divided to five reactors. Generally, the number of perfectly mixed reactors needed, must be determined by comparing the response curve of the divided system to that of the plug flow reactor.

Page 449: Applications of Markov Chains in Chemical Engineering

443

l i o k J opJ op "op BL^4

Yb a\C

w a a

w \a ou

m

" ^ "

a a

^ ^ K 50,

Fig.4.4-8. Two plug flow interacting reactors

Referring to Fig.4-1, yields for the configuration in Fig.4.4-8 the following

probabilities by considering reactor j and taking Qi as reference flow. From

Eqs.(4-12a) to (4-12c) one obtains a set of equations for the determination of aik-

Assuming that:

a = a67 = OC85 = a58 = a94 = a49 = aio,3 = OC3,10 = 0Cii,2

P = OC23 = CX34 = a45 = a 5 6

Y=a76 = a2,ii

5 = a87 = CX98 = cxio,9 = ocnjo (4.4-8a)

as well as

p + 5 = 1 (4.4-8b)

it follows that a + 6 = Y (4.4-8c)

The probability matrix for the above configuration reads:

Page 450: Applications of Markov Chains in Chemical Engineering

444

P =

1

2

3

4

5

6

7

8

9

10

11

12

1

Pii

0

0

0

0

0

0

0

0

0

0

0

2

Pl2

P22

0

0

0

0

0

0

0

0

Pll,2

0

3

0

P23

P33

0

0

0

0

0

0

PlO,3

0

0

4

0

0

P34

P44

0

0

0

0

P94

Pl0,9

0

0

5

0

0

0

P45

P55

0

0

P85

0

0

0

0

6

0

0

0

0

P56

P66

P76

0

0

0

0

0

7

0

0

0

0

0

P67

P77

P87

0

0

0

0

8

0

0

0

0

P58

0

0

Pss

P98

0

0

0

9

0

0

0

P49

0

0

0

0

P99

PlO,9

0

0

10

0

0 f

P3,10

0

0

0

0

0

0

PlO,10

Pii,ioP

0

11

0

2,11

0

0

0

0

0

0

0

0

11,11

0

12

0

0

0

0

0

P6,12

0

0

0

0

0

1

Assuming that all m's are equal, i.e. |ii = \i, the following probabilities are

obtained from Eqs.(4-14) to (4-19):

pil = 1 - iLlAt P22 = 1 - (P + Y)| At P33 = P44 = P55 = 1 - (a + P)|liAt

P66 = 1 - (1+ a)|LiAt

P77 = P88 = P99 = P10,10 = P11,11 = 1 - TI At

Pl2 = P6,12 = At

P23 = P34 = P45 = P56 = PjAAt

P87 = P98 = P10,9 = PI 1,10 = 8|XAt P2,ll=P76 = mAt

P3,10 = P49 = P58 = P67 = Pi 1,2 = PlO,3 = P94 = P85 = a|XAt

The parameters of the solution are a and p related by Eqs.(4.4-8b) and (4.4-8c)

as well as \i, Fig.4.4-8a demonstrates the effect of the circulation intensity a

between the reactors on the transient response in various reactors of the tracer,

introduced in reactor 1. In case a, a = 0.05, P = 0.5, 5 = 0.5 and y = 0.55; in case

b, a = 5, P = 0.5, 5 = 0.5, y = 5.5 and At = 0.001 where in both cases \i = 50.

As observed, by increasing a, the C-t curves for reactors 4 and 9 (see Fig.4.4-8

above), reactors 6 and 7 and reactors 2 and 11 become identical.

Page 451: Applications of Markov Chains in Chemical Engineering

445

0.8 h-

0.6 U

u 0.4

0.2

[

• i = l

p.

I- / A . 2

\l "••V 11 ,4 / 9

f ''-\ - •'•-'/ [.i? ' - •• ilr.-'-rr*^^-——_

1 1 1

y' y

, 1 2

/

- - ^ ^^''7

• • - • " • - = - ^ "" ' 7 - • • - . . . .

1 1

(a)

-j

-j a = 0.05

0 F^-

0 0.1 0.2 0.3

t

0.4 0.5

U

0.8

0.6

0.4

0.2

0

•i = l

i , 2 , l l

4,9

1

^.

/ •

, ' 12 /

/ /

- . ,6,7

1 1

• • ' " "

- — ' •• ' 1

(b)

a = 0.5 - j

-

1

0 0.1 0.2 0.3 0.4 0.5 t

Fig4.4-8a. Ci versus t demonstrating the effect of the circulation a

The variation of the mean residence time tm in reactors 1 to 11, computed by Eq.4-26, was obtained from the response curve of reactor 6. The resuhs are sununarized in the following Table for the various operating parameters a to 5 listed in the Table and defined by Eq.4.4-8a. The following trends are observed: a) Increasing |i (i.e. the flow rate), decreases tm. b) Taking a = 0 in case c, i.e. decreasing the number of effective reactors in Fig.4.4-8, decreases tm-

Page 452: Applications of Markov Chains in Chemical Engineering

446

^ 10

50

100

1 n

(a) a = 0.05

P = 0.5

Y = 0.55

5 = 0.5

tm 1.096

0.219

0.109

1 11

(b) a = 0.05

p = o Y= 1.05

6 = 1

tm 1.080

0.219

0.108

11

(c) 1 a = 0 p = o Y=l

6 = 1

tm 1 0.797

0.159

0.079

8

n - number of effective reactors

4>4-9 The following configuration is a simulation (see also case 4.4-6(2)) of

"partial mixing with piston flow and short-circuit" in continuous flow systems

treated in ref. [81].

%4

Q i o ^ ^ 1 2 1 ^ 4 = 4

^32

t,

ns

yfe

Fig.4.4-9. Mixing with plug flow and short-circuit

The following transition matrices are applicable:

< t < tp: 1

1

»= 2

4

Pll 0

0

2

P12

P22

0

4

P14

P24

1

tp<t:

1

2

P = 3

4

1

Pll 0

0

0

2 3

P12 P13

P22 0

P32 P33 0 0

4

P14

P24

0

1

Page 453: Applications of Markov Chains in Chemical Engineering

447

Referring to Fig.4-1, yields for the above configuration the following

information by considering reactor j (hence j = 1, a = 2,... and ^ = 4) as well as

Eqs.(4-12a) to (4-12c) and (4-14) to (4-17b) and Qi as reference flow.

Pi4 + p24 = 1 Pi4 + ai2 + ai3 =1 For 0 < t < tp: ai2 = p24 and a32 = 0; for tp < t: an + OL32 = p24 and

0C13 = a32. The following probabiUties were obtained:

Pll = 1 - (Pl4 + ^12)^1 At for 0 < t < tp, and

Pii = 1 - (Pi4 + ai2 + oci3)mAt for tpi < t.

P12 = ai2M'2At P13 = aiBiiisAt pu = P u M t

P22 = 1 - p24^2At P24 = p24|ll4At

P33 = 1 - ai3M'3At P32 = ai3^2At

The parameters of the solutions are: m = QiA i (i = 1,..., 4), an = Qii/Qi (i

= 2, 3) pi4 = qi4/Qi (i = 1, 2) and tp. In the numerical solution C(0) = [1, 0, 0, 0],

i.e. the pulse was introduced into reactor 1 in Fig.4.4-9. Common quantities in all

cases were: m = p,4 = 10, 13 = 50, an = 0.1, a n = 0.8 and P14 = 0.1. The

effect of tp, 0.1 and 1, is demonstrated in cases a and c in Fig.4.4-9a. The effect

of |I2, 50, 100 and 300, is depicted in cases b, c and d. For tp = 0.1, At = 0.001

and for tp = 1, At = 0.01. An interesting observation is the sudden increase of the

concentration in reactor 2 at t = tp in cases c and d. This is caused by the sudden

supply of the solute at this time due to the plug flow reactor.

u

1

0.8

0.6

0.4

0.2

0

1 1 \ i = 1 4^

" A / - \ /

~ , •' \^ 2

'^ ^ " " ~ - ^ : - i ^ ^ . , . .

1 1 J

(a)|

t = o.i H p

^1=50 - 2

t i l l

I 1

i \ X

b ^ \"--r ^ "^^^^ y ^ - ^

J

t =1 H p

2

-

1 1 1 1 0 0.2 0.4 0.6 0.8 1 0

t 0.5 1.5

Page 454: Applications of Markov Chains in Chemical Engineering

448

u

1

0.8

0.6

0.4

0.2

0

1 1 1 \ i = l 4-- " "~

~ \ / - A /

7 /\ y ^-^•='^-^^.

1

(c)l

t = 0.1 -1 p

^1= 100^ 2

1 1 1 1

1 1

,'

L

\ /

f i \ '

r " ^ L. ^ < i i

1 1

1 1 1

t = o.H p

l = 300 H 2

-

1 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

t t Fig4.4-9a. Ci versus t demonstrating the effect of tp and |i2

4,4-9(1) The following configuration is another simulation (see also cases 4.4-6(2)

and 4.4-9) of "partial mixing with piston flow and short-circuit" in continuous flow systems treated in ref.[81].

1 ,4

• .J 12 r ^ fefel9i ^ = 4

23 32

fe

Fig.4.4-9(1). Mixing with plug flow and short-circuit

The following transition matrices are applicable:

0 < t <t j )•

1

1

p = 2

4

Pll 0

0

2

Pl2

P22

0

4

P14

P24

1

t p < t :

1

p = 2

3

4

1

Pll 0

0

0

2 3

P12 0

P22 P23

P32 P33

0 0

4

P14

P24

0

1

Similarly to case 4.4-9, the following relations are obtained:

Page 455: Applications of Markov Chains in Chemical Engineering

449

Pl4 + p24 = 1 Pl4 + ai2 = 1 yielding that an = p24 and that a23 = OL32

pi 1 = 1 - mAt P12 = ai2|i2At P14 = Pi4^4At

For 0 < t < tp P22 = 1 - ai2^2At; for tp < t P22 = 1 - (OC23 + ai2)|X2At

In addition p24 = ai2|X4At p23 = a23|X3At P33 = 1 - a23M'3At and p32 =

OC23 2At.

The parameters of the solution are: tp, ai2, OC23 = R and |Xi (i = 1, ..., 4).

In the numerical solution C(0) = [1, 0, 0, 0], i.e. the pulse was introduced into

reactor 1 in Fig.4.4-9(1). Common quantities in all cases were: ^i = |i2 = ILL3 = ILI4

= 1 and ai2 = 0.5. The effect of R, 0,0.2 and 2, is depicted in cases a, b and c in

Fig.4.4-9(la). The effect of tp, 2 and 10, is demonstrated in cases c and d. For tp

= 2, At = 0.02 and for tp = 10, At = 0.1. Interesting observations are: a) For R >

2, the transient response of Ci remains unchanged; and b) for tp = 10, the effect of

R is negligible because the response was transferred straight into reactor 4, by­

passing reactors 3 and the plug flow.

u

1

0.8

0.6

0.4

0.2

0

U = i "T" ^

\ / _ y

7 ^

4^ (a)

A t = 2 R = 0 p J

-

1 1 1 1

8 10 0 8 10

— \\

A

A

I-

/ /

_-3_. . 1

/

" 2 ,

^ 4 '

1 _

(d)

1

t =10 p

1

-J

R = 2

~

8 10 0 4 6 t

8 10

Fig4.4-9(la). Cf versus t demonstrating the effect of tp and R

Page 456: Applications of Markov Chains in Chemical Engineering

450

4,4-10 The following configuration demonstrates a general cell model of a

continuous flow system described in ref.[77]. There are two possibilities to arrive

at state 4, i.e. directly and via the upper plug flow reactor. However, in order to

materialize these possibilities it was necessary to add state 3-a perfectly mixed

reactor 3. The residence time in this reactor is controlled by the quantity 113.

Qi

Fig.4.4-10. A cell model

Three cases will be treated below, depending on the magnitudes of tpi and

tp2. Case a: tpi = tp2

The following transition matrices are applicable:

0 < t <tpi:

P =

1

t p i < t :

1

2

4

Pll

P21

0

P12

P22

0

P14 0

1

1

2

P = 3

4

5

Pll

P21

0

0

0

P12

P22

0

0

0

P13 0

P33 0

0

P14 P15

0 P25

P34 0

P44 P45

0 1

Referring to Fig.4-1, yields for the configuration in Fig.4.4-10 the following

probabilities by considering reactor j (hence j = 1, a = 2,... and ^ = 5) and taking

Ql as reference flow. From Eqs.(4-12a) to (4-12c) one obtains that:

Pl5 + P25 + P45 = 1 (4.4.10a)

1 + OC21 = Pi5 + OL12 + ai3 + ai4 (4.4-lOb)

Page 457: Applications of Markov Chains in Chemical Engineering

451

ai2 = P25 + «2l for i = 2; ai3 = a34 for i = 3; ai4+ a34 = P45 for i = 4 (4.4-10c)

The following probabilities are obtained from Eqs.(4-14) to (4-19) for

0 < t <tpi: pii = l - ( a i 2 + ai4)mAt pi2 = ai2^2At pi4 = ai4^4At (4.4-lOd)

P22 = 1 - a2m2At

Fortpi<t: pii = 1 - (ai2 + ai3 + ai4 + Pi5)mAt

P21 = a2imAt

P13 = ai3^3At

where pi2and p 14 are given above.

P22 = 1 - (a21 + p25) 2At P25 = p25^5At

where p2i is given in Eq.(4.4-10e).

P33 = 1 - a34^3At P34 = a34M,4At P44 = 1 " P45M-5At

(4.4-lOe)

P15 = PlSM-sAt

(4.4-lOf)

(4.4-lOg)

P45 = p45^5At

(4.4-lOh)

The parameters of the solutions are: |ii = QiA i (i = 1,..., 5), tpi, tp2, ai2,

cti3, ai4 and a2i = R- The rest of the parameters can be obtained from Eqs.(4.4-

10a) to (4.4-lOc). Fig.4.4-10a demonstrates the effect of tpi (i = 1, 2), 0.01, 0.1

and 1 (corresponding to At = 0.0001, 0.001 and 0.01, respectively) and jlj = 10

and 100 (i = 1,..., 5) for C(0) = [1, 0, 0, 0, 0], i.e. the pulse was introduced into

reactor 1 in Fig.4.4-10. Other unchanged parameters were: ai2 = ai3 = ai4 =

0.25 and R = 0.1. The effect of tpj is demonstrated in cases a, b, c and the effect of

|i.i in cases c, d. In the latter case the response becomes faster by increasing |li. In

addition, increasing tpi causes reactor 3 to become ineffective.

u

1

0.8

0.6

0.4

0.2

0

^=^ t = 0.01 ^ = 10 ^ -i ^ " " \ ^ pi i

- ^.-^.-^-J " 1 1 LJ 1 I

-V - \

- \

t = 0.1 |X.= 10 ^ pi »

5 . - • - ' • J

^•^<^4=--.— J 1 1 1 1 "

0 0.02 0.04 0.06 0.08 0.1 0 0.2 0.4 0.6 0.8 1 t t

Page 458: Applications of Markov Chains in Chemical Engineering

452

1

0.8

0.6

0.4

0.2

0

ti=l 5-' A 4 ' ' /

f ^ v _

- • • ' ^

A

t =1 H pi

1 1 0 1 3 0

t t Fig4.4-10a. Ci versus t demonstrating the effect of tpi and [if

Case b: tpi > tp2 The following transition matrices are applicable:

0 < t < tp2:

P =

tpi<t:

tp2^t<tpi:

1

1

2

4

Pll P12

P21 P22

0 0

P14 0

1

1

P = 2

4

5

Pll

P21 0

0

P12 P14 P15

P22 0 P25

0 P44 P45

0 0 1

p =

1 2

3

4

5

Pll

P21

0

0

0

P12

P22

0

0

0

P13 0

P33 0

0

P14 0

P34

P44 0

P15

P25

0

P45 1

Eqs.(4.4-10a) to (4.4-lOc) are applicable also in this case. For 0 < t < tp2, the probabilities pn, pi2, pi4 are given by Eq.(4.4-10d); p22

andp2iby Eq.(4.4-10e). Fortp2<t<tpi:

pii = l - (a i2 + a i4+Pi5)Mt where pi2 and pu are given in Eq.(4.4-10d), pis in Eq.(4.4-10f), p22 in Eq.(4.4-lOg), P21 in Eq.(4.4-10e), p44 and P45 in Eq.(4.4-10h).

Page 459: Applications of Markov Chains in Chemical Engineering

453

For tpi < t: PI 1 = 1 - (ai2 + ai3 + ai4 + Pi5)mAt

pl3 and P15 are given in Eq.(4.4-10f), pi2 and pi4 in Eq.(4.4-10d), p22 and p25 by Eq.(4.4-10g) and p2i in Eq.(4.4-10e). Finally, P33, P34, P44 and P45 are give by Eq.(4.4-10h).

The parameters of the solution are: |ii = Qi/Vi (i = 1, ..., 5), tpi, tp2, ai2, ttl3> 0C14 and a2i = R. The rest of the parameters can be obtained from Eqs.(4.4-10a) to (4.4-lOc). Fig.4.4-10b demonstrates the effect of the residence time in the plug flow reactors for tpi = 2tp2 where tp2 = 0.01 and 0.1 (corresponding to At = 0.00005 and 0.0005, respectively) and m = 25 and 250 (i = 1, ..., 5) for C(0) = [1, 0, 0, 0, 0], i.e. the pulse was introduced into reactor 1 in Fig.4.4-10. Other unchanged parameters were: au = ai4 = 0.05, a\3 = 0.9 and R = a2i = 0.025. The effect of tpi, tp2 is demonstrated in cases a, b and c, d, respectively. The effect of |ii is demonstrated in cases a, c and b, d. Note the similarity between cases b and c although the time scale is different by a factor of ten.

t =0.02 t =0.01 ^^ ^pl p2 ^

1 = 25

t =0.02 t =0.01 (c)J pi p2 ^

|i.= 250

0 0.01 0.02 0.03 0.04 0.05 0 0.1 0.2 0.3 0.4 0.5 t t

Fig4.4-10b. Ci versus t demonstrating the effect of tpi and m

Page 460: Applications of Markov Chains in Chemical Engineering

454

Case c: tp2 > tp i

The following transition matrices are applicable:

0 < t <tpi:

P =

tp2^t:

tpi<t<tp2:

P =

1

2

4

1 2 4

Pll P12 P14

P21 P22 0

0 0 1

1 2 3

1

2

= 3

4

5

Pll P12 P13

P21 P22 0

0 0 P33

0 0 0

0 0 ( )

P =

4

P14 0

P34

P44 0

1

2

3

4

1

Pll

P21

0

0

5

P15

P25

0

P45 1

2

P12

P22

0

0

3

P13 0

P33

0

4

P14 0

P34

1

Eqs.(4.4-10a) to (4.4-10c) are applicable also in this case.

For 0 < t < tpi, the probabilities pi i, pi2, pi4 are given by Eq.(4.4-10d); p22

andp2iby Eq.(4.4-10e).

Fortpi<t<tp2:

Pll = 1 - (ai2 + ai3 + ai4)|LiiAt

where pi2 and pi4 are given in Eq.(4.4-10d), p n in Eq.(4.4-10f), p22 in Eq.(4.4-

10e)» P21 in Eq.(4.4-10e), P33 and P34 in Eq.(4.4-10h).

Fortp2<t:

Pii = l - ( a i 2 + ai3 + ai4 + Pi5)mAt

P13 and pi5 are given in Eq.(4.4-10f), pi2 and pi4 in Eq.(4.4-10d), p22 and P25 by

Eq.(4.4-10g) and p2i in Eq.(4.4-10e). Finally, P33, P34, P44 and P45 are give by

Eq.(4.4-10h).

The parameters of the solution are: |Lii = Qi/Vi (i = 1, ..., 5), tpi, tp2, a i2,

ai3, a i4 and a2i = R. The rest of the parameters can be obtained from Eqs.(4.4-

10a) to (4.4-10c). Fig.4.4-10c demonstrates the effect of the residence time in the

plug flow reactors for tp2 = 2tpi where tpi = 0.01 and 0.1 (corresponding to At =

0.00005 and 0.0005, respectively) and \i{ = 25 and 250 (i = 1, ..., 5) for C(0) =

Page 461: Applications of Markov Chains in Chemical Engineering

455

[1, 0, 0, 0, 0], i.e. the pulse was introduced into reactor 1 in Fig.4.4-10. Other unchanged parameters were: ai2 = ai4 = 0.05, ai3 = 0.9 and R = a2i = 0.025.

The effect of tpi, tp2 is demonstrated in cases a, b and c, d, respectively. The effect of m is demonstrated in cases a, c and b, d. Note the similarity, as in previous example, between cases b and c although the time scale is different by a factor of ten.

u

1

0.8

0.6

0.4

0.2

0

Tr^

^

t =0.01 t =0.02 % pi p2 1

s^ ^.-25 J

^ ' ^ ' ' H . , , ^ ^ ^ ^ - J " ^ ^ .v -J

3_ - - " T ]

— 1 ,1 „.1„..,..2,.„J

psl

^

1 1

t = pi

4

*

1

O l t = p2

**i"

1

\ ' S

^. ^ ' ^ • * - ' — i * ^

1

0 2

25

-"

ao^Ii;.

5

- 2 * _ _

1

~~M . . - -

u

(

1

0.8

0.6

0.4

0.2

0

3

^ \ i •

0.01

= 1

- . ™ J . -

0.02 0.03 t

t =0.01 t = 0.01 pi P2

f i . - 250

r 1 .

- 1 1 . ,

0 . 0 4 O.i

(ci

• ' 5 " " J

-^

H

— 2 .

" r"^ 1

0.1 0.2 0.3 0.4 0.5

0.01 0.02 0.03 t

0.04 0.05 0

Fig4.4-10c. Ci versus t demonstrating the effect of tpi and yu^

4 .4 -11 The configurations in Rg.4.4-11 demonstrate a model for small deviations

from plug flow and long tails in continuous flow system described in ref.[21, p.298]. The deviation from plug flow, as well as adding time delay, is achieved by introducing plug flow reactors in parallel, as shown below. In case a the upper reactor is of plug flow type. In case b the plug flow reactor was divided into perfectly mixed reactors in order to obtain a solution for the following cases, 4.4-11(1, 2, 3). In general, the perfectly mixed reactors are not of equal size, depending on the number of reactors in parallel and on their location.

Page 462: Applications of Markov Chains in Chemical Engineering

456

a)

I m in Q,

I

b)

Q,

Time delay at random location

/ ^

I UTT

D

do I CD I do I ... I do I do I do |_»;

I I I

Fig.4.4-11. Schemes for describing deviations from a plug flow reactor

4,4-11(1) The simplest scheme corresponding to Fig.4.4-11 is demonstrated in case

4.4-1(3). An additional scheme is shown in Fig.4.4-11(1) below. Reactors 2 and 4 were added to account for the recycle streams Q12 and Q34. The response of these reactors can be controlled by the magnitudes of |i2 and ^4. If reactor 2, for example, is a measurement "point" of the concentration, i.e. a very small reactor, |i2 should be assigned a relatively large value.

Page 463: Applications of Markov Chains in Chemical Engineering

457

Qi 1 I n_j_j ^4=5

Q.

I •^4 i

V 'P2

Fig.4.4-l l ( l ) . Simplifled scheme for describing deviations from

plug flow reactor

Three cases will be treated below, depending on the magnitudes of tpi and

tp2. Case a: tpi = tp2

The following transition matrices are applicable:

0 < t < t p i : t p i< t :

1

P = 3

5

From Eqs.(4-12a) to (4-12c) one obtains that:

P35 = a i3 = 1, "21 = ai2 and 034 = 043 (4.4-1 l(la))

The following probabilities are obtained from Eqs.(4-14) to (4-19) for

0 < t <tpi:

pil = l - m A t pi3 = ^3At p33=l-JA3At p35 = ^5At (4.4-1 l(lb))

Fo r tp i< t :

pil = l - ( l+a i2 )mAt pi2 = ai2H2At pi3 = ^3At (4.4-1 l(lc))

p22 = 1 - a2m2At P21 = a2iHiAt (4.4-1 l(ld))

P33 = 1 - (1+ a34)|X3At P34 = a34^4At p35 = l^sAt (4.4-1 l ( le))

1

pll 0

1 0

3

P13

P33

0

5

0

P35

1

1

2

P= 3

4

5

1

Pll

P21

0

0

0

2

P12

P22

0

0

0

3

P13

0

P33

P43 0

4

0

0

P34

P44

0

5

0

0

P35

0

1

Page 464: Applications of Markov Chains in Chemical Engineering

458

P44 = 1 - a43^4At P43 = a43^3At (4.4-1 l(lf))

The parameters of the solutions are: |ii = QiA^i (i = 1,..., 5), tpi = tp2 ,

0 < Ri = ai2 < «> and 0 < R2 = a34 < 00. Fig.4.4-1 l(la) demonstrates the effect

of the residence time in the plug flow reactors for tpi = tp2, 0.1 and 1

(corresponding to At = 0.0005 and 0.005, respectively), the effect of |ii =10 and

100 (i = 1,..., 5) as well as the effect of the recycle Ri = R2,0 and 10, for C(0) =

[1, 0, 0, 0, 0], i.e. the pulse was introduced into reactor 1 in Fig.4.4-11(1). The

effect of Ri = R2 is demonstrated in cases a and b, the effect of tpi = tp2 is

demonstrated in cases b and c and the effect of |LLi is demonstrated in cases c and d.

The following observation should be noted: a) The similarity between cases a and c

although the time scale is different by a factor of ten. b) The effect of Ri = R2 in

case c is negUgible due to the relatively large value of tpi = tp2 = 1. In other words,

no response is observed in reactors 2 and 4.

R =R =0 t =t =0.1 1 2 pi p2

(a5 R =R =10 t =t =0.1 (^ 1 2 pl p2

U

1

0.8

0.6

0.4

0.2

0

^ R =R =0,10 t =t =1 r 1 2 pl p2 \ i = 10 A i =1 • 5 . '

- \

/ - N" ->. 3 - / , ' \ ^ - . ^

1 1 1 1

(cjl

H

-\

-j J:-_ •-

0.1 0.2 0.3 t

0.4 0.5 0 0.01 0.02 0.03 0.04 0.05 t

Fig.4.4-ll(la). Ci versus t demonstrating the effect of tpi, m and

Ri

Page 465: Applications of Markov Chains in Chemical Engineering

459

Case b : tpi > tp2

The foUowing transition matrices are applicable:

0 < t <tp2:

P =

tpi<t:

tp2<t<tpi:

1

3

5

1

Pll 0

0

3

P13

P33 0

5

0

P35 1

p =

1 3 4 5

Pll Pl3 0 0

0 P33 P34 P35

0 P43 P44 0

0 0 0 1

P =

1 2 3

Pll P12 P13

P21 P22 0

0 0

0 0

0 0

4

0

0

5

0

0

P33 P34 P35 P43 P44 0

0 0 1

Eq.(4.4-1 l(la)) is applicable also in this case.

For 0 < t < tp2 the probabilities are given by Eq.(4.4-1 l(lb)).

For tp2 t < tpi the probabilities pn and pi3 are given by Eq.(4.4-1 l(lb)).

P33. P34 and P35 are given in Eq.(4.4-1 l(le)); P44 and P43 are given by

Eq.(4.4-ll(lf)).

For tpi < t the probabilities p n , pi2 and pi3 are given in Eq.(4.4-ll(lc));

P33' P34 and P35 are given by Eq.(4.4-ll(le)); P44 and P43 are given by Eq.(4.4-

ll(lf)).

The parameters of the solutions are: m = QiA^i (i = 1,.... 5), tpi, tp2,

0 < Ri = ai2 < °° and 0 < R2 = a34 < 00. A numerical solution was obtained for

tpi = 0.2 and tp2 = 0.1 (corresponding to At = 0.0005), |J.i = 10 and 2 (i = 1, ...,

5), Ri = R2, 0, 10 and 100 for C(0) = [1, 0, 0, 0, 0], i.e. the pulse was introduced

into reactor 1 in Fig.4.4-11(1). The effect of Ri = R2 is demonstrated in cases a, b

and c in Fig.4.4-1 l(lb) and the effect of fii in cases c and d.

Page 466: Applications of Markov Chains in Chemical Engineering

460

u

1

0.8

0.6

0.4

0.2

0

-

J.i = l

- / '

(ail

5 - ' ' J

R =R =oH 1 2

1 1 1 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5

t t

U

1

0.8

0.6

0.4

0.2

0

(cj

\ i = l . -•• -J

- \ 5. ' ' R =R =100H \ ' ' 1 2

- \ . ' l= loJ 3/- K -

1 I ' ' ^ 1 1

- p -v .

h

r ^ r r

(dJI

1 R =R =100H 1 1 2 ^ = 2 J

: _M73I' I I~I^^

2 1 1 1

0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 t t

Fig.4.4-ll(lb). Ci versus t demonstrating the effect of jiii and Ri

Case c: tp2 > tpi The following transition matrices are applicable:

0 < t < tp i :

P =

tp2^t:

tpl<t<tp2:

1

3

5

1

Pll 0

0

3

P13

P33

0

5

0

P35

1

p =

p =

1 2 3

Pll P12 P13

P21 P22 0

0 0

0 0

1

2

3

4

5

1 2 3 4 5

Pll P12 P13 0 0 P21 P22 0 0 0

0 0 P33 P34 P35

0 0 P43 P44 0

0 0 0 0 1

5

0

0

P33 P35 0 1

Page 467: Applications of Markov Chains in Chemical Engineering

461

Eq.(4.4-11(1 a)) is applicable also in this case. For 0 < t < tpi the probabilities are given by Eq.(4.4-1 l(lb)). For tpi < t < tp2 the probabilities pu, pi2 and pn are given by Eq.(4.4-

ll(lc)), P21 andp22by Eq.(4.4-ll(ld)), P33 andpssby Eq.(4.4-ll(lb)). For tp2 ^ t the probabilities pn, pi2 and pn are given by Eq.(4.4-ll(lc));

P21 and p22by Eq.(4.4-ll(ld)); P33, P34 and P35 are given by Eq.(4.4-ll(le)); P44 and P43 are given by Eq.(4.4-1 l(lf)).

The parameters of the solutions are: m = QiA i (i = 1,..., 5), tpi, tp2, 0 < Ri = ai2 < «> and 0 < R2 = a34 < <». A particular solution was obtained for tpi = 0.1 and tp2 = 0.2 (corresponding to At = 0.0005), fxi = 10 and 2 (i = 1, ..., 5), Rl = R2, 0, 10 and 100 for C(0) = [1, 0, 0, 0, 0], i.e. the pulse was introduced into reactor 1 in Fig.4.4-11(1). The effect of Ri = R2 is demonstrated in cases a, b and c in Fig.4.4-1 l(lc) and the effect of jLii in cases c and d. One should compare the behavior in the above figure corresponding to tp2 > tpi with that in Fig.4.4-

1 l(lb) above for tpi > tp2.

u

1

0.8

0.6

0.4

0.2

0

• \ i = l

- \

3/-- / ^ '

1A_ \ '

(ai

5, • ' ' -j

R =R = 0 -1 2

ki.= 1 0 -

1 1 1 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5

U

1

0.8

0.6

0.4

0.2

0

r-

- \ i = l

Y \

r 1

. - '

1

(cjl

y . • ' \

R =R = 1 0 0 -1 2

^1.= 1 0 -

_ 3 , 4

t

1 ',' ^

\ 1 4 1

(<a|

R =R =10oH 1 2

H = 2 ^

_—__—--r-^.—^~

5 _ 1 1

0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 t t

Fig.4.4-ll(lc). Ci versus t demonstrating the effect of |ii and Ri

Page 468: Applications of Markov Chains in Chemical Engineering

462

4.5 IMPINGING-STREAM SYSTEMS Impinging streams [73] is a unique and multipurpose configuration of a two-

phase suspension for intensifying heat and mass transfer processes in the following heterogeneous systems: gas-solid, gas-liquid, liquid-liquid and solid-liquid.

The essence of the method demonstrated in Fig.4.5, lies in bringing two streams of a suspension, flowing along the same axis in opposite directions, into collision. As a result of collisions between the opposed streams, a relatively narrow zone of high turbulent intensity is generated, which offers excellent conditions for intensifying the heat and the mass transfer rates. In this zone, too, the concentration of the particles is the highest [73], and continuously decreases towards the injection point of the system. The opposed flow configuration of the suspensions encourages multiple inter particle collisions as well as mutual penetration and multiple circulation of the particles from one stream into the other. The penetration of particles arises due to their inertia, whereas deceleration takes place due to the opposite flow of the gas stream as a result of drag forces. At the end of the deceleration path, the particle is accelerated and once again regains its original stream. After performing several such oscillatory motions, the particle velocity eventually vanishes until it is withdrawn from the system. The latter might occur even earlier due to inter particle coUisions.

GAS +

PARTICLES

uzzzzzzzzzzzzzn^^ * ^ • ^ • ^ •

EZZZZZZZZZ2ZZZZZ3 • *

• • •-GAS +

• • • : • • • • • zzzzzzzzzzzzzzzn

PARTICLES

IMPINGEMENT PLANE

Fig.4.5. The principle of impinging streams

The intensification of the transfer processes is due to the following effects: a) An increase of the relative velocity between the penetrating particles and

the opposed gas stream. Under extreme conditions, where the particle attains the gas velocity at the point of entering the opposite stream, the relative velocity may

Page 469: Applications of Markov Chains in Chemical Engineering

463

reach twice that of the gas velocity; the increase is thus significant. Consequently,

the external resistance will decrease.

b) An increase of the mean residence time of the particles in the system or

their holdup due to their penetration into the opposed stream, consequently

undergoing multiple circulation followed by damped oscillations. However, in a

dense system of particles, inter particle collisions might reduce the effect of the

increase of the mean residence time. Finally, an increase of the mean residence

time also allows a decrease of the geometrical size of the system.

c) In gas-liquid and liquid-liquid systems, shear forces exerted between the

phases or inter droplet colUsions, can result in a breakup of the droplets, leading to

an increase in their surface area, their rejuvenation, and therefore an increase of the

mass transfer rates.

d) Collision of the continuous phase of the opposed streams, namely, jet-jet

impingement, induces pressure pulsations or generates intense radial and axial

velocity components in turbulent flow. Consequently, good mixing is created in

the impingement zone of the streams. The mixing is also enhanced by the multiple

circulation of the particles in this zone.

As indicated previously, the key phenomenon in impinging-stream systems is

the penetration of particles into opposite streams, causing an increase of their

holdup in the reactor, of the relative velocity between the particles and the carrying

stream as well as a longer mean residence time of the particles. Consequently, the

mixing properties of the reactor may be significantly improved. A quantitative

analysis of this phenomenon and others related to impinging streams is presented in

the following examples, where the major aim is to determine the behavior of the

particles by introducing a pulse of these into one of the reactors.

In section 4.5, the following designations are made: Qi, Qy and qy are the

mass flow rates of the particles stream (kg particles/sec) and Ci is the concentration

of the tracer particles (kg tracer particles/kg particles). In such systems, the tracer

particles are, usually, those of the original ones. They are, however, made

radioactive or are painted, in order to distinguish them from the original particles.

The latter makes it possible to determine their concentration versus time in the RTD

experiments [73, p. 176], thus their mean residence time tm in the system, tm =

Page 470: Applications of Markov Chains in Chemical Engineering

464

Vi/Qi in a perfectly mixed reactor. An additional key quantity is the holdup of the

particles in the reactor, Vi (kg particles); instead of the volume of the reactor.

Thus, in consistent units, Eqs.(4-1) to (4-11) hold. An important quantity used in

this section for comparing various effects is the mean residence time tm of the

particles in the system.

4,5-1 The simplest model [73, p. 180] of a two impinging-stream reactor is shown

schematically in Fig.4.5-1. On the LHS is demonstrated the actual configuration of

the reactor and on the RHS the Markov-chain model. The latter employs the

following considerations and assumptions:

a) There are three main zones in which mixing of particles takes place: zones

1, 2 and 3. These zones are considered to behave like perfectly mixed vessels, i.e.

states.

b) The lower part of the reactor behaves like a plug flow reactor in which no

back mixing takes place. This is designated as tp in Fig.4.5-1.

c) The time needed for a particle to pass from one vessel to the other is equal

to zero. This is true since the actual borders of one vessel overlap with the

neighboring one. On the other hand, a particle might stay in one of the above zones

during a finite time.

d) The holdups, Vi, of the particles in each model vessel are the same.

The resulting model is shown on the RHS of Fig.4.5-1. Clearly each vessel

represents a state in a Markov process; vessel 4 is the collector of the particles from

which only the carrying stream is leaving at flow rate 2Qi. Q12, Q2i» Q23 ^^ Q 32

are recycle streams. This is due to the penetration of particles from one stream into

the other where the impingement zone is designated by 2 in Fig.4.5-1 and is

simulated as a perfectly mixed reactor. The effect of penetration is emphasized by

the fact that movement of particles to vessel 4 is only possible from vessels 1 and

3. Whenever a particle reaches vessel 4, it remains there, i.e. the vessel is a

trapping state.

Page 471: Applications of Markov Chains in Chemical Engineering

465

Qi ^o i f^ ( =q

'* 21 %

i71£~sl

I Fig.4.5-1. A model for a single stage two impinging-stream reactor

Noting that the residence time in the plug flow reactor is tp yields the

following matrices:

0 < t <ti

1

P = 2

3

Considering reactor j in Fig.4-1, hence j = l , a = 2, b = 3 and ^ = 4, as well

as Eqs.(4-12a) to (4-12c), and taking Qi as a reference flow, yields:

1 + CX3 = Pi4 + P34,1 + ^21 = ai2 + Pi4, for i = 2: an + OL32 = a2i + a23,

i = 3: as + a23 = a32 + p34

The following assumption were made:

OC12 = CX21 = a23 = OL32 = R, as = a i = 1; this yields that P14 = P34 = 1

The holdups of particles in all reactors are the same, i.e. m = ^l, i = 1,..., 4

From Eqs.(4-14) to (4-17b) the following probabilities were obtained:

For 0 < t < tp: p n = P33 = 1 - R|iiAt pi2 = P21 = P23 = P32 = RM'At

p-1

Pll

P21

0

2

P12

P22

P32

3

0

P23

P33

tp<t:

1

P= 2

3

4

1

Pll

P21

0

0

2

P12

P22

P32

0

3

0

P23

P33

0

4

P14

0

P34

1

Page 472: Applications of Markov Chains in Chemical Engineering

466

P22 = 1 - 2R^At

For tp< t : pii = P33 = 1 - (1 + R)|iAt

P14 = P34 = ILiAt

where the rest of the probabiUties are as for 0 < t < tp.

As seen, the parameters of the solutions are: |LI, R and tp. In the numerical

solution it was assumed that tp = 1 corresponding to At = 0.005, |LI = 10 and that

C(0) = [1, 0, 0, 0], i.e. the pulse was introduced into reactor 1 in Fig.4.5-1, RHS.

The effect of the recycle R, 0, 0.1, 1 and 10 is demonstrated in Fig.Fig.4.5-la. It

is observed that by increasing R, the central reactor 2 becomes active with respect

to the response of the pulse input in reactor 1 and the system of reactors 1, 2 and 3

behaves as a single perfectly-mixed reactor for R = 10.

u

1

0.8

0.6

0.4

0.2

0

_

-

-

i = l

2,3,4

1 • " 1 / \ 1

1/ 1/

»\

1 \

; V_

4 1

_J R = 0

-\

1,2,3 t i l l

u

0.5

0 0.5 1 1.5 t

\ l

L 2 -""^

1 1

4 ^ ' ' ' " '

/ / 1

" 1 / R = 0.1

- ^ _ J 1 1 1

2.5 0 0.5

1

0.8

0.6

0.4

0.2

0

r.^

_ \

h V 1

1

1

y

/ 4'

\

~ \ • V _ 1 1

R = l I

_

1 1 1.5

t 2.5 0 0.5 1 1.5

t

Fig4.5-la. Ci versus t demonstrating the effect of R

Page 473: Applications of Markov Chains in Chemical Engineering

467

4,5-2 The reactor [82; 73, p. 188] depicted schematically in Fig.4.5-2 is a modified

form of the original two impinging-stream reactor described in Fig.4.5-1 on the LHS with two additional air streams located below the upper streams where particles are introduced. A brief description of the vessels-flows model in Fig.4.5-2, is as follows. The inlet pipes to the reactor are simulated by two plug flow reactors 1 and 5. The entrance of the particles to the reactor are followed by three zones 2, 3 and 4, simulated by perfectly-mixed vessels. Particles leaving vessels 2 and 4 enter another mixing zone, designated as a perfectly mixed vessel 6, formed by the secondary air stream. The particles leave this reactor through a tubular reactor, where the time needed for a particle to pass from one vessel to the other is zero but it is finite for staying in the reactors. The recycle streams between vessels 2,3,4 simulate the harmonic motion of the particles.

unruji

Fig.4.5-2. An impinging-stream reactor with two pairs of tangential air feeds

Page 474: Applications of Markov Chains in Chemical Engineering

468

Case a: tpi > 0

Noting that the residence time in the plug flow reactors is tp and tpi, yields

the following matrices:

0 < t <tp >•

1

1

P= si 1

0

tp + tpi<t:

1

1

2

3

P = 4

5

6

7

Pll 0

0

0

0

0

0

5

0

1

2

P12

P22

P32

0

0

0

0

3

0

P23

P33

P43 0

0

0

4

0

0

P34

P44

P54

0

0

tp

p

5

0

0

0

0

P55

0

0

< t < t p

1

2

= 3

4

5

6

6

0

P26

0

P46 0

P66

0

+ tpi

1

Pll 0

0

0

0

0

7

0

0

0

0

0

P67

1

i:

2

P12

P22

P32

0

0

0

3

0

P23

P33

P43

0

0

4

0

0

P34

P44

P54

0

5

0

0

0

0

P55

0

6

0

P26

0

P46

0

1

Considering reactor j in Fig.4-1, hence j = l ,a = 2, b = 3, c = 4, d = 5, e =

6 and ^ = 7, as well as Eqs.(4-12a) to (4-12c), and taking Qi as reference flow,

yields:

1 + as = p67,0C12 = 1, for i = 2: an + a32 = OL23 + a26,

i = 3: a23 + 0643 = a32 + a34, i = 4: a54 + a34 = a43 + a46,

i = 5: as = as4, i = 6: a26 + OC46 = P67

The following assumption were made:

as = ai = 1, thus, as4 = 1 and P67 = 2

^23 = ^32 = a43 = a34 = R, thus, a26 = a46 = 1

yielding the following probabilities:

Page 475: Applications of Markov Chains in Chemical Engineering

469

Pll = l

P22= 1

P33= ]

P44= 1

P55= 1

P66= 1

[ - mAt [ - ( 1 +R)|Ll2At

[ - 2R|Ll3At

[-(l+R)|X4At

L - JlsAt

I - M'6At

P12 = ^2At

P23 = R^At P26 = ^6At

P32 = R|lt2At P34 = R|Ll4At

P43 = R|Li3At P46 = MeAt

P54 = |l4At

P67 = |l7At

In the numerical solution it has been assumed that [i[ = |X, yielding the

following parameters: \i, R, tp and tpi. In addition, tp = 0.2, tpi = 0.4

corresponding to At = 0.001 and that C(0) = [1, 0, 0, 0, 0, 0, 0], i.e. the pulse

was introduced into reactor 1 in Fig.4.5-2. The effect of the recycle R, 0 and 10 is

demonstrated in cases a, b and c, d for p, = 1 and 50, respectively, in Fig.4.5-2a.

The effect of i = 1, 50 is demonstrated in cases a, c and b, d, respectively.

u

1

0.8

0.6

0.4

0.2

0

^

p

L 1 1 1

R = 0 i = H

"^-- .^1

6 - 7

" r 3-5 "i 0 0.2 0.4 0.6 0.8

t

(b)|

k

\ R=10 "-\^^ ^i=H

^^ ' '--. i 1

2 3 J

1.2 0 0.2 0.4 0.6 0.8 t

1.2

U

I

0.8

0.6

0.4

0.2

0

-

i = l

....

~1 1 / 1 / 1 ^

:/Y.2

'l ~ 1

6 ;

\

1

(cJJ

R = 0 H 1 = 50

-

1

p

L

1

1

1 1

1 fwl 1 5 1

6

\

1,.„

. (411

R = l o H 1 = 50J

1 0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6

t t Fig.4.5-2a. Ci versus t demonstrating the effect of R and |X

Page 476: Applications of Markov Chains in Chemical Engineering

470

Case b : tpi = 0

In this case, the plug flow reactor before the exit from the system in Fig.4.5-

2, is absent. The results for identical parameters corresponding to case a are

depicted in Fig.4.5-2b. It was observed that for |j, = 1, the results for the present

case coincide with the results in case a above in the investigated range of t. The

results in cases a and b below for i = 6, 7 differ from the results in cases c and d

above due to the effect of tpi.

1

0.8

0.6

0.4

0.2

0

-

-

-

i = l

1

1 1 1 A,

2;/!'

1

>6 VLi-.

1

(a)J 7 1

-J R = 0 1 ^ = 5CM

-

1 1

-

-

p

1

1

I

i 1 ; 1

A 6 2-4

1 5 1 1

(bjl 7 1

R = 1 0 ^ ^ = 50 j

-

1 0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6

t t Fig«4.5-2b. Ci versus t demonstrating the effect of R and |X

As indicated before, a possible increase in the mean residence time tm of the

particles might be anticipated due to penetration of particles into the opposed

stream, consequently undergoing multiple circulation followed by damped

oscillations. This behavior is controlled in the above model by the quantity R, i.e.

the recycle stream, and is demonstrated as follows for extreme cases.

R = 0:

In this case the effective reactors are 1, 2 and 6. Excluding reactor 7, yields

the following equation for the total mean residence time in the reactors for a pulse

introduced into reactor 1:

V i V. V^ ^ - n l + ^ + W " m6 "*• tpi - "Q" " tp " Q " " 2 Q " V

Assuming that the holdup V of the particles in the reactors is the same, thus \i =

Ql A , yields that:

Page 477: Applications of Markov Chains in Chemical Engineering

471

tm = tp + tpi + | (4.5.2a)

R-»oo:

In this case all reactors are effective and reactors 2, 3 and 4 behave as a

single reactor of holdup V2 + V3 + V4 , thus

V^ V2 + V3 + V4 _V6_

*™ Qi """^"^ 2Qi •*" 2Q, • • Pi

or alternatively

t™ = tp + tp, + | (4.5-2b)

Eqs.(4.5-2a) and (4.5-2b) indicate that the relative increase in tm, due to the recycle

R, with respect to its value for R = 0 is 16.67%.

4>5-3 As indicated at the beginning of this section, there is an increase of the

holdup of the particles at the impingement zone, resulting an increase in the mean

residence time of the particles in the system. However, inter particle collisions

might decrease the above effect as well as back flow of the particles. It is the aim

of this case, depicted in Fig.4.5-3, to investigate these effects. Reactors 1 to 5,

demonstrate the impingement zone and reactor 3 with the highest concentration,

demonstrates the impingement plane. Reactor 6 is the exit of the particles from the

impingement zone and reactor 7 is the collector of the particles.

Page 478: Applications of Markov Chains in Chemical Engineering

472

'Miisjte^rftyfefi-^ I X I

% H 6

^ = 7 Q1+Q5

Fig.4.5-3. A scheme for demonstrating characteristics of impinging streams

The following matrix is applicable:

P =

1

2

3

4

5

6

7

1

Pll

P21

0

0

0

P61

0

2

P12

P22

P32 0

0

0

0

3

0

P23

P33

P43 0

0

0

4

0

0

P34

P44

P54

0

0

5

0

0

0

P45

P55

P65

0

6

P16

0

P36 0

P56

P66

0

7

0

0

0

0

0

P67

1

Considering reactor j in Fig.4-1, hence j = l,a = 2, b = 3, c = 4, d = 5, e = 6 and ^ = 7, as well as Eqs.(4-12a) to (4-12c), and taking Qi as reference flow, yields:

1 + CX5 = P67 1 + a6i + a2i = ai2 + ai6

for i = 2: an + a32 = a2i + a23, i = 3: a23 + OL43 = a32 + a34 + a36,

i = 4: a34 + a54 = a43 + a45, i = 5: as + a45 + a65 = ^54 + a56,

i = 6: ai6 + ase + a56 = 0C6i + a65 + p67 Assuming that:

as = a i = 1, a32 = a34 = a2i = a45, ai2 = a54 = a23 = OC43,

Page 479: Applications of Markov Chains in Chemical Engineering

473

ai6 = 0156, Ot61 = a65

and taking the following quantities as known, designated by

a2i = R. a36 = a, ai6 = Y

yields that:

«i2 = CC54 = a23 = CC43 = R + a/2

061 = a65 = Y+a /2 -1

a56 = Y

a32 = 034 = 045 = R

P67 = 2

Considering the above coefficients and Eqs.(4-14) to (4-19), yields the following

probabilities:

P12 = (R + a/2)mAt

P21 = R iiAt

P32 = RM.2At

P16 = Y|A6At

P23 = (R + a/2)H3At

P34 = R^4At

P43 = (R + a/2)H3At

P54 = (R + a/2)|A4At P45 = R|A5At

P56 = Y 6At

P6i = (Y+a/2-l)|XiAt i = l , 5

pil = 1 - (R + a/2 + Y)|iiAt

P22=l- (2R + a/2) i2At

P33 = 1 - (2R + a)^3At

P36 = aM.6At

P44 = 1 - (2R + a/2)|i4At

P55=l - (R + a/2 + Y)|X5At

P66= l-(2Y+a)|X6At

P67 = 2 i7At

Because of symmetry, |li = fis = |4., |i2 = 1 4 = \i', yielding the following

parameters: |a,i (i = 3, 6, 7), |i, |i', R, a and Y- The mean residence time in the

system (Fig.4.5-3), for a pulse introduced into reactor 1 and considering reactors 1

to 6, is determined below for R = 0 and R —> <».

R = 0:

Ignoring recycles Qgi and Qes, assuming that the holdups Vi = Vg, thus |i =

Ql/Vi, yields

•" Q, 2Q, 2n

Page 480: Applications of Markov Chains in Chemical Engineering

474

R-»«>:

^Yi V2+V3+V4 _ ^ ^ 2 . ± j _ ^ Qi"" 2Qi ^ 2 Q i " 2^1'" III'^2^3

where |LI' = QiA^2 = Q1/V4 and 113 = QiA^s. The above equation indicates that if

the holdup of the particles V3 in the impingement zone is increased, 113 is decreased

and hence tm is increased. The latter is an important property of the impinging-

stream configuration [73, p.3 and 138]. It has also been observed in the

calculations that increasing a = a36 and 7= ai6 = (X56 has a negligible effect on

tm. Finally, it should be noted that values of tm calculated by the above equations,

coincide with numerical values obtained by numerical integration according to

Eq.(4-26) for a pulse introduced into reactor 1 and collected in reactor 6.

Fig.4.5-3a demonstrates the effect of R = a32 = a34 = a45 = 0, 5, 25, 100

on the concentration profile for 7= a56 = 1 and constant jii = 1 (i = 1,..., 7) for a

unit pulse introduced into reactor 1.

Page 481: Applications of Markov Chains in Chemical Engineering

475

1

0.8

0.6

0.4

0.2

0

-

-

-

i = l

^ , •

1

R = 0

, . - • • ' • . . . . . . . ^ ' ^

2-5 1

7 '

1

-\

- i

J -|

___-

0 0.5 1 1.5 2 0 0.5 1 t t

1

0.8

0.6

0.4

0.2

0

-

ll-A 2

1 4,5 ^^6 " 1

R = 25

1

7

1

J

:

-

— -

0 0.5 1 1.5 2 0 0.5 1 1.5 t t

Fig.4.5-3a. Ci versus t demonstrating the effect of R

4,5-4 The major characteristic of impinging streams is the penetration of particles

from one stream into the opposite one through the impingement plane (Fig.4.5),

thus, increasing their meem residence time in the reactor as well as their relative

velocity with respect to the air. The scheme in Fig.4.5-4 demonstrates this effect in

the following way. Reactors 1 to 10 simulate the impingement zone of the particles

and in each reactor particles reside for some time. A pulse of particles introduced

in reactor 1 may be divided into three streams. One stream occupies reactors 3 to 5,

the other, reactors 6 to 8, and the third one will occupy reactors 9 and 10.

Eventually, the pulse accumulates in reactor 12 while passing reactor 11.

Page 482: Applications of Markov Chains in Chemical Engineering

476

^ OD ^ Ri ^

5 U - Q s

R,>| 6 U-|—I 7 ka I '-j 8 TRi ^lobi ^ o b i '

R3

11

I R3

Q,+Q5

Fig.4.5-4. A scheme for demonstrating the effect of different penetration distances in impinging streams

The following matrix is applicable for the configuration in Fig.4.5-4:

P =

1 1 2

3

4

5

6

7

8

9

10

11

12

1

Pll

P21

0

0

0

P61

0

0

P91

0

0

0

2

P12

P22

P32 0

0

0

0

0

0

0

0

0

3

0

P23

P33

P43

0

0

0

0

0

0

0

0

4

0

0

P34

P44

P54

0

0

0

0

0

0

0

5

0

0

0

P45

P55 0

0

P85

0

P10,5

0

0

6

P16

0

0

0

0

P66

P76

0

0

0

0

0

7

0

0

0

0

0

P67

P77

0

0

0

0

0

8

0

0

0

0

P58 0

P78

P87

0

0

0

0

9

P19

0

0

0

0

0

0

P88

P99

10

0

0

0

0

P5,10 0

0

0

P9,10

P10,9 PlO.lO

0

0

0

0

11

Pi.11 0

0

0

P5,ll 0

0

0

0

0

12

0

0

0

0

0

0

0

0

0

0

Pl l . l l Pll,12

0 1

The following simplifying assumptions were made: |Lii = |Li (i = 1,..., 12). For reactors 1, 2, 3,4, 5 all interactions are equal, i.e., Ri = ay. For reactors 1, 6,7, 8, 5 all interactions are equal, designated as R2. For reactors 1,9,10, 5 all interactions are equal, designated as R3.

Page 483: Applications of Markov Chains in Chemical Engineering

477

Considering Eqs.(4-12a) to (4-12c) and Eqs.(4-14) to (4-19), yields the

following probabilities:

Pll = P55 = 1 - (1 + Rl + R2 + Rs^At

P12 = P21 = P23 = P32 = P34 = P43 = P45 = P54 = Rl|lAt

P16 = P61 = P58 = P85 = P67 = P76 = P78 = P87 = R2^At

P19 = P91 = P5,10 = P10,5 = P9,10 = P10,9 = R3^At

Pl,ll = P5,ll = ^At P22 = P33 = P44 = 1 - 2RmAt

P99 = PlO,10=l-2R3^At pii,ii = l-2^At pii,i2 = 2^At

As seen the parameters of the solution are: |X and Ri, R2 and R3.

The mean residence time in the system (Fig.4.5-4), for a pulse introduced

into reactor 1 and considering reactors 1 to 11, is determined below for Ri = 0 and Ri^oo .

Ri = 0:

In this case only reactors 1 and 11 are effective for a unit pulse introduced into

reactor 1. Assuming that the holdups Vi = Vn, thus |LL = QiA^i, yields

^ Q i ^ 2 Q i 2ji

Ri-">oo:

In this case reactors 1 to 11 are effective. Assuming the same holdup of particles in

all reactors, Vi = V, where |LI = QiA , yields

^ Vi + ...-f Vio Vii _ I I V ^ 11 ^ " 2Qi " 2Qi " 2Qi "" 2^

The above equations indicate that the relative increase of tm due to the recycles Ri,

with respect to tm for Ri = 0, amounts to 266.7%.

Fig.4.5-4a demonstrates typical response curves to a unit pulse input

introduced into reactor 1 for Ri = 10, R2 = 5, R3 = 1, M, = 10 and At = 0.00005.

Page 484: Applications of Markov Chains in Chemical Engineering

478

0.01 0.02 0.03 0.04 0.05

Fig.4.5-4a. Ci versus t

4,5-5 The following schemes demonstrate configurations comprising of 2, 3 and 4

impinging streams. The effect of the number of impinging streams on the mean residence time of the particles in the system will be investigated below.

Page 485: Applications of Markov Chains in Chemical Engineering

479

case a: 2-impinging streams case b: 3-impinging streams

%,T±\. .ri^iJi ^ ^ R R

Mil ^

I [33

^ = 5

case c: 4-impinging streams

case d: 2-impinging streams case e: 2-impinging streams

apt! R *^ Ir* ±

*lS

'?(2Q| 5-4

'p.l

s 'SfiS

^ = 4

Fig.4.5-5. The effect of the number of impinging streams

Page 486: Applications of Markov Chains in Chemical Engineering

480

where R = ay = Qi/Qi

Case a: 2-impinging streams

The following matrix applicable for case a in Fig.4.5-5 is:

P =

1

2

3

4

5

1 2 3 4 5

Pll P12 0 P14 0

P21 P22 P23 0 0

0 P32 P33 P34 0 0 0 0 P44 P45

0 0 0 0 1

Considering Eqs.(4-12a) to (4-12c) and assuming that |ii = fi (i = 1,..., 5),

a i2 = a2i = a23 = a32 = R and that au = a34, yields from Eqs.(4-14) to (4-19),

the following probabilities:

Pll = P33 = 1 -(1 + R)^At P22 = 1 -2R^At

P14 = P34 = M t P21 = P12 = P23 = P32 = R^At

P44 = 1 - 2nAt P45 = 2nAt

Fig.4.5-5a demonstrates the effect of the recycle R for |i = 1 and At = 0.001

for a unit pulse introduced into reactor 1. It is observed that for R = 50, reactors 1,

2 and 3 behave as a single reactor due to the relatively high recycle.

R = 50 , . - - i

• • ' • • H 1 5 . - " 1

• •

• •

• 1 1»2,3 / J \ \ 1 m 1 il / •

l i \ / m 1 \ ^

1 '/ 1 T^--f--^H-^ 5 -1

Fig.4.5-5a. Ci versus t demonstrating the effect of R

Page 487: Applications of Markov Chains in Chemical Engineering

481

Case b: 3-impinging streams

The following matrix is applicable for case b depicted in Fig.4.5-5:

P =

1

2

3

4

5

6

1

Pll

P21

0

0

0

0

2

P12

P22

P32

P42 0

0

3 0

P23

P33 0

0

0

4 0

P24

0

P44 0

0

5

P15 0

P35

P45

P55 0

6 0

0

0

0

P56 1

Considering Eqs.(4-12a) to (4-12c) and assuming that (Xj = [i (i = 1, ..., 6),

«12 = a21 = 0123 = "32 = ^24 = 042 = R, a i s = 035 = 045, yields from Eqs.(4-

14) to (4-19), the following probabilities:

Pll = P33 = P 4 4 = l - ( l + R)fAAt p22=l-3RfiAt

P12 = P21 = P23 = P32 = P24 = P42 = R^At pi5 = P35 = P45 = ^At

P55 = 1 - SfiAt P56 = 3|iiAt

Fig.4.5-5b demonstrates the effect of the recycle R for fi « 1 and At = 0.005

for a unit pulse introduced into reactor 1. It is observed that for R = 50, reactors 1

to 4 behave as a single reactor due to the relatively high recycle.

Fig.4.5-5b. Ci versus t demonstrating the effect of R

Page 488: Applications of Markov Chains in Chemical Engineering

482

Case c: 4-impinging streams

The following matrix is applicable for case c depicted in Fig.4.5-5:

P =

1

2

3

4

5

6

7

1

Pll

P21

0

0

0

0

0

2

P12

P22

P32

P42

P52 0

0

3

0

P23

P33 0

0

0

0

4

0

P24

0

P44 0

0

0

5

0

P25

0

0

P55 0

0

6

P16 0

P36

P46

P56

P66 0

7

0

0

0

0

0

P67 1

As before, the following probabilities are obtained:

Pll = P33 = P44 = P55 = 1 - (1 + R)fiAt P22 = 1 - 4R^At

P12 = P21 = P23 = P32 = P24 = P42 = P25 = P52 = R| At

P16 = P36 = P46 = P56 = I At

P66 = 1 - 4jlAt P67 = 4|xAt

where R = ai2 = a2i = a23 = a32 = CX24 = 0C42 = OC25 = OL52

Fig.4.5-5c demonstrates the effect of the recycle R for |i = 1 and At = 0.005

for a unit pulse introduced into reactor 1. It is observed that for R = 50, reactors 1

to 5 behave as a single perfectly-mixed reactor due to the relatively high recycle.

1

0.8

0.6

0.4

0.2

0

\ R = l

L. \ y \i=l 7 '

\ 3,4,5

H

-.-.u. -1

[-

h-

R = 50

1 7.

/

1-5 / '

-j

-

1

5 -1

Fig.4.5-5c. Ci versus t demonstrating the effect of R

Page 489: Applications of Markov Chains in Chemical Engineering

483

Case d: 2-impinging streams

This is another scheme of 2-impinging streams depicted in Fig.4.5-5 case d

where part of the impingement zone is a plug flow reactor. Noting that the

residence time in the plug flow reactor is tp yields the following matrices:

0 < t < t

1

P = 3

4

2tp<t :

1

2

P = 3

4

P-1

Pll 0

0

1

Pll

1 P21

0

0

3

P13

P33 0

2

P12

P22

0

0

4

0

P34 1

3

P13

P23

P33 0

4

0

0

P34 1

tp < t < 2 tp:

1

1

2

P = 3

4

Pll 0

0

0

2

P12

P22 0

0

3

P13

P23

P33 0

4

0

0

P34 1

Considering Eqs.(4-12a) to (4-12c) and assuming that jii = |X (i = 1,..., 4),

o^l2 = o t2 i=R, oci3 = a23, yields from Eqs.(4-14) to (4-19) the following

probabilities:

ForO<t <tp:

pil = l - ^At Pl3 = ^At

P33 = 1 - 2|iAt P34 = 2|ilAt

For tp < t:

P l l = P 2 2 = l - ( l + R ¥ A t

P12 = P21 = R^At P13 = P23 = |XAt

P33 = 1 - 2 lAt P34 = 2p,At

Fig.4.5-5d demonstrates the effect of R, |LI and tp on the distributions Ci -1

for At = 0.005. Cases a and b show the effect of R; if R > 0 reactor 2 becomes

active after some time. Cases b and c depict the effect of \i and cases c, d

Page 490: Applications of Markov Chains in Chemical Engineering

484

demonstrate the effect of tp. In the latter case it is interested to note that reactor 2 is

inactive all the time due to the relatively high value of tp.

0

1

0.8

0.6

0.4

0.2

0

\ i = l

-/7

0.1 0.2 0.3 0.4 t

^1= 10 R = 5 t =0.1

4 /

9 ^i^^:^^--:^:^,.^.^ . -_

1 1 1 1

0.

(cjl

H

H

0.2 0.3 t

0.5

\ r \ L \

/

r '

^1= 10 R = 5 t =1

/ /

4/ /

3 " " " = ^ - - ^ 2 ^ —

"(dji

H

-J

•J

1 1 1 1 1 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5

t t

Fig.4.5-5d. Ci versus t demonstrating the effect of R, JLI and tp

Case e: 2-impinging streams In this scheme depicted in Fig.4.5-5 case e, an additional tubular reactor is

added, simulating the plug flow at the lower part of the reactor (Fig.4.5-1, LHS) in which no back mixing takes place. The residence time in the plug flow reactor simulating the impingement zone is tp and of the lower part it is tpi. In the following we consider the interesting case where tpi > tp; a unit pulse introduced into reactor 1, yielding the following information:

For 0 < t <tp, and because tpi > tp, no change in concentration takes place in the system.

For tp < t < 2tp (< tpi), there is a transfer of the tracer from reactor 1 to 2 only through the tubular reactor. As a result of this situation Qi = 0 and there are only

Page 491: Applications of Markov Chains in Chemical Engineering

485

circulation flows, i.e. Q12 = Q21. However, when t > tpi, Qi > 0. We define R

with respect to this flow, i.e. R = Q12/Q1 = OC12 = a2l- Under this condition, the

following matrix holds:

P =

1

Pll 0

P12 1

where pn = 1 - R|LiAt pi2 = R|xAt

For 2tp < t < 3tp (< tpi), the following matrix holds:

P =

1 2

Pll P12

P21 P22

where pi 1 = p22 = 1 - R|LiAt pi2 = p2i = R^At

For tpi < t, the following matrix holds:

1

2

3

4

1

Pll

P21 0

0

2

P12

P22 0

0

3

P13

P23

P33 0

4

0

0

P34 1

p =

where P l l = P 2 2 = l - ( l + R ¥ A t

P12 = P21 = R^At P13 = P23 = |iAt

P33 = 1 - 2|i,At P34 = 2|LlAt

As seen, the parameters of the solution are: R, \i, tp and tpi. Fig.4.5-5e

demonstrates the effect of the above parameters on the distributions Ci -1 for At =

0.00005 and 0.005. The following quantities were assigned for the parameters: tpi

= 5tp; tp = 0.01 and 1, R = 0, 1 and 10 whereas |i = 50, 100 and 500. Cases a, b

Page 492: Applications of Markov Chains in Chemical Engineering

486

and b, e show the effect of |LI; cases a, c depict the effect of tp and tpi; cases d, e, f

demonstrate the effect of R.

u

1

0.8

0.6

0.4

0.2

0

i = l

h-

h

X t =

1/^

/

\ \

= 0.01

:1

1

t Pl

= 0.05

50

(a

>^ /

/ , ' 4

1

t =0.01 1 \ '

F '^ /

I I I

t : pl

= 0.05

100

1 T

(bj

/ /

1 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070 0.01 0.02 0.03 0.04 0.05 0.06 0.07

u

1

0.8

0.6

0.4

0.2

0

i = l T""

\

t =1 t =5 p pl

R = l ^ = 50

1,2

3,4

4 1

•H

\ 1-3 1 1 1 1 1 1

0 1 3 4 t

1

0.8

0.6

0.4

0.2

0

1 1

i = l t :

- P

R =

-

2-4

= 0.01

= 0

t = pl

1 1

= 0.05 1 /

500 i ;

1; 1

'•iV

' (d)| 4 1

H

-\

1 1 1 1 1 2 1

1 i = 11

1 1 r i

1

1

\- ;

1 1

1 :'1\

• 1 \ \

t = p

R =

\

/

V

1

:0.01

:1

1

1 t = pl

3,4__ 1

1 = 0.05 .

500 '

1'

1

1 (e)l " "4 1

—j

1-3 1

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 t t

Page 493: Applications of Markov Chains in Chemical Engineering

487

o

1

0.8

0.6

0.4

0.2

0

~i"=T

-

\

• 2

1 1 I t=0.01 t =0.05 •

p pi R=10 ^ = 500 .

V

3,4 ^'V 1 1 1

4

-

1-3 1

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 t

Fig.4.5-5e, Ci versus t demonstrating the effect of R, \i, tp and tp i

Mean residence time tm of the impinging-stream systems in Fig 4.5-5

The comparison will be made for extreme cases, i.e. R = 0 (part of the

reactors are not interacting) and R —> ©<> (appropriate reactors are interacting and

considered as a reactor of volume equal to the total volume of the single reactors).

Note that R = Q12/Q1 = Q2l/Ql- The following examples elaborate the above for a

unit pulse introduced into reactor 1 and assuming that the holdups of the particles in

each reactor are the same, i.e. Vi = V, thus, |ii = |i = QiA^

Case c

R = 0: tm = tml + tm6 = Vj/Qi 4- V6/(4Qi) = l/^l + 1/(4^) = 5/(4^)

R -^ ^: tm = (Vi + V2 + V3 + V4 )/(4Qi) + V6/(4Qi) = 6/(4^)

Case d R = 0: tm = tml + tm3 = Vi/Qi + V3/(2Qi) = 3/(2^1)

R -^ «^: tm = tml + tp + tm2 + tm3

= Vi/(Qi + Q21) + tp + V2/(Q2 + Q12) + V3/(Ql + Q2)

for Q12 = Q21 and Qi = Q2 it is obtained that

tm = (5 + R)/(2^(l+R)) + tp

where tp = Vp/(2Qi2) = l/(2R^p) and ^p = QiA^p.

Page 494: Applications of Markov Chains in Chemical Engineering

488

Case e (tpi > tp)

R = 0: Fort<tpi tm = tpi

For t > tpi tm = tmi + tpi + tm3 = Vi/Qi + tpi + V3/(2Qi) =

3/(2^1)+ tpi

R-^oo: For t^ tpi tm = tpi

For t> tpi

tm = tml + tp + tm2 + tp + tm3 = (5 + R)/(2n(l+ R)) + tp + tpi

where tp is given above in case d, and tpi = Vpi/(2Qi) = l/(2|Xpi) and \ip\ =

QlA^pl Similarly, the following Table may be obtained for a pulse input introduced

into reactor 1:

Table 4.5-5. Expressions for the mean residence time tm in the reactor-system in Fig.4.5-5 (excluding the collector reactor %)

n-number of

impinging streams

2 (case a)

3 (case b)

4(case c)

1 n

2 (case d)

1 2 (case e)

t>tpi

R = 0 1

tin = (3/2)/^

(4/3)7^1 '

(5/4)/^

[(n+l)/n]/^l

(3/2)7^1

(3/2)/^ + tp

R ^ o o

tn, = (4/2)/jl

(5/3)/^

(6/4)/|X

[(n+2)/n]/n R > 0 tm =

(5+R)/(2|x(l+R)) + t„

1 R>0 1 tn,=

(5+R)/(2n(l+R)) 1 +tp + tDl 1

tp and tpj are given in cases d and e above.

It should be emphasized that the value of tm calculated by the above

equations, coincides with numerical values obtained by numerical integration

according to Eq.(4-26) for a pulse introduced into reactor 1 and collected in reactor

Page 495: Applications of Markov Chains in Chemical Engineering

489

The following ratios may be obtained from Table 4.5-5:

,R->oo n + 2 a i =

tm,R=0 ^ + 1

ai demonstrates the effect of the recycle R on increasing tm and it approaches 1 for

large n. Maximum ai = 4/3 for a 2-impinging-stream reactor (n = 2).

2 Vn (n+1) ^ ^ r^

tto = z = "7 :^ for R = 0 tn, n+i n(n + 2)

tm,n (n-H)(n + 2) . ^ aa = 7 = 7 -T— for R -» oo

^ Vn+i n(n + 3)

a2 and as show the effect of the number of streams n on the mean residence time

ratio for R = 0 and R -> <». It may be concluded that increasing n causes the ratios

to approach unity. The maximum ratio is obtained for n = 2, yielding a2 = 9/8 =

1.125 and as = 12/10 =1.2

4>5-6 Fig.4.5-6 shows a single stage four impinging-stream reactor [73, p. 186].

On the RHS is schematic of the reactor and on the LHS is the model of reactors and

flows. It is assumed to comprise eight perfectly-mixed reactors designated as 1,

..., 8. All reactors are assumed to have equal holdups. The model assumes also

tangential feed of the streams and that more mixing zones are generated by the

direct impingement of the streams. Recycle occurs through intermediate perfectly-

mixed reactors 2,4, 6, 8 where impingement is expected.

Page 496: Applications of Markov Chains in Chemical Engineering

490

is ?L^r^^<^^M^ 5 rv^

y

Fig.4.5-6. A model for a single stage four impinging-stream reactor

For the configuration in Fig.4.5-6, the following matrix holds:

1

2

3

4

5

6

7

8

9

10

1

Pll

P21

0

0

0

0

0

P81

0

0

2

P12

P22

P32

0

0

0

0

0

0

0

3

0

P23

P33

P43

0

0

0

0

0

0

4

0

0

P34

P44

P54 0

0

0

0

0

5

0

0

0

P45

P55

P65

0

0

0

0

6

0

0

0

0

P56

P66

P76

0

0

0

7

0

0

0

0

0

P67

P77

P87

0

Os

8

P18

0

0

0

0

0

P78

P88

0

0

9 10

P19 0

0 0

P39 0

0 0

P59 0

0 0

P79 0

0 0

P99 P9,I0

0 1

Considering reactor j in Fig.4-1 and making the following designations: j = 9, a = 1, b = 2,..., Z = 8 and ^ = 10. Applying Eqs.(4-12a) to (4-12c), taking Qi as reference flow and making the following assumptions, i.e.

ay = a5 = as = ai = 1

Page 497: Applications of Markov Chains in Chemical Engineering

491

R = ai2 = a2i = a23 = OC32 = OC34 = a43 = a45 = OC54 = a56 = a65 = a67 =

a76 = OC78 = ag? = CX18 = agi

yields that

0C19 = a39 = a59 = a79 as well as p9,io = 4

Assuming also that all reactors have the same holdup of particles, i.e. Vi = V (i =

1, ..., 10) or |ii = ^1= QiA^, and applying Eqs.(4-14) to (4-19), yields the

following probabiUties:

Pll = P33 = P55 = P77 = 1 - (1 + 2R)|LlAt

P22 = P44 = P66 = P88 = 1 - 2R|XAt

P12 = P21 = P23 = P32 = P34 = P43 = P45 = P54 = P56 = P65 = P67 = P76 = P78

= P87 = P18 = P81 = R^At

P19 = P39 = P59 = P79 = |LiAt p99 = 1 - 4^At P9J0 = 4|lAt

As seen the parameters of the solution are |Li and R.

The mean residence time in the system (Fig.4.5-6), for a pulse introduced

into reactor 1 and considering reactors 1 to 9, is determined below for R = 0 and R - ^ 0 0 .

R = 0: Assuming that the holdups Vi = V9, thus |X = QiA^i, yields

tm = Vi/Qi+V9/(4Qi) = (5/4)/^

R->oo:

tm = (Vi + ... + V8)/(4Qi) + V9/(4Qi) = (9/4)/^

The above equations indicate that the relative increase in the mean residence time

with respect to R = 0, due to the recycles, is 80%.

In the numerical solution the effect of the recycle R, 0, 2 and 50 and of \i, 1

and 0.1, is demonstrated in Fig.4.5-6a for At = 0.005 and a unit pulse introduced

into reactor 1 in Fig.4.5-6. Note that due to symmetry C2 = Cg, C3 = C7 and C4 =

C6. The effect of R is demonstrated in cases a, b and c; the effect of \i in cases c

and d.

Page 498: Applications of Markov Chains in Chemical Engineering

492

1

0.8

0.6

0.4

0.2

0

1 \"i=l

- \ /

- X ~ J .9 j^_2;8_

1

1 1 1 0 , . • • • - - " "

1 1

' (a)l

-

| X = 1 - 1 R = 0 J

-

T" ""

1

0.8

0.6

0.4

0.2

0

1

-

1

i = l

1

1

iQ-

1

1 (c^

^i=lJ R = 50

A

1

10. . . - --

^=i i R = 2

1 2 3 4 5 0 1 t

-0.5 0 0.5 1 1.5 t

2 - 1 0 1

Fig.4.5-6a. Ct versus t demonstrating the effect of R and p.

4,5-7 An extension of the single-stage two impinging-stream reactor to a 3-stage

reactor [73, p.l95], and similarly to a multi-stage reactor, is demonstrated in Fig.4.5-7. The reactor is composed of sections separated by plates a, with appropriate openings c, d etc. Two successive plates and their top views A-A' and B-B' are shown. Between two plates there is a partition designated by b. The partition is dividing the gas-particle stream so that the impinging-stream effect is maintained at each of the reactor's stages. In order to operate the multi-stage reactor, two gas-solid streams are fed to the top of the reactor. The streams collide in the impingement zone 2 (Fig.4.5-7, top view). The combined gas-solid stream formed after the impingement enters through opening c and flows downwards where it is divided into equal horizontal streams by the partition b. The streams impinge above opening d and enter the next stage where they are divided again by

Page 499: Applications of Markov Chains in Chemical Engineering

493

partition b. Eventually, the solid particles leave at point e and the gas exits through pipe f located at the reactor's exit.

The vessel-flow arrangement which is modeled, is shown on the RHS of Fig.4.5-7. The n stages may be envisioned as n identical reactor segments, shown schematically in Fig.4.5-1, LHS. Each segment consists of four mixed vessels where the multi-stage reactor is terminated by a plug flow reactor. The inlet pipes are also simulated by plug flow reactor. It is assumed that the transition time from one stage to the other is negligible as compared to the time spent in the mixed vessel.

top view

single-stage reactor zone

Fig.4.5-7. Structure and model of a 3-stage two impinging-stream reactor

Page 500: Applications of Markov Chains in Chemical Engineering

494

The transition probability matrix for the 3-stage reactor is given in the

following and can easily be extended to a multi-stage reactor. For tp < t < tpi the

matrix is of 12 by 12 since reactor 13 is yet inactive whereas for tpi < t, it reads:

P =

1

2

3

4

5

6

7

8

9

10

11

12

13

1

Pii

P21

0

0

0

0

0

0

0

0

0

0

1 0

2

Pl2

P22

P32

0

0

0

0

0

0

0

0

0

0

3

0

P23

P33

0

0

0

0

0

0

0

0

0

0

4

Pl4

0

P34

P44

0

0

0

0

0

0

0

0

0

5

0

0

0

P45

P55

P65

0

0

0

0

0

0

0

6

0

0

0

0

P56

P66

P76

0

0

0

0

0

0

7

0

0

0

P47

0

P67

8

0

0

0

0

P58

0

P77 P78

0

0

0

0

0

0

P88

0

0

0

0

0

9

0

0

0

0

0

0

0

P89

P99

Pl0,9

0

0

0

10

0

0

0

0

0

0

0

0

P9,10

PlO,10

Pll,10

0

0

11

0

0

0

0

0

0

0

P8,ll

0

PlO,ll

Pll,ll

0

0

12

0

0

0

0

0

0

0

0

P9,12

0

Pll,12

Pl2,12

0

13

0

0

0

0

0

0

0

0

0

0

0

Pl2,13

1

The derivation of the probabilities is similar to case 4.5-1. The following

probabilities were obtained assuming symmetry in the flows with respect to

reactors 2, 4, 6, 8, 10 and 12, and that all reactors have the same holdup of

particles, thus, |ii = |i, i = 1,..., 13:

Pll = P33 = P55 = P77 = P99 = Pll,ll = 1 - (1 + R)| At

P22 = P66 = P10,10 = 1 - 2R|lAt

P44 = P88 = 1 - 2|lAt

P12 = P21 = P23 = P32 = P56 = P65 = P67 = P76 = P9,10 = Pl0,9 = PlO,ll = Pll,10 = R| At

P45 = P47 = P89 = P8ai = ^At

P14 = P34 = P58 = P78 = P9,12 = Pl 1,12 = I At

whereas for tp < t < tpi: pi2,i2 = 1 and for tpi < t: pi2,i2 = 1 - 2|iAt pi2,i3 = 2|LiAt

Page 501: Applications of Markov Chains in Chemical Engineering

495

As seen, the parameters of the solutions are: |i, R, tp and tpi. In the

numerical solution it was assumed that tp = 0, tpi = 1,5 corresponding to At =

0.001 and 0.005, respectively. Other parameters are: R = 0, 1, 20, i = 1, 5 and

the unit pulse was introduced into reactor 1 in Fig.4.5-7, RHS. Fig.4.5-7a

demonstrates the relationship Q-t in which the effect of R is demonstrated in cases

a, b and c, the effect of |X is depicted in cases c and d whereas the effect of tpi in

cases dande. Note that due to synmietry Ci =€3, C5 =C7andC9 =Cii.

u

1

0.8

0.6

0.4

0.2

0

1

\ i = l - \

4 ^

/ 5 -

1

1 1 1 1 ^= 1 R = 0 t =5

12

, . • • - • • " \ , /

^ < S ^ '' 8

T2:ro:i3, , 7 ^ ' ^

1

1 3 . •

' ^ -..

1

(aj

'' 1

A

-|

1 : : V

Page 502: Applications of Markov Chains in Chemical Engineering

496

1

0.8

0.6

1 ] 1 r

i = l

1 r ^1= 1 R = 20 t = 5

(c)

12 13.''

U

0.6

0.4

0.2

0

. ' Li = 5 R = 20 t =5 i = l 12 ' ^ pi

2\r / / / 5 , 6 , 8

llJ^^. J.

9,10

J I L

0 1

Page 503: Applications of Markov Chains in Chemical Engineering

497

u

1

0.8

0.6

0.4

0.2

0

1

i = l

1 •A 5,6

2: V /

«

1 1 1 |i = 5 R = 20 t =1

pi , . . - - •

12 ,'13

/ ' ^ /

;". 9,10

1 1 1

' (e l

-

-

-

1 -0.5 0 0.5 2.5 1 1.5 2

t

Fig.4.5-7a. Ci versus t demonstrating the effect of R, |i and tpi

The mean residence time in the system (Fig.4.5-7), for a pulse introduced

into reactor 1 and considering reactors 1 to 12, is determined below for R= 0 and R->oo.

R = 0:

In this case reactors 1, 4, 5, 7, 8, 9, 11, and 12 are effective for the pulse

introduced into reactor 1. Assuming that the holdups of the reactors are identical,

i.e. Vi = V, thus |Li = QiA^, yields:

tm = tp + tml + tni4 + tm5 + tm7 + tm8 + tni9 + tmll + tml2 + tpi

= tp +tpi+(13/(2^1)

R ^ o o :

In this case reactors 1 to 12 are effective, and reactors 1-2-3, 5-6-7 and 9-10-11

may be considered as single reactors due to the recycle R. The above assumption is

valid also here, thus it follows that:

tm = tp + tpi + (15/(2^)

The above equations indicate that the relative increase of tm due to the recycles R,

with respect to tm for R = 0, and ignoring tp and tpi is 15.4%.

Page 504: Applications of Markov Chains in Chemical Engineering

498

Chapter 5

APPLICATIONS OF MARKOV CHAINS IN CHEMICAL PROCESSES

The major objective of this chapter is to demonstrate how Markov chains can

be appUed to determine the transient behavior of comphcated open systems

undergoing simultaneously heat and mass transfer processes as well as chemical

reactions. It is an extension of chapter 4 in which the RTD of a complicated system

was investigated.

5.1 MODELING OF THE PROBABILITIES The model. A multi-component and multi-reactor system, arranged

according to the general model depicted in Fig.5-1, is considered, which extends

the scheme in Fig.4-1. It covers numerous flow arrangements and processes

encountered in Chemical Engineering.

The general scheme consists of a central reactor designated by j and

peripheral reactors a, b,..., Z. A process may terminate at the exit of the peripheral

reactors, but the model makes it possible also to collect in reactor | the streams

leaving the peripheral reactors. In reactor % no chemical or physical processes take

place.

The flow system comprise the following flows: Qj, Qa, Qb> ••» Qz - flows

from feed vessels to reactors j , a, b, ..., Z. Qj, Q'a, Qb, •••» Qz - flows leaving

reactors j , a, b, ..., Z, outside. There are also interacting flows between the

reactors, i.e. each reactor is feeding all the others. Finally, flows qi^ are from each

reactor to the collector ^.

Each of the streams and the reactors may contain species f where f = 1,2,...,

F; the total number of species is F. The concentrations of the species at the exit of

the reactors, i.e., Cg, Cfi (i = a, ..., z) and C'f are not, in general, equal to the

concentrations inside the reactors, Cg, Cfi (i = a, ..., z) and Cf . If the

Page 505: Applications of Markov Chains in Chemical Engineering

499

concentrations C'fj, Cfi (i = a, ..., z) are equal to zero, such a flow configuration

simulates, for example, a concentration process. If the concentration of the species

at the exit of reactor ^ equals zero, C'f = 0, the species are completely accumulated

in reactor ^ which is considered as "total collector" or "dead state" for the species.

If C'f = Cf , the species are not accumulated in reactor ^. If 0 < C'f < Cf , the

species are partially accumulated in reactor ^ which is considered as a "partial

collector" of the species.

The basic element in the flow system is the perfectly-mixed reactor. In the

multi-reactor system heat and mass transfer operation (absorption, desorption,

dissolution of solids, heat generation or absorption as well as heat interaction

between the reactor and the surroundings etc.) as well as chemical reactions may

occur simultaneously, or not. The processes are governed by Eqs.(5-8), (5-12),

(5-16), (5-19), (5-23) and (5-25) in the following, on the basis of which transition

probabilities are derived as well as the single step transition matrix.

A specific configuration is determined by appropriate selection of the

interacting flows and the number of reactors as well as the operating conditions.

When the central reactor j is considered, usually, j = 1 and a = 2, b = 3, etc. As

indicated, the collector is designated by ^. When the central reactor is not

considered, usually, a = 1, b = 2, etc. Each reactor designates a state, thus, the

total number of reactors is the number of states.

Page 506: Applications of Markov Chains in Chemical Engineering

500

a) Overall scheme

b) Collector

Fig.5-1. Scheme of a continuous flow system

Page 507: Applications of Markov Chains in Chemical Engineering

501

Definitions. A^ = fk designates a system with respect to its chemical formula f

and location (reactor) k in the flow system. The system is a fluid element

containing F chemical species for which f = l , 2, 3, ...,F where each figure

designates a certain species with its chemical formula. In addition, k = a, b,..., Z,

where each letter designates a reactor in the flow system composed of Z + 1

perfectly mixed reactors, including reactor .

The states of the system are the following concentrations and enthalpies (or temperatures) of the different species inside the various reactors: a) The concentrations Cf nj of species f at the peripheral feed vessels to reactor j in Fig.5-1 as well as the specific enthalpy hinj (or Tinj) of the fluid in the feed vessel to j ; b) The concentrations Cf,in,k of species f at the feed vessels to reactors k and the specific enthalpy hin,k (or Tin,k) of the fluid in the feed vessels to k; c) The concentrations Cfk of species f in reactors k as well as the specific enthalpies hk, h (or Tk, T^), respectively, of the fluid in reactors k and in collector .

Thus, the state space SS for the system will read:

SS = [Ci^inj, C2,in,J5 C3^in,j, ..., Cp^nJ?

Cl,in,a»

Cl,in,b»

^l,in,z»

Clj,

Cla,

Clb,

Clz,

Ci^,

hinj?

hj.

C2,in,a>

C2,in,b»

C2,in,z»

C2j,

C2a>

C2b.

C2z,

C2^,

hin,a»

ha,

C3,in,a» •••»

C'3,in,b» •••>

C'3,in,z» •••»

C3j , ...,

C3a , ...,

C3b , ...,

C3z , ...,

C3^ , ...,

hin,b > •••»

hb , . . . ,

CF,in,a»

CF,in,b»

C'F,in,z>

Cpj,

Cpa,

Cpb*

Cpz,

Cp^,

hin,z»

hz, h ] (5-1)

where the total number of states, only if all above elements do exist, is (F+l)(2Z+3). Z is the number of perfectly-mixed reactors excluding the collector and F is the total number of species.

A compact form of the above equation for f = 1, 2, 3,..., F reads:

Page 508: Applications of Markov Chains in Chemical Engineering

502

SS = [Cf^inJ, Cfjn,a» Cf in,b» •••» C!f in,z>

Cfj, Cfa, Cfb , ..., Cfz, Cf ,

hinj» hin,a> hin,b » ••• hin,z?

hj, ha, hb ,..., hz, h ] (5-la)

If the system occupies a state, it means that the concentrations of species f in

all reactors are prescribed as well as the enthalpies of the fluid in the reactors.

Finally, the movement of a fluid element from a state to another is the transition

between the states. The initial state vector is given by Eq.(2-22), i.e.,

S(0) = [si(0), S2(0), S3(0), ..., sz(0)]

For the case under consideration it reads:

S(0) = [Ci,in,j(0), C2,inj(0), C3,in,j(0) , ..., CpanjCO),

Cl,in,a(0)» C2,in,a(0)» C3,in,a(0) , ..., CF,in,a(0).

Cl,in,b(0), C2,in,b(0), C3,in,b(0) , ..., CF,in,b(0)»

Cl,in,z(0), C2,in,z(0), C3,in,z(0) , ..., CF,in,z(0),

Cij(O),

Cia(O),

Cib(O),

Ciz(O),

Ci^(O),

hind(O),

hi(0),

C2j(0),

C2a(0),

C2b(0),

C2z(0),

C2^(0),

hin,a(0),

ha(0),

C3j(0) , ...

C3a(0) ,...

C3b(0) ,...

C3z(0) , ...

C3^(0) , ...

hin,b(0) ,...

hb(0) ,...

, CFJ(O),

, CFa(O),

, CFb(O),

, CFZ(O),

, CF^CO),

, hi„,z(0).

, hz(0).

The above equation is expressed compactly for f = 1,2, 3 , . . . , F by:

S(0) = [Cf.i„,j(0), Cf,in,l(0), Cf,in.2(0), ..., Cf,in,z(0),

Cfj(O), Cfa(O), Cfb(O) ,..., Cfz(O), Cf^(O),

hin,j(0), hi„,a(0), hi„,b(0) , ..., hin.z(0),

hj(0), ha(0), hb(0) ,..., hz(0), h4(0)] (5-2a)

Page 509: Applications of Markov Chains in Chemical Engineering

503

The state vector at step n, or time t is, similarly, given by:

Cl,inj(n),

Cl,in,a(n),

Ci,in,b(n),

Ci,in,z(n),

Cij(n),

Cia(n),

Cib(n),

Ciz(n),

Ci^(n),

hin,j(n),

hj(n),

C2,in,j(n),

C2,in,a(n),

C2,in,b(n),

C2,in,z(n),

C2j(n),

C2a(n),

C2b(n),

C2z(n),

C2^(n),

hin,a(n),

ha(n),

C3,in,j(n) , .

C3,in,a(n) , ..

C3,in,b(^) . ••

C3,in,z(n) , ..

C3j(n) , .

C3a(n) , .

C3b(n) , .

C3z(n) ,

C3^(n) ,

hin,b(n) ,

hb(n)

CF,inj(n),

CF,in,a(n),

CF,in,b(n),

CF,in,z(n),

CFj(n),

CFa(n),

CFb(n),

CFz(n),

CF^(n),

hin,z(n),

.., hz(n), h^(n)] (5-3)

A compact representation for f = 1, 2, 3,..., F is given by:

S(n) = [Cf4n,j(n), Cf,in,i(n), Cf.in,2(n),..., Cf,in,z(n),

Cfj(n), Cfa(n), Cfb(n) ,..., Cfz(n), Cf^(n),

hin,j(n), hin,a(n), hin,b(n) , ..., hin,z(n),

hj(n), ha(n), hb(n) ,..., hz(n), ^(n)] (5-3a)

The transition from step n to n+1 is carried out according to:

S(n+1) = S(n)P + L(n) (5-4)

where P is the one-step transition probability matrix given by Eq.(5-27). S(n) and

S(n+1) are state vectors at times t and t+At, or step n to n+1, given by Eq.(5-3).

L(n) is an expression corresponding to time t or step n which is spelled out in the

appropriate equations below. Specific elements of the state vector, corresponding

to Fig.5-1 are derived in the following and are given by Eqs.(5-9a), (5-13a), (5-

17a), (5-21a), (5-24a) and (5-26a).

Page 510: Applications of Markov Chains in Chemical Engineering

504

Scheme of the chemical reactions. In the derivations below, chemical

reactions are involved. To remain consistent with the nomenclature in chapter 3.1

and the modified nomenclature in this chapter, we repeat part of the material as

follows. Consider a chemically reacting system containing species Ai, A2, A3, ....,

Ap. It is assumed that a certain species Af can react simultaneously in several

reactions, designated in the following by superscripts (m), m = 1, 2, ..., R where

R is the total number of reactions in which Af is involved. The following scheme

of irreversible reactions by which Af is converted to products is assumed, where

each set of reactions involving reversible reactions can always be written according

to the scheme below in order to apply the following derivations.

7st reaction:

2nd reaction:

mth reaction:

Rih reaction:

Jl)

J2)

(1)

... + a . A^+ ... -> products (2)

... + a A^ + ... -» products (m)

dL Aj.+ ... + a. Aj. + . . .

JR)

- ^ " \ -(R)

... + a A^ + ... -* products (5-5)

M): a is the stoichiometric coefficients of species f in the mth reaction. The rate of

conversion of species f in the mth reaction based on volume of the fluid in the

reactor V, i.e. r , is defined by:

(m) _ J [ f V dt

d N ^ / V (m) (m) (m) (m) I V m ; ^ a , ^ a

= - k p q i q i ...c;f . . .c^ (5-6)

where Nf are the number of moles of species f and Cf is its concentration in moles

of f/m . k "*, in consistent units, is the reaction rate constant with respect to the

conversion of species f in reaction m. In the case of a plus sign before k , this

means (moles of f formed)/(s m^) in reaction m. The discrete form of Eq.(5-9)

reads:

/ x / \ (Msi (ad (in) On)

r ; % ) = -k;"'^c;i (n)q2 (n)... C f (n)... C^ (n) (5-6a)

Page 511: Applications of Markov Chains in Chemical Engineering

505

where the reaction rate and the concentrations Cf(n) refer to step n. In addition, the

conservation of the molar rates for all reacting species in reaction m in Eq.(5-5) is

given by:

(m)

Jm)

(m)

Jm)

- r , .(m)

(m) \

(5-7)

This makes it possible to compute the reaction rates of all species on the basis of

rf^ given by Eq.(5-6).

Derivation of probabilities from mass balances

Reactor j . A mass balance on species f undergoing various mass transfer

processes as well as chemical reactions due to changes in the operating conditions

in the central reactor j in Fig.5-1, gives:

d[V.C^]

dt ^ j f,inj Z-rf^kj fk

^Skfpo^j^S.i +

k

' ft J (5-8)

k = a, b, ..., Z where k t j , ^ f = 1, 2,..., F p = 1, 2, ..., P. The last

summation with respect to m, is on all reactions in Eq.(5-6) that species f is

involved with, i.e. m = 1,..., R.

Cf,in,j is the concentration of species f at the inlet stream Qj flowing from a

feed vessel into reactor j ; Cfk and Cg are, respectively, the concentrations of species

f in reactors k and j . C'fj is the concentration of f in the stream Q j leaving reactor

j -kfpjapjACfp j = ^fp jACfp jVj is the mass transfer rate of species f into (or from)

reactor j by some transfer mechanism designated by p (such as absorption,

dissolution, etc., or simultaneously by several processes). kfpj(m/s) is the mass

transfer coefficient for process p with respect to species f corresponding to

conditions in reactor j ; apj is the mass transfer area for process p corresponding to

conditions prevailing in reactor j . If the mass transfer area apj is not known, the

Page 512: Applications of Markov Chains in Chemical Engineering

506

volumetric mass transfer coefficient |LI .(1/S) defined in Eq.(5-10) is used. AC^ •

is the driving force for the transfer process p with respect to species f at conditions

prevailing in reactor j . It should be noted that a positive sign before kfpj indicates

mass transfer into the reactor whereas a negative sign indicates mass transfer from

the reactor outside, t^ is the reaction rate of species f by reaction m per unit

volume of reactor (or fluid in reactor) j corresponding to the conditions in this

reactor. A plus sign before k " in Eq.(5-7) means (moles of f formed in reaction

m)/(s m ).

Integration of Eq.(5-8) between the times t and t+At, or step n to n+1,

assuming that Vj remains constant, yields:

Cg(n+1) = q . , .[ ij At] + C /n)]" 1 - C^V h + «:{Cq(n)/C^/n)} jiij At

+XCfk( )f«kj ijAt] + [X^^fp/Cfp/n)At +X4r^(n) •- p m

At (5-9)

k = a, b, ..., Z where k ; t j , ^ f= 1,2,..., F p= 1,2,..., P m = 1, ..., R

An alternative form of Eq.(5-9) in terms of transition probabilities reads

Cfj(n+1) = Cf i jPinj + Cfj(n)pjj +X^fk(n)Pkj + Lfj(n) (5-9a) k

where detailed expressions for the probabilities are summarized in Eq.(5-28a).

CQ(n+l) is the concentration at time t+At, or step n+1, of species f in reactor

j ; Cf^nj is the concentration of species f at the inlet stream Qj flowing from a feed

vessel into reactor j ; Cg(n) and Cfk(n) are, respectively, the concentrations at time t

or step n of species f in reactors j and k. C'fj(n) is the concentration of f in the (m)

Stream Q j leaving reactor j . AC^ .(n) and r . (n) are, respectively, at time t or step n, the driving force for the transfer process p and the reaction rate per unit volume

of species f by reaction m, corresponding to the conditions in reactor j . pinj is the

single step transition probability from the state of the feed reactor to the state of

reactor j ; Pjj is the probability of remaining in the state of reactor j and pkj is the

transition probability from the state of reactor k to the state of reactor j . The

complete expressions for the probabilities in Eq.(5-9a) are given, respectively, in

Page 513: Applications of Markov Chains in Chemical Engineering

507

Eq.(5-9). Lfj(n) is the last term on the RHS of the equation corresponding to

species f (= 1, 2, 3,..., F) in reactor j and at time t. Other definitions are:

. fPdfw M = S i (5-10) i J

where Hfpj, the volumetric mass transfer coefficient (1/s), indicates that this

quantity corresponds to species f for process p in reactor j . In addition,

%=-i %=-i h=t <^-ii> ^ J ^ J ^ J

Reactors i. A mass balance on species f undergoing various mass transfer

processes as well as chemical reactions due to changes in the operating conditions

in reactor i in Fig.5-1, gives:

d[V. C ]

dt '' QiCf,in,i + QjiCfj + X ^ k i C J " ZQikC, H- Q x , + qi^c,+ Q;C, k J L k

p m

i, k = a, b, ..., Z where k ^ t i j , 4 f = l , 2 , ..., F p = l , 2 , ..., P m = l , ..., R.

Cf,in,i is the concentration of species f in the stream Qi flowing from the feed

vessel into reactor i, Cfj is the concentration of species f in reactor j , Cfk is the

concentration of species f in reactor k and Cfi is the concentration of species f in

reactor i. Cfi is the concentration of species f in stream Q'l leaving reactor i.

kfp^apiAC^ . = IX .ACf jV is the mass transfer rate of species f into (or from)

reactor i by some transfer mechanism designated by p (such as absorption,

dissolution, etc., or simultaneously by several processes). kfpj(m/s) is the mass

transfer coefficient with respect to species f for the process p corresponding to

conditions in reactor i; api is the mass transfer area for process p corresponding to

conditions in reactor i. If the mass transfer area apj is not known, the volumetric

mass transfer coefficient |ui .(1/s) defined in Eq.(5-14) is used. AC^ j is the

driving force with respect to species f for the transfer process p at conditions of

reactor i. It should be noted that a positive sign before kfp j indicates mass transfer

Page 514: Applications of Markov Chains in Chemical Engineering

+ C ,(n)| 1~

508

into the reactor whereas a negative sign indicates mass transfer from the reactor

outside. T^ is the reaction rate of species f by reaction m per unit volume of

reactor i corresponding to conditions in this reactor. In the case of a plus sign

before k "" in Eq.(5-6), this means (moles of f formed in reaction m)/(s m^).

Integration of Eq.(5-12) between the times t and t+At, or step n to n+1,

assuming that Vj remains constant, yields:

C^/n+1) = Cf i„,i[ai^iAt] + Cfj(n)[ajiJiiAt]

Pi + ocy +X^ik"^ a;{c;.,(n)/q(n)} V A t l V k ^ -I

k L p m J

i, k = a, b , . . . , Z where k ; t i , j , ^ f = l , 2 , . . . , F p = l , 2, . . . , P m = l , . . . , R.

An alternative form of Eq.(5-13) in terms of transition probabilities reads

Cfi(n+1) = Cf i,,iPi„,i + Cg(n)pj. + C,^(n)p, +^C^in)p^, + ^ ( n ) (5-13a) k

where detailed expressions are summarized in Eq.(5-29a).

Pin,i is the single step transition probability from the state of the feed reactor

to the state of reactor i; pji is the transition probability from the state of reactor j to

the state of reactor i; py is the probability of remaining in the state of reactor i and

pki is the transition probability from the state of reactor k to the state of reactor i .

The complete expressions for the probabilities in Eq.(5-13a) are given,

respectively, in Eq.(5-13) where Lfi(n) is the last term on the RHS of Eq.(5-13)

corresponding to species f (= 1, 2, 3 , . . . , F) in reactors i (= a, b,. . . , Z) and at time

t or step n. Other definitions appear after Eq.(5-12) whereas the (n) or (n+1) in

Eq.(5-13a) stands for t and t+At, or step n to n+1. In addition, the following

definitions are appUcable:

_ fp,i pi „ - J l (t -— (5-14) i 1 rJ

Page 515: Applications of Markov Chains in Chemical Engineering

509

Wp,i» the volumetric mass transfer coefficient (1/s), indicates that this quantity

corresponds to species f for process p (for example: absorption, p = 1; desorption,

p = 2; dissolution, p = 3; etc.) in reactors i or j . In addition,

^ J ^ J ^ J ^ J ^ J

Reactor ^. A mass balance on species f in reactor ^ in Fig.5-1 for k = a, b,

..., Z where k 9t j , ^ reads:

\-^ = %^n -"^^^ff^ - k +Xqk^k (5-16) k ^ k ^

It is assumed that the volume of the fluid in the reactor remains unchanged and that no chemical reactions take place in the reactor as well as other mass transfer processes. Cf is the concentration of species f in reactor ^, C'f is the concentration of species leaving reactor ^ (not necessarily equal to Cf ), Cfk is the concentration of species f in reactor k and Cg is the concentration of species f in reactor j . If the concentration C'f = 0, the species are completely accumulated in reactor ^ which is considered as "total collector" or "dead state" for the species. If C'f = Cf , the species are not accumulated in reactor ^. If 0 < C'f < Cf , the species are partially accumulated in reactor ^ which is considered as a "partial collector" of the species. Integration of Eq.(5-16) between the times t and t+At, or step n to n+1, yields:

C^(n+1) = Cg(n)[Pj ji At] ^X fk^^^^Pk^J^^^^^ k

+ Cf Cn)!" 1 ~ U.^ + ^p^Ac;^(n)/Cf^(n)}^^Atl (5-17)

An alternative form of Eq.(5-17) in terms of transition probabilities reads:

Cf^(n+1) = Cfj(n)pj +X* fk^ )Pk^ + Cf ( )P^ (5-17a)

Page 516: Applications of Markov Chains in Chemical Engineering

510

Detailed expressions for the probabilities are summarized in Eq.(5-28a). Pjt is the

transition probability from the state of reactor j to the state of reactor ^ and pj t is

the transition probability from the state of reactor k to the state of reactor ^. The

complete expressions for the probabilities in Eq.(5-9a) are given, respectively, in

Eq.(5-9). In addition, the following definition are applicable:

Qk . O k . Qi « k = Q - « k = Q - ^rq: (5-18)

Finally, it should be noted that the determination of the parameters of the type

ttij is carried out by the following mass balances which extend Eqs.(4-12a) to (4-

12c). An overall mass balance on the flow system in Fig.5-1, i.e. on reactors j and

k (k = a, b,..., Z where k j) as well as ^, gives:

Qj+XQk = %+21%^+XOk yields 1 + Z«k = Pj +ZPk^+ Z«k k k k k k k

(5-18a)

ttk. ctk'. Pj^, Pk are given by Eq.(5-18). A mass balance on the flows of reactor j , fork = a, b,..., Z where k;tj and ^, yield

Qj-^XQkj = qj^+XQjk+Qi yields i + X«kj = Pj^+E«jk+«i k k k k

(5-18b)

where ttkj, ttjk, ocj' and Pj are given by Eqs.(4-6) and Eq.(5-18). If Qj is not

considered, one of the in flows Qi (i = a, b, ..., Z) or one of the internal flows,

should be taken as a reference flow in Eq.(5-18).

A mass balance on reactors i in Fig.5-1 (k = a, b,..., Z where k ;t i, j and ^),

reads:

Page 517: Applications of Markov Chains in Chemical Engineering

511

Qi + Qji + S Q W = %+Qij + S Q i k + Q i yields k k

"i + "ji + S « k i = h + "ij + X«ik + «i (5-18c)

where Oji, aki, ccik and Pi are defined in Eq.(5-15) and ai, a'j in the following:

(5-18d)

Derivation of probabilities from energy balances

Reactor j . The energy balance [83, p.39] on reactor j referring to Fig.5-1,

ignoring kinetic and potential energies as well as shaft work, reads:

dU. f V W V • -1 "dt = inJ J ^2.Mkj - hj j Z h n ^ + hjnj + \^\f\^ - T )

-XX4fVj [f (5-19) f m

k = a, b, ..., Z where k ^ t j , ^ f= 1,2,..., F m = 1,..., R.

The following definitions are applicable: Uj is the internal energy of the content of reactor j (kcal/kg or kg-mole). h-, h^-, h-t and hjj are, respectively, the

total flow rates (moles or mass) into reactor j from the feed vessel, from reactor k to

reactor j , from j to ^ and from j to k. hinj and hj are, respectively, the specific

enthalpies (per unit mole or mass of mixture) of the inlet stream flowing from a

feed vessel into reactor j and the stream leaving reactor j . n'j is the stream leaving

reactor j outside with enthalpy hj. khj(kcal/(s m^ K)) is the heat transfer coefficient

corresponding to the conditions in reactor j and ahj is the heat transfer area to

reactor j . If the heat transfer area ahj is not known, it is accustomed to use the

volumetric heat transfer coefficient |i (1/s) defined in Eq.(5-22). TQJ is the

temperature from which heat is transferred into reactor j and Tj is the temperature in

reactor j . r . is the reaction rate of species f by reaction m per unit volume of the

Page 518: Applications of Markov Chains in Chemical Engineering

512

reactor (or fluid in the reactor) j corresponding to the conditions in this reactor. As

Eq.(5-19) stands, the minus sign before the double summation indicates that

species f is consumed by the chemical reaction; thus, a plus sign will appear in

Eq.(5-6) before kf. AH J^ is the heat of reaction m at conditions in reactor j . A

minus sign before the heat of reaction indicates generation of heat.

An altemative expression can be obtained by assuming that U = H - PV = H,

that a mean value is taken for the densities in each reactor, and making the

following transformations:

Hj(n) = VjPjhj(n) ATj(n) = \ . - T/n)

j = QjPind kj = QkjPk ^jk = QjkPj ^j^ = Qj Pj ^r^iPi (^"2°)

Integration of Eq.(5-19) between the times t and t+At, or step n to n+1, while

considering the above definitions and making some rearrangements, yields that:

hj(n+l)= hi„j[Pi„jprVjAt]

+ hj(n)|

+Xh>)KjPkPj"VjAt] k

f m

(5-21)

k = a, b, ..., Z where k ^ j , ^ f = 1, 2,..., F m= 1, ..., R.

An altemative form of Eq.(5-21) in terms of transition probabilities reads:

hj(n+l) = hj jPi j i + hj(n)pjj j +X''k(n)Pkj,h + LQ,h( ) (5-21a)

where detailed expressions are sunmiarized in Eq.(5-28b). hj(n+l) is the enthalpy

per unit mass (or mole) of the fluid in reactors j at time t+At, or step n to n+1; hj(n)

and hk(n) are the enthalpies per unit mass (or mole) of the fluid in reactors j and k at

Page 519: Applications of Markov Chains in Chemical Engineering

513

time t, respectively, hj(n) is the enthalpy of the stream leaving reactor j outside.

p|j^: 1 is the transition probability with respect to enthalpy (or temperature) from the

state of the inlet reactor (to reactor j) to the state of reactor j ; p- ^ is the probability

to remain in the state of reactor j with respect to enthalpy (or temperature); pj : j^ is

the transition probability with respect to enthalpy (or temperature) from the state of

reactor k to the state of reactor j . The complete expressions for the probabilities in

Eq.(5-21a) are given, respectively, in Eq.(5-21) where L : j (n) is the last term on

the RHS of Eq.(5-21) corresponding to species f (= 1, 2, 3,. . . , F) in reactor j and

at time t. pinj is the density of the fluid entering reactor j from a feed vessel, pj

and pk are, respectively, the densities of the fluid in reactors j and k and pj is the

density of the stream leaving reactor j ; r|." (n) is the reaction rate at time t, or step n,

of species f by reaction m per unit volume of reactor j corresponding to conditions in this reactor, whereas Ix jCl/s) is defined by:

J pon

Cp j is the specific heat of the fluid mixture in reactor j .

Reactors i. The energy balance on reactor i, ignoring kinetic and potential

energies as well as shaft work, reads [83, p.391:

"dT " in,i i + hjAji + X M k i ~ Vi^ +Z^Vik + ^Ai + hjii; ^ k ^ ^ k ^

^ khiahi(To,i - T.) - X X 4 r ^ ^ i ^ " r f (5-23) f m

i, k = a, b,..., Z where k9ii,j, ^ f=l ,2 , . . . , F m=l, . . . , R.

The definitions of the various quantities appearing in the above equation and

those below are similar to those which follow Eq.(5-19) and (5-21a) whereas j is

replaced by i. Introducing similar transformations to those in Eq.(5-20), as well as

nj = Qjp'j and hj = QjPin i» where Q'i is the volumetric flow rate leaving reactor i

and Q is the flow rate of the fluid from a feed vessel into reactor i; pi is the density

of the fluid leaving reactor i andpjn,i is the density of the fluid from the feed vessel.

Page 520: Applications of Markov Chains in Chemical Engineering

514

Thus, the following equation is obtained by integrating Eq.(5-23) between t and

t+At, or step n to n+1: hjCn-f 1) = hi„. KPi^iprViAt] 4- hj(n)[ajiPjp-ViAt]

+ h-(xi)\ 1 - J Pi^+ tty + 2 l ^ i k + ^iPiPr^t K(^)/ hi(n)]l |i jAtl

+ ^hi^(n)[ai^iPi^prViAt]

|i,iCp .ATi(n) - p r ^ X Z ^ 4 f ^^n)AH(f)} f m

At (5-24)

i, k = a, b, ..., Z where k ^ ^ i j , ^ f = l , 2 , . . . , F m = l , ..., R.

An alternative form of Eq.(5-24) in terms of transition probabilities reads:

hi(n+l) = hi^ .pi„. j + h/n)pji 1 + hi(n)pii j +Shk(n)Pki,h + k.h^^) (5-24a) k

where detailed expressions are summarized in Eq.(5-29b). p. ^ j is the transition

probability with respect to enthalpy (or temperature) from the state of the inlet reactor (to reactor i) to the state of reactor i; p. ^ is the transition probability with

respect to enthalpy (or temperature) from state of reactor j to state of reactor i; pj. j

is the probability to remain in the state of reactor i with respect to enthalpy (or

temperature); pj j is the transition probability with respect to enthalpy (or

temperature) from the state of reactor k to the state of reactor i. [i^^^ is given by

Eq.(5-22) while j is replaced by i. The complete expressions for the probabilities in

Eq.(5-24a) are given, respectively, in Eq.(5-24) where Lfi,h(n) is the last term on

the RHS of Eq.(5-24) corresponding to species f (= 1, 2, 3,. . . , F) in reactor i (=

a, b, . . . , Z) and at time t.

Reactor ^. The energy balance on reactor ^, ignoring kinetic and potential

energies as well as shaft work, chemical reactions and mass transfer processes,

reads [83, p.39]:

Page 521: Applications of Markov Chains in Chemical Engineering

515

ST = ( j' i +XMk%] - h[^i^ + X"k^ ] (5-25)

k = a, b, ..., Z where k^j,^.

U^ is the internal energy (kcal) of the content of reactor ^. HJ and hj ^ are,

respectively, the total flow rates (moles or mass) into reactor ^ from reactors j and

k. hj and hk are, respectively, the specific enthalpies (kcal/kg or kg-mole) of

reactors j and k, respectively. Applying part of the quantities in Eq.(5-20), substituting U^ = H^ =

(Vtpt)ht, and integrating Eq.(5-25) between t and t+At or step n to n+1, yields:

h^(n+l) = hj(n)[pj PjP^V^At] + X^k(n)[Pk^PkP^ V^At] k

+ h^in)\ 1 - f PjP^^Pj + SpkP^'Pk^^^^At] (5-26)

k = a, b, ..., Z where k 9= j , ^.

ht(n+l) is the specific enthalpy at time t+At of the content in reactor ^; ht(n),

hj (n) and hj(n) are, respectively at time t, the specific enthalpies of the content in

reactors ^, k and j . pt is the density of the fluid in reactor ^, |it and P^E ^^

defined in Eq.(5-18). An alternative form of Eq.(5-26) in terms of transition

probabilities reads

h^(nH-l) = hj(n)pj^ h ^X^^k^^^Pk^h "*" ^ ^ P ^ h (5-26a) k

k = a, b,..., Z where k 9t j , ^

where detailed expressions are summarized in Eq.(5-30b). Pjt h is the transition probability with respect to enthalpy (or temperature)

from the state of reactor j to the state of reactor ^; pj t is the transition probability

with respect to enthalpy (or temperature) from the state of reactor k to the state of reactor ^ and pt t ^ is the probability with respect to the enthalpy (or temperature)

Page 522: Applications of Markov Chains in Chemical Engineering

516

to remain in the state of reactor . The complete expressions for the probabilities in

Eq.(5-26a) are given, respectively, in Eq.(5-26).

The single-step probability matrix and summary of the probabilities

The single-step transition probabilities appearing in Eqs.(5-9a), (5-13a), (5-17a), (5-21a), (5-24a), (5-26a), which are defined in Eqs.(5-9), (5-13), (5-17), (5-21), (5-24), (5-26), can be arranged in the following transition matrix P given by Eq.(5-27). The general representation of the above equations is given by Eq.(5-4). For the convenience of the reader the probabilities and the appropriate equations are also summarized below in Eqs.(5-28a,b) to (5-30a,b).

It should be emphasized that the matrix representation becomes possible due to the Euler integration of the differential equations, yielding appropriate difference equations. Thus, flow systems incorporating heat and mass transfer processes as well as chemical reactions can easily be treated by Markov chains where the matrix P becomes "automatic" to construct, once gaining enough experience. In addition, flow systems are presented in unified description via state vector and a one-step transition probability matrix.

Page 523: Applications of Markov Chains in Chemical Engineering

517

cl f,in,j|

f.in.a

c1 f.in.b

"c1 f,in,z|

cl fj

fa

c fb

fz

h in.j

h in.a

h in.b

in.z

h j

IT a h b

z

in

c* f.in.j 1

0

0

0

0

0

0

1 ^ 0

0

0

0

0

p p

0

p p

c f.in.a 0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

c f.in,b 0 •

0 •

1 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

c f.in.z

• 0

' 0

• 0

• 1

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

c p

in.j

0

0

0

p jj

p aj

p bj

p zj

0

0

0

0

0

0

0

0

0

0

c fa 0

p in.a

0

0

p ja

p aa

p ba

p za

0

0

0

0

0

0

0

0

0

0

c fb 0 •

0 •

p in.b

0 •

p • jb

p ab

p bb

P zb

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

c fz

• 0

• 0

• 0

.. p in.z

• p jz

• p az

• P bz

• P zz

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

c 0

0

0

0

',

^^

^b^

'^ p

0

0

0

0

0

0

0

0

0

h in.j

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

h in.a

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

h •• in.b

0 •

0 ••

0 ••

0 •

0 •

0 •

0 •

0 •

0 •

0 •

0 •

1 •

0 •

0 •

0 •

0 •

0 •

0 •

. h in.z

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 0

• 1

• 0

0

• 0

• 0

• 0

h* j

0

0

0

0

0

0

0

0

0

p in.j.^

0

0

0

p jj.h

p aj.h

p bj.h

p zj.h

0

h a

0

0

0

0

0

0

0

0

0

0

p in.a.t

0

0

p ja.h

p aa.h

p ba.h

p za.h

0

h .. b

0 •

0 ••

0 ••

0 ••

0 ••

0 ••

0 ••

0 ••

0 •

0 •

0 •• 1

p in.b.h

0 ••

p jb.h

p abji

p bb.h

p zbji

0 •

. h h

• 0 0 1

0 0 1 0 0 1

• 0 0 1

• 0 0 j

• 0 0 1 0 0 1

0 0 1

• 0 0 1 • 0 0 1

• 0 0 j

• 0 0

P 0 1 in,z.h 1

p p 1 jz.h j ,h 1

• p p 1 az.h a Ji 1

• p p 1 bz.h b^h 1

• p p zz.h z ,h 1

• 0 p«

C — Cf inj f,in,j

h* = hi

(5-27)

The probabilities in the above matrix are spelled out in the following. The

various quantities appearing in the equations are defined as follows: Ojk and akj in

Eq.(5-ll); aik, aki, aij and aji in Eq.(5-15); ai in Eq.(5-14). p.. in Eq.(5-15),

Pl ^ in Eq.(5-18) and pj in Eq.(5-ll). ij in Eq.(5-14), |ij in Eq.(5-10) and l in

Eq.(5-18). iilj j in Eq.(5-22) and, similarly, ij j where i replaces j . \if - in Eq.(5-

10) and |Lif • in Eq.(5-14). AC^ • is the driving force with respect to species f for

the transfer process p at conditions of reactor i, ATj(n) = To,i - T-(n), and similarly

Page 524: Applications of Markov Chains in Chemical Engineering

518

for j which replaces i. p-j^: and p-^ j are the densities of the inlet streams into

reactors j and i from the feed vessel in Fig.5-1. p-, p: and pt are the densities of

the fluid in reactors i, j and ^, respectively.

Finally, attention should be paid to the remarks corresponding to the

following equations: a) In these equations, i, k = a, b, ..., Z where k ^ i, j , ^ f

= 1, 2, ..., F p = 1, 2,..., P m = 1, 2,..., R. b) The determination of the above

quantities, i.e. aik, ttki, ocij, ocji, oci, Pjt, Pj t and Pjt, is based on Eqs.(4-12a) to

(4-12c) following Eq.(5-18) above.

Reactor j :

For mass transfer

Cfj(n-M) = q .„ .pi, j + C,j(n)pj- +XCfk(n)Pkj + L,/n) (5-28a) k

where

Pinj = i^' Pjj = 1 - f Pj^ + S « j k + ai{Cg(n)/Cg(n)} V At ^ k ^

Pkj = afcj jAt Pjk = aj^^^At Pj = aj^^^At

Lfj(n) =

=

%/") = '

S^^fpJACfp.(n) '- p

X^^fp.J^Cfp/n) ^ P

£4f(n)

+ X4f (n)' m

At +rg(n)At

At

Page 525: Applications of Markov Chains in Chemical Engineering

519

For heat transfer

hj(n+l) = hi„ jPi„ j h + hj(n)pjj J, +X^k(n)Pkj,h + \h^^^ k

where

Pind.h = PinjPj"Vj At Pkj.h = ttyPk pj-Vj At

Pjj.h = 1 - f Pj^+ S « j k + a:{h:(n)(n)h/n)}yj At

(5-28b)

Lfi.hW = H,jCp/Tj(n) - p r i ^ X { 4 f (n)AH(f} f m

At

= {^,jCp.AT/n)}At-pri{rg(n)AH,j}At

5(n)AH^ = 5^X^r(f(n)AH(f} f m

If p. . = p., it follows from above equations that p j j = Pin j h ^ ^^^^ ^^^^ Pkj "

Pkj,h'' ^ ^ J ~ ^' PJJ»h = Pjj. As the expression for Lfj,h(n) stands, the minus sign

before the double summation indicates that species f is consumed by the chemical

reaction; thus, a plus sign will appear in Eq.(5-6) before kf. AH^^^ is the heat of

reaction m at conditions in reactor j . A minus sign before the heat of reaction indicates generation of heat.

Page 526: Applications of Markov Chains in Chemical Engineering

520

Reactors i = a, b, ..., Z:

For mass transfer

Cf,(n+1) = q i „ iPi„ i + Cq(n)pji + C,,(n)p, +^C^in\i + Lf,(n) (5-29a)

where

Pin,i = "ih^t Pii = 1 - [ pi + tty + X « i k + ai{Cfi(n)/Cfi(n)}yiAt

Pji = «jiMt Pki = aki^iAt Pi = aikRkAt Pi = ai l At

L,(n) = S^^fp.i^Cfp/n) + Xr^^(n) "- p

At

• P

At + rj.,(n)At

For heat transfer

hj(n+l) = hi„ iPi„ i J, + hj(n)pji, + hi(n)pii^ +Xhk(n)Pki,h + ^.h^^) (5-29b) k

where

Pin.i,h = aiPin,iPr^^i i^t p-i = a^iPjprV jAt pj^i, = a^iPkPr'jiiAt

Pii.h = 1 - fPi + «ij +X«ik+ a;{h;(n)/hj(n))| V A I

Lfi.h(n) = [jthiCp,iATj(n) - prl^;£{4?')(n)AH(f} L f m

= { lhiCpiATi(n)}At - {prirf.,(n)AH j A t

rfl(n)AHri = XZ^4i"^^n)AH(f >}

At

f m

Page 527: Applications of Markov Chains in Chemical Engineering

521

If p. . = p., it follows from above equations that p^^ i = Pin i h ^^ ^^^^ ^ ^^^ Pki ~

Pki,h'if«'i = 0,pii,h = Pii.

Reactor ^:

For mass transfer

Cf^(n+1) = Cfj(n)pj +X^fk^^^Pk^ •*• Cf^(^)P^^ (5-30a) k

where

Pj^=p.^^^At Pk = Pk^^^At

p^^ = 1 - ( pj + XPk^]^C;^(n)/Cf^(n)}H^At

For heat transfer

h^(n+l) = hj(n)pj^ j^ +X^k(«)Pk^.h + h (n)P^ ,h (5~30b) k

where

Pj^h = Pj^PjP^V^At p^^ 1, = p^^p^p^i^i^At

P th = 1 - f PjP 'Pj + XpkP 'Pk^ W If p. = pt, it follows from above equations that p:. j^ = Pjt as well as that pj^ j^ =

Pj t. If, in addition, C'f = Cf , then p. . jj = ptt•

5.2 APPLICATION OF THE MODELING AND GENERAL GUIDELINES

The above modeling is applied to numerous flow configurations which have

appeared in Chemical Engineering textbooks. Additional ones of particular interest

have also been included. Generally, any flow configurations consists of a series of

perfectly-mixed and plug-flow reactors, as well as recycle streams, by pass and

Page 528: Applications of Markov Chains in Chemical Engineering

522

cross flow etc., or part of the above. The guidelines appearing in Chapter 4.2 are also applicable here.

In treating a certain configuration, the first step is to reduce the general configuration in Fig.5-1 to the case under consideration. The second step is to define the state space generally given by Eq.(5-1); thus, the transition probability matrix P given by QE.(5-27). The probabilities appearing in the matrix are given in Eqs.(5-28a) to (5-30b) and the coefficients appearing in the equations are determined by Eqs.(5-18a) to (5-18c). A further step is to specify the initial state vector S(0) given by Eqs.(5-2). The transition from step n to n+1, namely, the determination of S(n+1) given by Eqs.(5-3) is carried out according to Eq.(5-4). Detailed demonstrations of the above guidelines will be made in typical cases presented in the following.

5.2-1 REACTING SYSTEMS

5.2-1(1) The flow system is shown below. It comprises of two reactors where only in

the first one chemical reaction takes place. If the second reactor is assumed as a "total collector" ("dead state") of the reactants and products, p^^ = p22 = 1. If this reactor is not a "dead state", it follows from Eq.(5-30a), assuming C'f = Cf , that:

p ri-fpj - XPk ^ ^ ^ (5.2-l(la))

The carrying fluid in which the reacting species are dissolved, enters the first reactor and leaves the second one at rate Qi.

Fig.5.2-l(l). The flow system

The configuration in Fig.5-1 is reduced to that in Fig.5.2-l(l) by choosing reactors j and ^ designated as j = 1 and ^ = 2, respectively. Considering

Page 529: Applications of Markov Chains in Chemical Engineering

523

Eq.(5-18a) for a'k = 0, yields that a i = Qi/Qi = P12 = qi2/Ql = 1 while taking Qi

as reference flow.

From Eq.(5-la) the state space reads:

55 = [Cf,in,l,Cfi,Cf2] f = l , . . . , 4 (5.2-l(lb))

noting that no heat and mass transfer take place in the case under consideration, i.e.

I fpo = Mhj = 0- The latter are given by Eqs.(5-10) and (5-22). From Eq.(5-3a) the

state vector reads:

S(n) = [Cf,in,l(n), Cfi(n), Cf2(n)] (5.2-l(lc))

The probability matrix given by Eq.(5-27) is reduced to the following one:

Cf.in.l

P = Cfi

Cf2

Cf,in,l

1

0

0

Cfi

Pin,l

Pll

0

Cf2

0

P12

P22 5.2.1(ld))

From Eqs.(5-28a), for j = 1, and (5-30a) for ^ = 2, noting that P12 = 1, it

follows:

Pin,l = m At Pll = 1 - |Lli At P12 = |Ll2At P22= 1 - ^2 At (5.2-l(le))

where p22 is obtained from Eq.(5.2-l(la)). Other equations are:

For reactor 1

Cfi(n+1) = Cf,in,lPin,l + Cfi(n)pii + Lfi(n) f = 1,..., 4 (5.2-l(lf))

where

Lfi(^) =X4?^(n)^t = r ]£r("^)(n) At = r/n)At (5.2-l(lg))

Subscript f 1 designates species f in reactor 1. Since chemical reaction takes place

only in reactor 1, subscript f replaces fl.

For reactor 2

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22 (5.2.1(lh))

Page 530: Applications of Markov Chains in Chemical Engineering

524

The following reactions, similar to case 3.13-4, known as the Brusselator

model [60], are assumed:

i = 1: Ai -^ A3 i = 2: 2A3 + A2 -^ 3A3 i = 3: A3 -^ A2

i = 4: A3 —> A4

thus, m = 1, ..., 4 in the equation for Lfi(n) above. Considering Eq.(5-7), yields

the following relationships:

For i= l : - r l i ) = ra) for i = 2: - ^ = -r^2)= ^

Fori = 3:-r^3) = r 3) Fori = 4: -r ^^ = r ^

Thus,

1 M ~ ^ 1 1 ^ 1 " ^1^1

where kn indicates the rate constant for reaction i = 1 in reactor 1. However, since

chemical reaction takes place only in reactor 1, ki should replace kn and similarly

with the other k's, i.e. one of the subscripts will be omitted for the sake of

simplicity. In addition:

^2 ~ ^2 •*" 2 ~ ~ '^2^2^3 •*" ^ ^ 3

Considering Eqs.(5.2-l(le)) to (5.2-l(lg)) yields for reactor 1 the following

equations where f = 1,..., 4 :

Cii(n+1) = Ci,i„,iPi„j + Cii(n)[pii - kiAt]

C2i(n+1) = C2.in,iPin.i + C2i(n)[pii - kjCf ,(n)At ] + k3C3i(n)At

C3i(n+1) = C3,i„,iPi„ i + C3i(n)[pii - (k3 + k4)At] + kiCii(n)At

+ k2C|i(n)C2i(n)At

Page 531: Applications of Markov Chains in Chemical Engineering

525

C4i(n+1) = C4,in,lPin,l + QlWpi l + k4C3i(n)At

where pin,i, pn and pi2 are given by Eq.(5.2-l(ld)) above. In addition, for

reactor 2:

Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22 C22(n+1) = C2i(n)pi2 + C22(n)p22

C32(n+1) = C3i(n)pi2 + C32(n)p22 C42(n+1) = C4i(n)pi2 + C42(n)p22

In the numerical solution it was assumed that the reactors are of an identical

volume, thus, |ii = QiA^i = |i2 = QiA^2 = V^- The transient response of Cn (case

a), C21 and C31, i.e. the concentrations of species 2 in reactor 1 in Fig.5.2-l(l), is

depicted in Fig.5.2-l(la) where the effect of i is demonstrated. The initial state

vector C(0) = [Cf,in,i(0), Cfi(O), Cf2(0)] reads: [100, 1, 0] for f = 1 as well as [0,

0, 0] for the species f = 2, 3, 4. Other parameters are: ki = 10, k2 = 0.1, k3 = 2,

k4 = 1 and At = 0.05. As seen, increasing \i, i.e. the flow rate into reactor 1,

brings the system to oscillate at |i = 0.01 which diminishes at |i = 0.06.

Page 532: Applications of Markov Chains in Chemical Engineering

526

u

1

0.8

0.6

0.4

0.2

0

n = o . I I I

T > ^ = ' !|'^

fy \ . - • • " " IS \-'' n '-''^

(a)

-

H

0.5

\i = 0.02

1.5

25

20

15

10

5

2 %

[i = 0.01

-

h

1 _

2 /

A--.-

1

.•••'

1/ '' 3

1 i 1/

~~(b)|

/ H

50 100

\x = 0.03

50 100 t

150

150

7

6

5

^4

^ 3

2

1

0

/^;

1 / '"'

Vl

\i = 0.06 1 1

3

V f=2

1 1

(e)| — H

-|

-

-

1 = 0.1

5 10 t

15 %

Fig.5.2-l(la). Cii, C21, C31 versus t demonstrating the effect of |X

5>2-l(2) This is an extension of case 3.13-2 for an open system. It comprises here of

two reactors and only in the first one chemical reaction takes place. The flow

scheme is shown in Fig.5.2-l(l). The configuration in Fig.5-1 is reduced to the

Page 533: Applications of Markov Chains in Chemical Engineering

527

present case by choosing reactors j and ^ designated as j = 1 and ^ = 2,

respectively. Considering Eq.(5-18a) for a'k = 0, yields that ai = Qi/Qi = P12 =

qi2/Ql = 1 while taking Qi as reference flow. Eqs.(5.2-l(lb)) to (5.2-l(lh)) of

case 5.2-1(1) are applicable also in the present case for f = 1,..., 6.

The following reactions of case 3.13-2, known as the Belousov-Zhabotinski

reaction [59], were also applied here: k, kj

i= 1: Ai+A2-> A3 + A4 i = 2: A3 + A2-^2A4 kj k4

i = 3: Ai + A3 ^ 2A3 + A5 i = 4: 2A3 Ai + Ae

i = 5:A5^gA2 (5.2-1(2))

thus, m = 1,..., 5 in Eq.(5.2-l(lf)) for Lfi(n). Considering Eq.(5-7), yields the

following relationships:

1.(2)

Fori=l:-r( i> = -r(') = r(i) = r(i) For i = 2: - r ) = - r ) = - 1 -fO) 1.(4)

For i = 3: - r(3> = - r ) = ^ = r ) For i = 4: - - | - = r ) = r ) 1.(5)

For i = 5: - r 5) = —

Thus, 1-1 -— x-i 11-1 • *• 1 ~~ """ ^1 ^ 1 ^ 0 """ ^ "^^1 ^ ' d. '

T^ = r^ +13 +13 +r3 = kjCjC2 - k2C2C3 - k^C^C^ + 2k3C|C3 - 2k4C3

^4~^4 "*' 4 = k|C|C2 + 2k2C2C3 r5 = r3 +T^ = k3CjC3-k^C^

Considering Eqs.(5.2-l(le)) to (5.2-l(lg)) yields for f = 1, ..., 6 the

following equations for reactor 1:

Page 534: Applications of Markov Chains in Chemical Engineering

528

Cii(n+l) = Ci,in,lpin,l +Cii(n)[pii - {kiC2i(n)+ k3C3](n)}At]

+ k4C3i2(n)At

C2i(n+1) = C2 ,in,lpin,l + C2i(n)[pii - {kiCii(n) + k2C3i(n)}At]

+ fk5C5i(n)At

C3i(n+1) = C3,in,lPin,l + C3i(n)[pii - {k2C2i(n) + 2k4C3i(n)}At]

+ kiCii(n)C2i(n)At + k3Cii(n)C3i(n)At

C4i(n+1) = C4,in,lPin,l + C4i(n)pii + kiCii(n)C2i(n)At

+ 2k2C2i(n)C3i(n)At

C5i(n+1) = C5,in,lPin,l + k3Cii(n)C3i(n)}At +C5i(n)[pii _ k5Cii(n)At]

C6i(n+1) = C6,in,lPin,l + C6i(n)pii + k4C3i2(n)At

where pin,i, pii and pi2 are given by by Eq.(5.2-l(ld)). In addition, for reactor 2:

Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22 C22(n+1) = C2i(n)pi2 + C22(n)p22 C32(n+1) = C3i(n)pi2 + C32(n)p22 C42(n+1) = C4i(n)pi2 + C42(n)p22

C52(n+1) = C5i(n)pi2 + C52(n)p22 C62(n+1) = C6i(n)pi2 + C62(n)p22

In the numerical solution it was assumed that the reactors are of an identical

volume, i.e. ^ii = Qi/Vi = (X2 = Q1/V2 = |Li. In addition, the second reactor has

been assumed as "total collector" of the reactants and products, i.e. p22 = 1. If this

reactor is not a total collector, i.e. a "dead state", Eq.(5.2-l(a)) is applicable and

P22 = 1 - M 2At. The transient response of C3i,C5i in reactor 1, €32,052 in reactor

2, corresponding to Fig.5.2-l(l), is depicted in Fig.5.2-1(2) where the effect of |X

and g in Eq.(5.2-1(2)), is demonstrated. The initial state vector C(0) = [Cf,in,l(0),

Cfi(O), Cf2(0)] reads: [0.015, 0, 0] for f = 1, [0.004, 0, 0] for f = 2 as well as [0,

0, 0] for f = 3,..., 6. Other parameters were: ki = 0.05, k2 = 100, k3 = 10^, k4 =

10, ks = 5, g = 1, 5, 10 and At = 0.0005. As seen, increasing \i, i.e. the flow rate

into reactor 1 brings the system to oscillate at |LI = 0.5 and g = 1 as seen in cases c

and d. The oscillations diminish at large values of t and also by increasing g as

demonstrated in cases e and f .

Page 535: Applications of Markov Chains in Chemical Engineering

529

1.2 lO"

1 lo' U

8 1 0 >

^ 6 lO-'L

^"4 lO-'L

2 lO'^l -

0 10

\i « 0.005, g = 1

u

u

» » (^

f 1 = s i / ^ ' l

/ """'"•? 1 J 12 = 32, 52

1 - 1 1

L h

u

^ i - 0,05, g = l

* ' ^J 3 1 / " ^

/ ' ' ^ 51 J / £=^2,i2j _ ._...j J. _._i

0.5 1.5 0 0.5 1.5

6 10'

5 lO'^k

4 10

^ 3 10"

u"2 lO' U

1 lO' L

0 lOV — — -1 10"

^ - 0 . 5 , g = l

h-

H

h

-

1

fl=3J

1

't-

' (cj

f2 = 32,l

J

> i - l , g = l

0.5 1 t

1.5 0

5 10"

4 10"

3 10' G

^ 2 10-

^"iio-"

0 10°

-1 10'

^ - l , g = 5

JA 1= 1 ' / r >•

/ . " 3 2 / 51 . ^

^ - - - • " 52 ^ - •

1 1

^''H j ^

, - . j - ' # - - -

H

H

Ji- 1, g=10

0.5 1 t

1.5 0

Fig.5.2-1(2). C31, C51, C32andC52 versus t demonstrating the

effect of fiand g

5.2-1(3) This is an extension of case 3.13-5 for an open system comprising of two

reactors where only in the first one chemical reactions take place. The scheme is

Page 536: Applications of Markov Chains in Chemical Engineering

530

shown in Fig.5.2-l(l). The configuration in Fig.5-1 is reduced to the present one by choosing reactors j and ^ designated as j = 1 and ^ = 2, respectively. Considering Eq.(5-18a) for a'k = 0, yields that ai = Qi/Qi = P12 = qi2/Ql = 1 while taking Qi as reference flow. Eqs.(5.2-l(la)) to (5.2-l(lg)) in case 5.2-1(1) are applicable also in the present case for f = 1,..., 7.

The following model [57] appearing in case 3.13-5, which simulates the Belousov-Zhabotinski reaction, was appUed also for the open system:

kj k2

i = 1: Ai + A2 " A3 + A4 i = 2: A3 + A2 ~ 2A4

i = 3: Ai + A3 " 2A5 i = 4: A6 + A5 ^ A3 + A7

k_3 k_4

ks h i = 5: 2A3 Ai + A4 i = 6: A7 4 gA2 + Ae

thus, m = 1,..., 6 in Eq.(5.2-l(lf)) for Lfi(n). Considering Eq.(5-7), yields the following relationships:

1.(2)

Fori = l:-r(i) = -ry) = r») = rV) Fori = 2: - f ) = -42) = - i -

r; .(3) 5 Fori = 3:-r|3) = -r(3> = - | - For i = 4: - r ) = - r 4) = r ) = r )

r(5) 1.(6)

Fori = 5 : - 4 - =r^^^ = r'i^ For i = 6: - r ) = ^ = r 6)

Thus,

rj = rj +rj +rj =--kjCjC2+k_|C3C4-k3CjC3 + k_3C5 + k5C3

r ^ ^ ^ 9 ' ^ 9 2 ^ — 1 1 9 —1 ' ^^4 9 9 ' —2 4 S 6 7

r ^ ro 4"ro iro ' " " "" i i 9 —1 " 4. 9 9 ' ' —9 4

" % ^ i ^ 3 ••" ^- -3 5 4 5 6 -4 3^7 5 3 - i ^ v jv 4

Page 537: Applications of Markov Chains in Chemical Engineering

531

1 = 1^ +1*4 "^^4 ~'^iCjC2 "" K.jC^C^ + 2k2C2C2~ 2K_2C4 + k^C^

" ^ - 5 ^ 1 ^ 4

^5~^5 "*" 5 ~ 2k3CjC 3 - 2k_3C5 - k^C^C^ - k_^C3C ^

r^ = r^ + r ^ = - ^ € 5 0 ^ + k_4C3C j + k^C 7

r = /"*) + r( ^ = k r r - k T C - k C 7 7 7 4 5 6 *'-4^3^ 7 *6 7

Considering Eqs.(5.2-l(le)) to (5.2-l(lg)) yields for f = 1, ..., 7 the following

equations for reactor 1 where the different rf(n) are given above:

Cfi(n+1) = Cf,in,lPin,l + Cfi(n)pii + rf(n)At

Pin,l> Pll above and pi2 below are given by by Eq.(5.2-l(ld)). In addition, for

reactor 2:

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22

where P22 = 1 - M'2At is obtained from Eq.(5-30a) noting that C'f2 = Cf2.

In the numerical solution it was assumed that the reactors are of an identical

volume, i.e. |Lii = Qi/Vi = |i2 = Q1/V2 = |ii. The initial state vector C(0) =

[Cf,in,l(0), Cfi(O), Cf2(0)] reads: [1, 0, 0] for f = 1, [0.2, 0, 0] for f = 2, 6 as well

as [0, 0, 0] for f = 3, 4, 5, 7. Additional parameters were: ki = 1, k-i = 10^, k2 =

40, k.2 = 0, k3 = 100, k.3 = 20, k4 = 1000, k.4 = 100, ks = 1, k.5 = 0, k6 = 1 and

At = 0.015.

The transient response of C n , C21, C31 and C71, i.e. the concentrations in

reactor 1 where reactions take place, is depicted in Fig.5.2-1(3). The effect of |Ll =

0.00001, 0.0001, 0.0005 as well as g = 0 and 0.5 is demonstrated.

Page 538: Applications of Markov Chains in Chemical Engineering

532

0.0005

0.00041

0.0003

u 0.00021

0.0001

o|

-0.0001

0.02|

0.015

0.01

^0.005

01

k

L_

\i = 0.00001,g = 0 1 1 1 1

n = \y^

71 t i l l

/H -j

-J

- -

10 20 30 40 50 t

-0.005.

\i = 0.0005, g = 0

k

\-

r^ • *•'•'

1 1

fl = l l

21. .

1 1

1, 71;' '.

: . • • * '

.v''3r\'

1

' (c)|

-j

y^ H

1 0 10 20 30 40

t 50

\i = 0.0002, g = 0

i = 0.0005, g = 0.5

-0.005

Fig.5.2-1(3). Ci i , C21, C31 and C71 versus t demonstrating the effect of III and g

5.2-1(4^ This is an extension of case 3.13-3 for an open system comprising of two

reactors; in the first one chemical reaction takes place. The flow scheme is shown

in Fig.5.2-l(l). The configuration in Fig.5-1 is reduced to the present one by

choosing reactors j and ^ designated as j = 1 and ^ = 2, respectively. Considering

Eq.(5-18a) for a'k = 0 = 0, yields that a i = Qi/Qi = P12 = qi2/Ql = 1 while taking

Ql as reference flow. Eqs.(5.2-l(lb)) to (5.2-l(lh)) in case 5.2-1(1) are

applicable also in the present case where f = 1,..., 7.

The following model [65], appearing in case 3.13-3, was applied also for the

open system:

Page 539: Applications of Markov Chains in Chemical Engineering

533 ki k, k3

i = 1: Ai + A2 ~* A3 i = 2: A3 - • A2 i = 3: 2A2 - • A4

i = 4: A3 + A4 -* 3A2 i = 5: A4 - • A5

thus, m = 1,.... 6 in Eq.(5.2-l(10) for Lfi(n). Considering Eq.(5-7), yields the

following relationships: r(3)

Fori- l:-r<i)--r^»>-r(i> Fori -2: - r f - 42) For i - 3 : - - | - - r f

r(4) Fori - 4: - r > - - i ^ --j- Fori - 5: - i ) = i s)

Thus,

rj - r - - kjCjCj + k.jCg

= - kjCjC^ + k.iC3 + k2C3 - 2k3C 2 + ^^4^^^^

^3 *" ^3 •*" ^3 •*• ^3 " ^ 1 ^ 2 " - 1 ^ 3 " ' ^ ^ 3 ~ k 4 C 3 C 4

r - r^f + r(f + r(|) - kgC^ - k^CgC - kjC 4

^ 5 - 4 ' ^ - k 5 C ,

Considering Eqs.(5.2-l(le)) to (5.2-l(lg)) yields for f = 1, ..., 5 the following

equations for reactor 1 where the different rf(n)'s are given above:

Cfi(n+1) = Cf4n,iPin,l + Cfi(n)pii + rf(n)At

Pin,l> Pll above and pi2 below are given by by Eq.(5.2-l(ld)). In addition, for

reactor 2

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22

where p22 = 1 - \^2^^ is obtained from Eq.(5-30a) assuming that C*f2 = Cf2. In the

numerical solution it was assumed that the reactors are of an identical volume, i.e.

m = QiA/ i = ji2 = Q1/V2 = Ji. The initial state vector C(0) = [Cf4n,i(0), Cfi(O),

Page 540: Applications of Markov Chains in Chemical Engineering

534

Cf2(0)] reads: [20, 1, 1] for f = 1, 2 and [0, 0, 0] for f = 3, 4 and 5. Additional parameters were: ki = 2.5, k.i = 0.1, k2 = 1, ks = 0.03, k4 = 1, ks = 1 and At = 0.005. The transient response of Cn, C21, C31 and C41 in reactor 1 shown in Fig.5.2-l(l), is depicted in Fig.5.2-1(4) where the effect of \i is demonstrated.

2

1.5

ii = 0

U 1 U

0.5

0

1

k

1

fl =

r

I

1

11

1 ""

1 1

21

3 1

^" 1 " I

1

--

1

\i = OA

10

i = 0.01

1.5

1

0.5

0

14

12

10

8

6

4

2

0

-2

1 1 1 1

21 . --"" 11

1 ^ vL ^- --H i l l

1 1

H

-

1 4 6

t i = 5

10

-

h

h

1 1

/

1 1

IK r 1 1 1

1 1 31

21

11

1 1

1 1

- H

H

1

10

Fig.5.2-1(4). Cii, C21 and C31 versus t demonstrating the effect of |Li

5>2-lf5) This is an extension of case 5.2-1(1) for an open system comprising of three

reactors; in the first two ones chemical reaction takes place, and the third reactor is a "total collector" of the reactants and products. If this reactor is not a total collector, QE.(5.2-l(a)) is applicable, i.e. in the following matrix P33 = 1 - ILI3 At. Note that in previous cases, 5.2-1(1) to 5.2-1(4), chemical reaction took place only in one reactor. The flow scheme is shown in Fig.5.2-1(5), which is slightly different than the scheme in case 4.3-4.

Page 541: Applications of Markov Chains in Chemical Engineering

535

Qi+Qs

!S ' Fig.5.2-1(5). The flow system

The general configuration in Fig.5-1 is reduced to the present one by choosing reactors j , a and ^ designated as j = 1, a = 2 and ^ = 3, respectively. Considering Eqs.(5-18a) to (5-18c) for a'k = 0, yields that P13 = 1 + a2, 1 + OL21

= CX12 + Pi3 and a2 + an = a2i, where a2 = Q2/Q1, Pi3 = qis/Ql, OC12 = Q12/Q1 and a2i = Q2l/Ql- A numerical solution was obtained in the following for a2 = 1; thus Pi3 = 2 and a2i = ai2 + 1. From Eq.(5-la) the state space in the present case reads:

SS = [Cf,in,l, Cf,in,2, Cfi, Cf2, Cf3] f = 1, ..., 4 (5.2-l(5a))

noting that no heat and mass transfer take place, i.e. |ifpj = |ihj = 0 which are given

by Eqs.(5-10) and (5-22). From Eq.(5-3a) the state vector reads:

S(n) = [Cf,in,i(n), Cf,in,2(n), Cfi(n), Cf2(n), Cf3(n)] (5.2.1(5b))

The probability matrix given by Eq.( 5-27) is reduced to:

Cf,in,l

Cf,in,2

P = Cfi

Cf2

Co

Cf,in.l 1

0

0

0

0

Cf,in,2 0

1

0

0

0

Cfi

Pin,l

0

Pll

P21

0

Cf2

0

Pin,2

P12

P22

0

Cf3

0

0

P13

0

1 5.2-l(5c))

From Eqs.(5-28a) for j = 1, (5-29a) for a = 2 and (5-30a) for ^ = 3, it

follows, noting that P13 = 2 and a2i = O-u + 1, for reactor 1 that:

Pin,i=HiAt pii = l - (2 + ai2)mAt pi2 = ai2|X2At pi3 = 2M.3At

Page 542: Applications of Markov Chains in Chemical Engineering

536

for reactor 2:

Pin,2 = H2At p 2 2 = l - (l+ai2)^l2At p2i = (1 + ai2)^2At P23 = 0

for reactor 3:

P33 = 1 (5.2-l(ld))

In addition, for all species, i.e. f = 1,..., 4:

for reactor 1: Cfi(n+1) = Cf,in,iPin,l + Cfi(n)pii + Cf2(n)p2i + Lfi(n)

for reactor 2: Cf2(n+1) = Cf,in,2Pin,2 + Cfi(n)pi2 + Cf2(n)p22 + Lf2(n)

for reactor 3: Co(n+l) = Cfi(n)pi3 + Cf3(n) (5.2-l(le))

where

Lfi(n)=^r(7)(n)At = rf,(n)At L jCn) =^r[™Hn)At = r jWAt (5.2-l(lf)) m m

Subscripts fl, f2, f3 designate species fin reactors 1, 2, 3, respectively.

The following reactions, as in cases 5.2-1(1) and 3.13-4, known as the

Brusselator model [60], are assumed:

i = 1: Ai —> A3 i = 2: 2A3 + A2 --> 3A3 i = 3: A3 -> A2

i = 4: A3 -> A4

thus, m = 1, ..., 4 in the equations for Lfi(n) and Lf2(n) above. Considering the

derivations in case 5.2-1(1), the following equations are obtained which enables

one to compute Lfi(n) and Lf2(n):

Page 543: Applications of Markov Chains in Chemical Engineering

537

f = l :

for reactor 1: rjj(n) = - k^Cjidi) for reactor 2: rj2(n) = - kjCj2(n)

f=2 :

for reactor 1: r2|(n) = - k2C2i(n)C^j(n) + k3C3j(n)

for reactor 2: r22(n) = - k2C22(n)C^2(^^ "*" ^3^32(11)

f = 3 :

for reactor 1: r3j(n) = kjCjj(n) + k2C2iC^i(n) - (k3+ k^)C^^(xi)

for reactor 2: r32(n) = kjCj2(n) + k2C22C32(n) - (k3+ k4)C32(n)

f = 4:

for reactor 1: r jCn) = k4C3j(n) for reactor 2: r42(n) = k4C32(n)

Thus, the above equations as well as Eqs.(5.2-l(le)) to (5.2-l(lg)), makes it

possible to calculate the concentration distributions in reactors 1,2 and 3 of species

f = 1, ..., 4. In the numerical solution it was assumed that the reactors are of an

identical volume, thus, |ii = QiA^i = |i2 = Q1/V2 = II3 = QiA' s = |ii. The initial

state vector C(0) = [Cf,in,i(0), Cf,in,2(0), Cfi(O), Cf2(0), CoCO)] reads: [100, 10,

1, 1, 0] for f = 1 as well as [0, 0, 0, 0, 0] for f = 2, 3, 4. Other parameters were:

ki = 10, k2 = 0.1, k3 = 2, k4 = 1, an = 0, 1, 5, 10 and 50, |ii = 0, 0.02, 0.03 and

0.05 and At = 0.05.

The transient response of C21 and C31 versus t, i.e. the concentrations in

reactor 1 in Fig.5.2-l(l) and of €22* the concentrations in reactor 2, as well as the

attractor C31 against C21, are depicted in Fig.5.2-l(5a) where the effect of ai2 is

demonstrated for [i = 0.02. In Fig.5.2-l(5b) the effect of |Li is shown for ai2 = 5.

Page 544: Applications of Markov Chains in Chemical Engineering

538

6

5

4

U3

2

1

n

1

-

^ 1

«,2=1 t i l l

1 1 1 1

1 1

-H

^ v H

, (d)

0 2 4 6 p 8 S i

10 12 14

^ 1 2 " ^^

Fig.5.2-l(5a). C21, C22 versus t and C31 versus C21 demonstrating the effect of ai2

Page 545: Applications of Markov Chains in Chemical Engineering

539

n = o n = o 1 1 1 1 1

i 1 1 1 1

1 1

1 0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

[ f 1

i = 0.03

1 1 1 1

1 1

1 -2 0 4 6 8 10 12

0.06 1

1

i = 0.06 1 1 1 1 1

1 1 1 1 1

1 1

1 -1 0 1 2 3 ^ 4 5

S i 6 7

Fig.5.2-l(5b). C21, C31 versus t and C31 versus C21 demonstrating the effect of |i

Page 546: Applications of Markov Chains in Chemical Engineering

540

5,2-1(6) IMPOSSIBLE PRODUCTS' BEHAVIOR IN BELOUSOV-ZHABOTINSKI MODEL CREATING A

HUMOROUS PATTERN This case terminates the examples of chapter 5.2-1 on reacting systems. We

apply here the Belousov-Zhabotinski model [57], presented in case 5.2-1(3), for

operating conditions which generate non-realistic results, on the one hand, but a

humorous unexpected behavior, on the other, due to a certain presentation of the

results. The equations are those appearing in case 5.2-1(3) for the following data.

The initial state vector C(0) = [Cf,in,i(0), Cfi(O), Cf2(0)] reads: [5.0 10-3, 10-5, Q]

for f = 1, [3.5 10-5, 0, 0] for f = 2, [0, 0, 0] for f = 3, 4, 6, 7 and for f = 5 it reads

[1.24 10- , 10-11,0]. Additional parameters were: ki = 0.084, k.i = 1.0 10 ' k2 =

4.0 108, k.2 = 0, k3 = 2000, k.3 = 2.0 10 , k4 = 1.3 10 , k.4 = 2.4 10 , ks = 4.0

106, k.5 = 4.0 10-11, k6 = 1.0 106» g = 1, ^ = 0.03 and At = 0.01. These data are

slightly similar to those appearing in [57].

The transient response of C21 and C31, i.e. the concentrations in reactor 1

where reactions take place, is depicted in Fig.5.2-l(6a) as well as the plot of C21

against C31 is demonstrated. As observed, the C2i-t and C3i-t behavior is

unrealistic after 25 time units since negative concentrations are obtained and

somehow chaotic behavior. However, the plot of C21 against C31 creates a

combined eyes and nose which were complimented by the author to a face.

Page 547: Applications of Markov Chains in Chemical Engineering

541

5 10"

4 10

3 10" h

2 10"' h

u l l O ' h

- l l O ' h

-2 10"

Fig.5.2-l(6a). C21 and C31 versus t

-2 10-' -1 10

Fig.5.2.1(6b). C21 versus C31

Page 548: Applications of Markov Chains in Chemical Engineering

542

5.2-2 ABSORPTION SYSTEMS In the following cases absorption processes with and without chemical

reaction will be demonstrated.

5.2-2(1) The flow system shown below comprises of three reactors. In the first two

reactors absorption of a single component takes place, whereas the third reactor is

assumed as "total collector" for the absorbed gas. If this is not the case, Eq.(5.2-

1(c)) is applicable and in the following matrix P33 =1-113 t. The fluid in which

the species are absorbed, enters the first reactor and leaves the third one at a rate

Qi.

%, [oh 1

V 2

^23 do 3

^13

Fig.5.2-2(1). The flow system

The configuration in Fig.5-1 is reduced to that in Fig.5.2-2(1) by choosing

reactors j , a and ^ designated as j = 1, a = 2 and ^ = 3, respectively. Considering

Eqs.(5-18a) to (5-18c) for a'k = 0, yields that P13 + P23 = 1, 0C2l + p23 = a i2

where Ojk and Pj^ are given by Eq.5-11.

From Eq.(5-la), the state space for species f = 1 reads:

SS = [Ci,in,i, Cii , C12, C13]

where from Eq.(5-3a) the state vector reads:

S(n) = [Ci,in,i(n), Cii(n), Ci2(n), Ci3(n)]

The probability matrix given by Eq.( 5-27) is reduced to:

(5.2-2(la))

(5.2-2(lb))

Page 549: Applications of Markov Chains in Chemical Engineering

543

P =

'l,in,l

Ci i

Cl2

Cl3

Cl,in,l

1

0

0

0

Cu Pin.l

Pll

P21

0

Cl2

0

P12

P22

0

Cl3

0

P13

P23

1 5.2-2(lc))

From Eqs.(5-28a) for j = 1, (5-29a) for a = 2 and (5-30a) for ^ = 3, noting

that there is a single mass transfer process, i.e. absorption and hence p = 1, it

follows that:

Pin,l = mAt pii = l-(ai2+Pl3)HlAt p2l=a2imAt

P12 = ai2 M'2At P22 = 1 - (a21 + p23)|X2 At

Pl3 = Pl3H3At P23 = p23^3At (5.2-2(ld))

For reactor 1:

Cii(n+1) = Ci,in,lPin,l + Cii(n)pii + Ci2(n)p2i + Lii(n) (5.2-2(le))

where from Eq.(5-28a)

Lii(n) = ^ii,iACn,iAt (5.2-2(lf))

andfromEq.(5-14)

C* J is the equilibrium concentration of the species 1 absorbed on the surface of the

liquid in reactor 1 corresponding to its partial pressure in the gas phase above the

liquid.

For reactor 2:

Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22 + Li2(n) (5.2-2(lg))

where

Page 550: Applications of Markov Chains in Chemical Engineering

544

Li2(n) = ^ii,2ACn,2^t (5.2-2(lh))

and 11 2^12 *

M'11,2 ~ V ^^11,2 ~ ^12 ~ C^2(n)

C*2 is the equilibrium concentration of species 1 absorbed on the surface of the

liquid in reactor 2 corresponding to its partial pressure in the gas phase above the

liquid.

For reactor 3:

Ci3(n+1) = Cii(n)pi3 + Ci2(n)p23 + Ci3(n)p33 (5.2-2(li))

In the numerical solution it was assumed that the reactors are of an identical

volume, thus, |ii = Qi/Vi = |X2 = Q1/V2 = II3 = Q1/V3 = |LI, and that the third

reactor behaves as "total collector" for the reactants and products, namely, P33 =

l.The transient response of Cn , C12 and C13, i.e. the concentrations of species 1

in reactors 1, 2 and 3 is depicted in Fig.5.2-2(la) where the effect of |l = 1, 10

(cases a, b in the figure) and the mass transfer coefficient for absorption (for which

p = 1 in Eq.5-28a) of species lin reactor 2, i.e. |LIII,2 = 1, 50 (cases d, e) as well

as C*2 = 3-10-6' 6-10-6 (cases c, d) is demonstrated. The initial state vector C(0) =

[Cf,in,i(0), Cfi(O), Cf2(0), Cf3(0)] = [0, 0, 0, 0] for f = 1. Other parameters were: OC12 = 1, Pl3 = 0, mi,l = 50, C*i = 310-6 and At = 0.0001.

Page 551: Applications of Markov Chains in Chemical Engineering

545

3 10

2.5 10" h

.- 2 10- h

1.5 10" h

1 10

5 10

j-6

)-' ,-6

,-6

,-6

H = l

y^ i w

- / -

- /

_/

f =i - " " "

^l= 10

^0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.05 0.1 0.15 0.2 0.25 0.3 t t

1.5 10

1 10" k

u 5 10"T

C =3 10 , i = 50 12 11.2

_ ~

p

c

1

^^

1

• 1

1

1

3 ^

1

2

1

*-

J (d)-

^ J

1 0 0.05 0.1 0.15 0.2 0.25 0.3 Q O.05 0.1 0.15 0.2 0.25 0.3

t t

1.5 10 •5 C = 3 10 , i = 1 12 11.2

1 10" r5r

u 5 10'

-

h

' ' ' ' ' (e)J

3 ^ ^ i =1 ^ ^ ^

-_--x: - r 1 1 1 •

6h

0 0.05 0.1 0.15 0.2 0.25 0.3 t

Fig.5.2-2(la). Cn, C12 and C13 versus t demonstrating the effect of

A simpler case which can be treated by applying the above model is the absorption of one component, f = 1, into a single reactor, i.e. reactor 1 where reactor

Page 552: Applications of Markov Chains in Chemical Engineering

546

3 is the "total collector". Under these conditions, ai2 = 0 and P13 = 1 in Fig.5.2-

2(1). In the numerical solution it was assumed that the reactors are of an identical

volume, i.e. a constant |i. Other parameters kept unchanged were: C*^ = 3-10"^ and

C*2 = 0. The initial state vector C(0) = [Ci,in,i(0), Cii(O), Ci3(0)] = [0, 0, 0] and

At = 0.0001. The transient response of Cn and C13, i.e. the concentrations of

species 1 in reactors 1 and 3 is depicted in Fig.5.2-2(lb) where the effect of |x = 0.1,

10 (cases b, c in the figure) and |Liiij = 10, 50 (cases a, c) is demonstrated.

3 10-

2.5 10"

2 10-

^1.5 10-

1 10-

5 10"

0

-

- /

— /

^^=°-^'V.= 1 1

l i = 1 1 ^ - " " ^

13 1 1 r

10

1 • " " ^

-\

-\

-J -J

\i = OA,\i =50 11,1

t - 1 1 r 1 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.05 0.1 0.15 0.2 0.25 0.3

t t

3 10'

2.5 10'

.- 2 10"

^L5 10-

1 10"

5 10"''

0

\i= 10,

1 1 /' i l = 11 •'

~ / ^ "^• '

4 / I / k 1 1

i = 50 11,1

1 1

1 1

1 1 (c)

A

-\

-

1 0 0.05 0.1 0.15 0.2 0.25 0.3

t Fig.5.2-2(lb). Ci i and C13 versus t demonstrating the effect of \i and

^ 11,1

Page 553: Applications of Markov Chains in Chemical Engineering

547

5>2-2(2) The flow system is shown in Fig.5.2-l(l). It comprises of two reactors and

only in the first one the chemical reaction given below takes place, i.e.:

i = l : A i + A2->A3 (5.2-2(2a))

The carrying fluid of reactant 1 (f = 1) dissolved in it enters the first reactor and

leaves the second one at rate Qi. Reactant 2 (f = 2) is absorbed in reactor 1 and

reacts there with reactant 1 yielding product 3 (f = 3). The configuration in Fig.5-1

is reduced to that in Fig.5.2-l(l) by choosing reactors j and ^ designated as j = 1

and ^ = 2, respectively. Considering Eqs.(5-18a) to (5-18c) for a'k = 0, yields

that ai = Qi/Qi = P12 = qi2/Ql = 1 while taking Qi as reference flow.

From Eq.(5-la) the state space reads:

SS = [Cf,in,l, Cfi, Cf2] f = 1,..., 3

From Eq.(5-3a) the state vector reads:

S(n) = [Cf,inj(n), Cfi(n), Cf2(n)]

(5.2-2(2b))

(5.2-2(2c))

The probability matrix given by Eq.( 5-27) is reduced, for f = 1,..., 3, to:

p =

Cf,in,l

Cfl

Cf2

Cf,in,l

1

0

0

Cfl

Pin.l

Pll

0

Cf2

0

P12

P22 (5.2-2(2d))

From Eqs.(5-28a), for j = 1, and (5-30a) for ^ = 2, noting that P12 = 1 and

C'f2 = Cf2 , it follows that:

Pin.l=l^l^t pii = l - m A t pl2 = 2At p22=l-^2At (5.2-2(2e))

Considering Eq.(5.2-2(2a)), yields the following relationship:

r(l) = r. = Fj- f j - f j - Tj- kjjCjCj - kjCjCj (5.2-2(2f))

Page 554: Applications of Markov Chains in Chemical Engineering

548

where k n indicates the rate constant for reaction i = 1 in reactor 1. However, since

the chemical reaction takes place only in reactor 1, k] should replace k n . For

reactor 1, it follows from Eq.(5-28a) that:

Cfi(n+1) = Cf,in,iPin,l + Cfi(n)pii + Lfi(n) f = 1,..., 3 (5.2-2(2g))

where

Lii(n) = rii(n)At = ri(n)At = - kiCii(n)C2i(n)At

L2i(n) = [^21,1^^21.1^^) + ^*2i(^)]^^ = [1 21,1 21 •" C2i(n)) + r2(n)]At

= [^21,1(^21 - C2i(n)) - kiCjj(n)C2i(n)]At

L3i(n) = r3^(n)At = r3(n)At = kiCii(n)C2i(n)At

For reactor 2:

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22 f = 1,..., 3 (5.2-2(2h))

In the numerical solution it was assumed that the reactors are of an identical

volume, thus, |LII = QiA^i = |i2 = QlA^2 = 1 - The transient response of Cn , C21

and C31, i.e. the concentrations in reactor 1 of species 1, 2 and 3 in Fig.5.2-2(1),

is depicted in Fig.5.2-2(2). The effect of i is demonstrated in cases a, b as well as

in cases d, e, and f; the effect of the mass transfer coefficient for absorption, i.e.

^21 1 (for which p = 1 in Eq.5-28a), is depicted in cases b, c. The initial state

vector C(0) = [Cf,in,i(0), Cfi(O), Cf2(0)] was [0, 0, 0] for f = 2, 3 in all cases and

[310-5, 0, 0] as well as [0, 310-5, 0] for f = 1 when exploring the effect of i in

cases a, b and d, e, f. Other conmion parameters were: ki = 10 » C21 = 3-10-5 and

At =10-4.

Page 555: Applications of Markov Chains in Chemical Engineering

549

3 10"

2.5 10"

^1.5 10-

1 10" \L

5 10'

. ^ = 10, |i = 100,C = 3 10• C (0) = 0 [i= 100, i = 100, C =3 10' C (0) = 0 5 '^ '^Zl.l l,in.l 1 2 U UrU T ^

LL

\ ^ ]/ y r-<.

"1

/ .1

^ ^

\

y

L

X-

1 2

f = l

1

-

1

-"

(a)

-

~-

"TT^ (b)

0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.05 0.1 0.15 0.2 0.25 0.3 t t , |X = 100, ji = 1000, C = 3 10"\ C (0) = 0

3 lQ-5 21,1 l.inj r

2.5 10' L"

^ 2 10" L ^L5 10-

1 10"'

5 10 r6

1 1 -r [/ 1/ 1 1

' 2 ' f= l

3

1 1

' (c) A

-

-

1

3.5 10"

2.5 10

0 0.05 0.1 0.15 0.2 0.25 0.3

^1=100, |i =100, C =0,C(0) = 3 10" ^ =10,^1 = 100,C = 0, C (0) = 3 10* r5 ^ ^21.1 l.in.l 1 *_ ^21,1 l.in.l 1

1

\ l

Li- V ^

1/ 1

1 1 1 1 (ej 2 J

—j

_ 3

0 0.01 0.02 0.03 0.04 0.05 0 0.05 0.1 0.15 0.2 0.25 0.3

. 1 = 0, i = 100, C =0,C(0) = 3 10' 3.5 10' ^u ijrui I

3 10"

2.5 10"

^ 2 10"

^ ^ 5 10-

1 10' 1

5 10'

0 0.05 0.1 0.15 0.2 t

Fig.5.2-2(2). Cii, Ci2 and C13 versus t the effect of \i and 1121 1

Page 556: Applications of Markov Chains in Chemical Engineering

550

5.2-2(3^ The flow system is shown in Fig.5.2-l(l). It comprises of two reactors and

the chemical reaction given below takes place only in the first reactor. The

following reactions were considered in case 3.13-1 for a closed system:

i = 1: Ai -^ A2 i = 2: A2 -^As (5.2-2(3a))

The carrying fluid, entering reactor 1 at rate Qi, may or may not contain the

reactants or the products. However, in reactor 1 absorption of reactant 1 (f = 1),

i.e. Ai, takes place with the formation of products 2 and 3 (f = 2 and 3). The

configuration in Fig.5-1 is reduced to that in Fig.5.2-l(l) by choosing reactors j

and ^ designated as j = 1 and ^ = 2, respectively. Considering Eq.(5-18a) for a'k

= 0, yields that a i = Qi/Qi = P12 = qi2/Ql = 1 while taking Qi as reference flow.

Thus, Eqs.(5.2-2(2b)) to (5.2-2(2e)) of the previous case are also applicable here

and with respect to Eq.(5.2-2(3a)) the following are the rate equations [55]:

ri = - kiCiC2 r2 = kiCiC2 - k2C2 rs = k2C2 (5.2-2(3b))

For reactor 1, it follows from Eq.(5-28a) that:

Cfi(n+1) = Cf,injPin,l + Cfi(n)pii + Lfi(n) f = 1,..., 3 (5.2-2(3c))

where

Lii(n) = [^lll jACji^Cn) + rii(n)]At = [^nj^C^ - Ci Cn)) + r^WlAt

= [|iiii,i(C*i - Cii(n)) - kiCii(n)C2i(n)]At

L^iCn) = r2i(n)At = r^WAt = kjC^ j(n)C2|(n)At - k2C2i(n)

LjjCn) = r3j(n)At = r3(n)At = k2C2|(n)At

For reactor 2:

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22 f = 1,..., 3 (5.2-2(3d))

Page 557: Applications of Markov Chains in Chemical Engineering

551

where pin,i, Pii and pi2 are given in Eq.(5.2-2(2e)); p22 = 1- |X2 At. In the

numerical solution it was assumed that the reactors are of an identical volume, thus,

|Xi = Qi/Vi = 1X2 = Q1/V2 = |i. The transient response of C n and C21, i.e. the

concentrations of species 1 and 2 in reactor 1 in Fig.Fig.5.2-l(l), is depicted in

Fig.5.2-2(3) where the effect of |LI (cases a, b and c) and k2 (cases d, e and f) is

demonstrated. In case g the operating conditions are identical to case e where C21

is plotted versus C n . The initial state vector C(0) = [Cf,in,i(0), Cfi(O), Cf2(0)]

reads: [0,0, 0] for f = 1, 3 and [0, 310-10,0] for f = 2. Other conmion parameters

were: ki = 10^, At = 510-4, c^j = 1 and ii^^ | = 10" . As seen, the oscillatory

behavior of Ci 1 depends on |LI and k2.

Page 558: Applications of Markov Chains in Chemical Engineering

552

0.0002

0.00015L-

| i = 1, k = 50

U 0.0001

5 10

0.0005,

0.0004

j).0003

^0.0002

0.0001

0|

0

)0001

8 10"

6 10"

4 10"

2 10"

- /

^ = \^^^

1

10, k = 50 2

• n v 1

1 1

1

l _

2

1

(b)

-j

-

-

0 0.2 0.4 0.6 0.8 t

1 10''r

8 10"*

6 10-*

"4 lo--

2 10-*

0

('"'

2

i = 100, k = 50 2

1 1 1 (c)

J

^

-

f 1 1 1 0 0.2 0.4 0.6 0.

-

k

l = 1, k = i ^ 2

1 1

2 . - ' -"*"

f = i

'" m

-

r - - 1 1

0.00035

0.0003

0.00025

0.0002

0.00015

0.0001

5 10'

0

V /

A • /

/ / r

l = l,

1

A i / \ ^ y

k = 150 2

1

.

A/^ /

(e)j

-

/^^--^

0.5 1.5 0 0.5 1.5

0.0007 0.0006 0.0005

J3.0004

u"o.0003

0.0002

0.0001

0 0

p

H ..-'

i = 1, k = 400

1 1 . . . - • • .

1 t=JLH 1

d

L_

^1= 1, k = 150 '^ 2

0.5 1.5 0.0001 0.0002 c 00003

Fig.5.2-2(3). Cii and C21 versus t for the effect of [i and k

Page 559: Applications of Markov Chains in Chemical Engineering

553

5>2-2(4) The flow system shown in Fig.5.2-l(l) comprises of two reactors. The

following reactions known as the Brusselator model [60], taking place in the first

reactor, were also considered in case 3.13-4 for a closed system. In reactor 1

absorption of reactant 1 (f = 1), i.e. Ai, takes place with formation of products 2,

3 and 4 (f= 2, 3 and 4).

kj 2 k k

i = 1: Al -> A3 i = 2: 2A3 + A2 ^ 3A3 i = 3: A3 4 A2 i = 4: A3 - t A4 (5.2-2(4a))

The following are the rate equations:

ri = - kjCi r2 = - k2C2C3 + k3C3

r3 = kjCi - 2k2C2C? + 3k2C2C^ - (k3 + k4)C3 r^ = )^^C^ (5.2-2(4b))

For reactor 1, it follows from Eq.(5-28a) that:

Cfi(n+1) = Cf,in,iPin,l + Cfi(n)pii + Lfi(n) f = 1,..., 3 (5.2-2(4c))

where

L,i(n) = [|AiJ jACji i(n) + r,j(n)]At = [|X,, ,(C*, - Cj,(n)) + ri(n)]At

= [^iii,i(Cii - Cii(n)) - k,Cij(n)]At

L2i(n) = r2j(n)At = r2(n)At = [ - k2C2,(n)Cf,(n) + k3C3,(n)]At

L3i(n) = r3i(n)At = r3(n)At = [kjCjiCn) + k2C2i(n)Cf,(n) - (kj + k4)C3j(n)]At

L4i(n) = r4i(n)At = r4(n)At = k4C3i(n)At

For reactor 2:

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22 f = 1, •.., 3 (5.2-2(4d))

where pin.i, Pii and pi2 are given in Eq.(5.2-2(2e)); p22 = 1- ^2At. In the

nvimerical solution it was assumed that the reactors are of an identical volume, thus,

Hi = QiA' i = |X2 = QiA^2 = V" The transient response of Ci i, C21 and C31, i.e.

Page 560: Applications of Markov Chains in Chemical Engineering

554

the concentrations of species 1, 2 and 3 in reactor 1 in Fig.5.2-l(l), is depicted in

Figs.5.2-2(4a), 5.2-2(4b), 5.2-2(4c). In addition the variation of C31 versus C21

is depicted. The data applied were: the initial state vector C(0) = [Cf,in,i(0),

Cfi(O), Cf2(0)] = [0, 0, 0] for f = 1, ..., 4, ^ij j ^ = 30, |X = 0.06, ki = 1, k2 = 1,

k3 = 2, k4 = 1, Cj 1 = 1 and At = 0.05.

The effect of |l = 0, 0.06, 0.1 and 0.2 is demonstrated in Fig.5.2-2(4a). The

effect of |Lijj J = 30, 5 and 0.5 is demonstrated in Fig.5.2-2(4a), cases c and d for

which IUji 1 = 30, as well as in Fig.5.2-2(4b), cases a to d. The effect of Cti=

1.5, 1 and 0.15 is demonstrated in Fig.5.2-2(4a), cases c and d for which Cj j= 1,

as well as in Fig.5.2-2(4b), cases e to f. Finally, the effect of the reaction rate

constant k2 = 10, 1, 0.7, 0.5 and 0.3 is demonstrated in Fig.5.2-2(4a), cases c and

d for which k2 = 1, as well as in Fig.5.2-2(4c). As observed, the oscillatory

behavior of the concentrations depends on the above operating parameters.

Page 561: Applications of Markov Chains in Chemical Engineering

555

t = o t = o

3

2.5

c 2

1.5

1

0.5

0

1

- / - /

1 ':' / - - ' , '

1 = 1

1

= 0.1 1

1

' (e)

-

-

1

1 = 0.1

0 10 20 30 t

40 50

U

2,

1.5

1

0.5

01

1

1

^ 1

1

= 0.2 1 1

. f= l

3

1 1

' (g)

^-""""^

1 0 0.5 1 1.5 2 2.5 3 0 0.5 1

t Fig.5.2-2(4a). The effect of [i

Page 562: Applications of Markov Chains in Chemical Engineering

556

2.5

U

C , = 1 . 5 1

/

1:/ 1

1 1 2

- H ^

- '3

1 1

' (e)|

J

1

2

1.5

U 1

0.5

0

!

-

c 1

1

11=1-5 1 1

y

1 1

(f)

H

" •

4 6 t

10 0 0.5 1 c 1-5 21

C\=0.15

2.5

20 40 60 80 100 0 t

Fig.5.2-2(4b). The effect of ^ (a to d) and C^^ (e to h)

0.1 ^ 0.2 S i

0.3

Page 563: Applications of Markov Chains in Chemical Engineering

557

u

1.6

1.4

1.2|

0.8 0.61 0.4 0.2

k = 2

1

MAL-i r ^^^--<^^'

\r.-''' 1

10

* - -.

1

f = i

' ^

, 2

1

(a)

—'

" ^

-

u

3 | - ;

2

1

0

1 2 t

k = 0.7 2 1 1

^; V, \: '' '•• •-2"

^ f = l

K 3 f 1 1

(c)

-

50 100

50 100

150

10

8

6

^^4

2

0

k = 0.3 2

1

4

/•• .' 2

-.' ,'

T f=i '•

x^ . /

(g)

H

-

t 1 1

u

150

1

0.8

0.61

0.4

0.2

-

-^^^

k = 10 2

1 1 1

1 1 1

(b)

..^^^"^^ -

Fig.5.2-2(4c). The effect of ki

Page 564: Applications of Markov Chains in Chemical Engineering

558

5>2-2(5) IMPOSSIBLE REACTIONS CREATING AESTHETIC PATTERNS

The flow system is shown in Fig.5.2-l(l) and comprises of two reactors.

The following "reactions", only partially possible, are based on the Lorenz

equations [85, p.697]. The reactions in case 3.14-3 occur in the present example

only in the first reactor. In this reactor, j = 1, absorption of reactant 2 (f = 2), i.e.

A2, takes place with the formation of "products" 1 and 3 (f = 1 and 3) according to

the following Lorenz "reactions": 10

ri = IOC2 - lOCi for which Ai A2 maybe written

10

r2 = 28Ci - C2 - C1C3 for which no reaction may be realized 8/3

T3 = C1C2 - (8/3)C3 for which A3 Ai + A2 may be written

1

For reactor 1, it follows from Eq.(5-28a) that:

Cfi(n+1) = Cf,in,iPin,l + Cfi(n)pii -f Lfi(n) f = 1, ..., 3 (5.2.2(5a))

where

Lii(n) = rii(n)At =ri(n)At = [10C2i(n) - lOC^^WlAt

L2i(n) = [ 121,1 21 i(n) + r2i(n)]At = [ 21,1( 21 - C2i(n)) + r2(n)]At

= [ 21,1( 21 - C2i(n)) + 28Cii(n) - C2i(n) - Cii(n)C3i(n)]At

L3i(n) = r3i(n)At = r3(n)At = [Cii(n)C2i(n) - (8/3)C3i(n)]At

For reactor 2:

Cf2(n+1) = Cfi(n)pi2 + Cf2(n)p22 f = 1,..., 3 (5.2-2(5b))

where pin,i, p u , P12 and P22 are given in Eq.(5.2-2(2e)). In the numerical

solution it was assumed that the reactors are of an identical volume, thus, |LII =

Ql/Vi = |X2 = Q1/V2 = |X. The transient response of Cn, C21 and C31, i.e. the

Page 565: Applications of Markov Chains in Chemical Engineering

559

"concentrations" of species 1, 2 and 3 in reactor 1 in Fig.5.2-l(l), is depicted in

Figs.5.2-2(5a), 5.2-2(5b), 5.2-2(5c).

In addition the variation of C31 versus C21 and C31 versus Cn are depicted,

which reveal very nice patterns from the artistic point of view. This is, by the way,

a mean to generate art from scientific models, which has been demonstrated also in

case 5.2-1(6) above.

The data applied in the computations were: the initial state vector C(0) =

[Cf,in,l(0), Cfi(O), Cf2(0)] = [0, 0.01, 0] for f = 1, 2, 3, ^21,1 = 0' C*i = 0 and At

= 0.01.

The effect of |X = 0, 1,2, 2.07, 3 and 8 is demonstrated in the following in

Figs.5.2-2(5a), 5.2-2(5b), 5.2-2(5c). As seen, for |i = 0 the "concentrations",

which are positive and negative, reveal chaotic behavior as expected from the

Lorenz equations. By increasing \i, i.e. reducing the mean residence time of the

fluid in the reactor, the "concentrations" become positive all the time, and non-

chaotic for |i > 2.

Page 566: Applications of Markov Chains in Chemical Engineering

560

n = o

'21 'II

Fig.5.2-2(5a). The effect of |i = 0 and 1

Page 567: Applications of Markov Chains in Chemical Engineering

561

H = 2

u

40

30

20

10

0

-II

i = 2.07 1 1

1 1

1 1

J=3

.2

1

1

10 t

15 20

Fig.5.2-2(5b). C n , C21 and C31 versus t, C31 versus C21 and C31 versus C n demonstrating the effect of |i = 2 and 2.07

Page 568: Applications of Markov Chains in Chemical Engineering

562

H = 3

Fig.5.2-2(5c). Cii, C21 and C31 versus t, C31 versus C21 and C31 versus Cn demonstrating the effect of |x = 3 and 8

Page 569: Applications of Markov Chains in Chemical Engineering

563

5.2-3 COMBINED PROCESSES In the following, combined processes are presented which incorporate several

chemical engineering operations acting simultaneously.

5>2-3(l) - Chemical reaction and heat transfer The flow system shown below comprises two reactors. In the first one the

chemical reaction A—> A2 takes place and simultaneously the heat of reaction is

removed by a cooler. The carrying fluid in which the reacting species are

dissolved, enters the first reactor and leaves the second one at rate Qi.

Fig.5.2-3(1). The flow system

The configuration in Fig.5-1 is reduced to that in Fig.5.2-3(1) by choosing

reactors j and ^, designated as j = 1 and ^ = 2, respectively. From Eq.(5-18a) for

a'k = 0, it follows that a i = Qi/Qi = P12 = qi2/Ql = 1 while taking Qi as

reference flow.

From Eq.(5-la) the state space reads:

SS = [Cf,in,l, Cfi, Cf2, hin,i, hi, h2] f = 1, 2 (5.2-3(la))

and from Eq.(5-3a) the state vector reads:

S(n) = [Cf,in,l, Cfi(n), Cf2(n), hin,i, hi(n), h2(n)] f = 1, 2 (5.2-3(lb))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

Page 570: Applications of Markov Chains in Chemical Engineering

564

P =

Cf,in,l

Cfl

Co

hin.l

hi

h2

Cf,in,l

1

0

0

0

0

0

Cfl

Pin.l

Pll

0

0

0

0

Cf2

0

P12

P22

0

0

0

hin.l

0

0

0

1

0

0

hi

0

0

0

Pin,l,h

Pll,h

0

h2

0 1 0

0

0

Pl2,h

P22,h (5.2-3(lc))

Noting that the process in reactor 1 incorporates the chemical reaction A,—> Aj for which ri = - kiCi as well as a heat transfer process, it follows from

Eqs.(5-28a) and (5-30a) for j = 1 and = 2 that:

Pin,l = m^t pii = l-HiAt pi2 = ^2At P22 = l (5.2-3(ld))

The last probability indicates that the second reactor behaves as a "total collector" of

the reactants and products. If C'f2 = Cf2, i.e. the reactants and products are not

accumulated in the reactor, Eqs.(5-30a) gives P22 = 1 - M 2At.

From Eq.(5-28b) it follows for j = 1 that:

Pin,l.h= Pin.lPT »il ^t Pii,h = Pll

From Eq.(5-30b) it follows for j = 1 and = 2 that:

Pl2,h = PlP2V2At P22,h = 1 - P1P2VzAt

From Eqs.(5-28a) and (5-28b) it follows that:

Lj.j(n) = rfj(n)At

Lf, b(n) = {^hiCp iATj (n)}At - p7i{rf,(n)AH^j}At

(5.2-3(le))

(5.2-3(lf))

(5.2-3(lg))

Thus, from Eqs.(5-28a) and (5-30a) it follows for j = 1, ^ = 2 and f = 1,2 that for

the 1st reactor:

Cfi(n+1) = Cf,in,lPin,l + Cfi(n)pii + Lfi(n)

Page 571: Applications of Markov Chains in Chemical Engineering

565

hi(n+l) = hin.ipm,14i + hl(n)pil4i + Lf 14,(11) (5.2-3(lh))

For the 2nd reactor:

Cf2(n+1) = Cn(n)pi2 + Cf2(n)

h2(n+l) = hi(n)pi2j, + h2(n)p22ji (5.2-3(li))

In the case under consideration the following expressions are applicable for f = 1,

2: Lii(n) - rij(n)At - ri(n)At- - k^CjiAt L2i(n) = - Lii(n)

L i i > ) - KiCp.1'^'^0 - Ti (n)]}At- p-H- kiC„(n)AHrt}At

L2i.h(n) = Lii.h(n) (5.2-3(lj)

where AH j is negative for an exothermic reaction. It is also assumed that

Pin.i - Pi - P2 = P as well as Cp 1 - Cp 2 = Cp

yielding from Eqs.(5.2-3(le)) and (5.2-3(10) that

Pm.lJi = f*!' * Pi 1 Ji = P2241 = 1 - »*l t Pl2Ji = t*2At

If all reactors are of the same volume m = M'2 = M- Also for Ah = CpAT it follows

that

hi(n+l) = hin.i + Cp|Ti(n) - Tin,i] (5.2-3(lk)

Substitution of the above expression into Eq.(5.2-3(lh)), considering Eq.(5.2-

3(lj) and the above probabilities, yields:

Ti(n+1) = Ti(n)[l - (m + Hhi)At] + [\i{Tia,i + ^hlTo]At

+ (pCp)-l{kiC„(n)AH,i}At (5.2-3(11)

where AH j is negative for an exothermic reaction.

Similarly, Eq.(5.2-3(li)) yields:

Page 572: Applications of Markov Chains in Chemical Engineering

566

T2(n+1) = Ti(n)ji2At + T2(n)(l - ^2At) (5.2-3(lm)

Eqs.(5.2-3(lh)), (5.2-3(li)) for Cfi(n+1) and Ci2(n+1) and (5.2-3(11), (5.2-

3(lm) make it possible to calculate the concentration and temperature distributions

versus time in reactors 1 and 2 for species f = 1,2. In the numerical solution it was

assumed that the reactors are of an identical volume, thus, |Xi = Qi/Vi = |i2 =

QlA^2 = l - The initial state vector S(0) = [Cf,in,i(0), Cfi(O), Cf2(0), hin,i, hi(0),

h2(0)], in terms of temperatures, reads:

S(0) = [Cf,in,l(0), Cfi(O), Cf2(0), Tin,i, Ti(0), T2(0)] for f = 1, 2

In the computations ki(l/min) = exp(17.2-11600/[1.987T(K)]), AHH =

-18000 cal/gr-mole A, p = Igr/cc and Cp = Ical/gr ^C. Other conmion parameters

were: Tin,i = Ti(0) = T2(0) = 25^C and At = 0.0005. The transient response of

Ti(^C) and T2(^C), i.e. the temperatures in reactors 1 and 2, as well as the

concentrations Cn, C21, C12 and C22 versus t, i.e. the concentrations of Ai and

A2 in reactors 1 and 2 are demonstrated in Figs.5.2-3(la) and 5.2-3(lb).

The effect of |Li(l/min) = 0, 10 for a heat transfer coefficient |Lih(l/min) = 100

is depicted in cases a to d; other common parameters are given in the figure. The

effect |ih = 10, 100 for |i = 10 is given in cases c to f. The effect of Cii(0) = 0.2,

0.5, i.e the initial concentration of Ai in reactor 1, is shown in cases c, d, e, h.

The effect of Ci in (0) = 0, 0.2, i.e. the inlet concentration of Ai into reactor 1, is

shown in cases g to j .

Page 573: Applications of Markov Chains in Chemical Engineering

567

n = 0, ^ = 100, c (0) = 0.2, c = 0 n 11 l.in

0.2

u

^ \ \

0

i = 0, \i = 100, C 1 1 1

1 1 1

(0) = 0.2,C =0 1 l.m

' ' (b) fl = ll

-

21

f2 = 12, 22 1 1

0 0.1 0.2 0.3 0.4 0.5 t

1= 10, i = 100, C (0) = 0.2,C =0 ^ ^ h 11 l.in

\i = 10, |i = 100, C (0) = 0.2, C = 0 ^ ^h ir ^ l.in

0.2|

U

^"o.i

1 _ f 1 =

/

/

1

11 1 1

/

21, 22 1 1

-

' ' (d)

• l2=12 1

1 1 0 0.1 0.2 0.3 0.4 0.:

t

45

40

35

30

25

t = 1

- /

r\ '•

10,

\ i

^ h=

= 1

1

10,

1

\ 2

1

C (0) = 11

1

r

= 0.2, c 1 1

rr

= 0 .in

(e)

A

-

10, ^^= 10, C (0) = 0.2, C =0

0 0.2 0.4 0.6 0.8 t

0.2 0.4 ^ 0.6 0.8

Fig.5.2.3(la). Ti(oC) and TiC^C), Cn, C21, C12 and C22 versus t, demonstrating the effect of |ii and |ih

Page 574: Applications of Markov Chains in Chemical Engineering

568

36 n = 10, Li = 100, C (0) = 0.5, C = 0 ^ '^h 11 l.in

^ ^ ^ = 10, [i,= 100, C, (0) = 0.5, C,. = 0

'—^ ^ ^ 1 ^—7^ fl = l l f 2 = 1 2 " 1

i = 10, i = 100, C (0) = 0.5, C. = 0.2

1 r (i)'

J L 0 0.2 0.4 0.6 0.8 1

t

2.5

2

l = 10, i = 100, C (0) = 0.5, C = 0.2 n 11 l.in

1.5L

u 0.5

01

1

— _

L_

L / -r i

1 1 1 1 (j)j X 1

/ -H

/ f2 = 12/ - J

/ 1 / / -J

/ /

/ J < -.- fl = l l .21,22

/ — t 1 V 1

0 0.2 0.4 0.6 0.8 1 t

Fig.5.2.3(lb). Ti(oC) and T2(«C), Cn , C21, C12 and C22 versus t, demonstrating the effect of Ci(0) and Ci,in

5>2-3(2) - Cooling heat transfer The flow system shown below comprises of three reactors. Cooling or

heating of the entering fluid takes place in reactors 1 and 2.

\ i \ i '^0,2 ^ 0,2

Fig.5.2-3(2). The flow system

Page 575: Applications of Markov Chains in Chemical Engineering

569

The configuration in Fig.5-1 is reduced to that in Fig.5.2-3(2) by choosing

reactors j , a and ^, designated as j = 1, a = 2 and ^ = 3, respectively. From

Eqs.(5-18a), (5-18b) for a'k = 0, it follows that p23 = 1. oci2 = 1 + OC21; a i =

Ql/Ql taking Qi as reference flow.

From Eq.(5-la) the state space reads:

55 = [hin,i, hi, h2, hs]

and from Eq.(5-3a) the state vector reads:

S(n) = [hin,i, hi(n), h2(n), hsCn)]

(5.2.3(2a))

(5.2.3(2b))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

hin.l

hi

h2

h3

hin.l

1

0

0

0

hi

Pin,l.h

Pll.h

P21,h

0

h2

0

Pl2.h

P22,h

0

h3

0 1 0

P23,h

P33,h 1

p =

(5.2-3(2c))

It follows from Eqs.(5-28b), (5-29b) and (5-30b) for j = 1, i = a = 2 and ^ =

3, assuming that the density of the fluid flowing in the system remains constant,

that: Pin,i,h = m At pii,h = 1 - ai2M.iAt pi2,h = ai2^2At

P21,h = a2imAt p22,h = 1 - (1 + a21)R2At P23,h = 3At

P33h=l-^3At (5.2-3(2d))

Lfi.h(n) = lhlCp,lATl (n)At = ^hiCp.ilTj.o " T,(n)]At

Lf2.h(n) = lh2Cp,2AT 2(n)At = ^h2Cp.2[T2.o " T2(n)]At (5.2-3(2e))

In addition:

Page 576: Applications of Markov Chains in Chemical Engineering

570

hi(n+l) = hin,ipin,l,h + hi(n)pii,h + h2(n)p2i,h + Lfi,h(n)

h2(n+l) = hi(n)pi2,h + h2(n)p22,h + Lf2,h(n)

h3(n+l) = h2(n)p23,h + h3(n)p33,h (5.2-3(2f))

The following substitution is made for replacing enthalpies by temperatures,

i.e. Ah = CpA; thus:

hi(n+l) = hin,i + Cp{Ti(n) - Tin,i} i = 1, 2, 3

Substitution of the above expression into Eq.(5.2-3(2f)), assuming that C ^ = C 2 = C and considering the above probabilities, yields:

Ti(n+1) = Ti(n)[l - (ai2Hi + |Lihi)At] + T2(n)a2i|iiAt + [^iTin,i + ^hiTo,i]At

T2(n+1) = Ti(n)ai2R2At + T2(n)[l - { h2 + (1 + a2i)^2}At] + |ih2To,2At

T3(n+1) = T2(n)ji3At + T3(n)[l - 113M] (5.2-3(2g)

where ai2 = 1 + 0L2I' Eqs.(5.2-3(2i)) make it possible to calculate the temperature

distributions versus time in reactors 1, 2 and 3. In the numerical solution it was

assumed that m = |i2 = H3 = M. The initial state vector S(0) = [hin,i, hi(0), h2(0),

h3(0)], in terms of temperatures, reads S(0) = [Tin,i, Ti(0), T2(0), T3(0)] =

[50OC, 250c, 250c, 250C]. Other conmion parameters were TI Q = T2,o = lO^C

and At = 0.005. The transient response of TiC^C), T2(oC) and T^i^C), i.e. the

temperatures in reactors 1, 2 and 3, is depicted in Fig.5.2-3(2a). The effect |Ll = 0,

1,10,100 is demonstrated in cases a to d; the effect of ai2 = 0,25 in cases b and e

and of fihi = Mh2 = 0» 10 in cases e and h.

Page 577: Applications of Markov Chains in Chemical Engineering

571

i = 0, i = i = 10, a = 0 h r h2 12 hi h2 12

1 1

4 " 3 \ ^ A " •V 1

2

1 1

1 1 (b)

^ -^

1 1 1

0 1 2 3 4 t

32

30

28

p 2 6

22

20

18 (

30

25

^ 2 0 U h^lS

10

5

^ = 10, i =\i =10 , hi h2

1 1 1 1 = 1

" \ 2

—. ^^.3 "^ - - ->-

1 1 1

a = 0 12

1

1

3 0.1 0.2 0.3 0.4 t

1 1 1 1

2

1 i 1 1

0 1 2 3 t

I

1 4

(c)

-

0

(e)

50 [

45

40

35

30

25

20 5

50

45

40

35

30

25

1 20,

\ 0

D

, = 100,n^=H^=10.a^=0

1 1 1 "1 (d)

,' / ] •' ^ 1

/ J 1 1 1 1

0.02 0.04 0.06 0.08 0.1 t

/ / / /

/ /

1 1 1 1 2 4 6 8 1

t 0

Fig.5.2-3(2a). Ti(oC), TiCC) and T3(oC) versus t, demonstrating the effect of )x, ^hi = |Xh2 and ai2

Page 578: Applications of Markov Chains in Chemical Engineering

572

5.2-3(3) - Heat transfer in impinging streams Impinging streams were thoroughly treated in chapter 4.5 when studying the

RTD of such systems. The flow system below comprises of four reactors three of

which are equipped with heat exchangers.

in,l

M4

obi-^Mobi " ^ rdo j = l ^ = 2 b = 3

\ l ' 0,1 ^0,2 T Q 2

5=4

T T ^03 ^0.3

4n,3

34

Fig.5.2-3(3). The impinging-stream flow system

The configuration in Fig.5-1 is reduced to that in Fig.5.2-3(3) by choosing

reactors j , a, b and ^, designated asj = l , a = 2, b = 3 and ^ = 4, respectively.

From Eqs.(5-18a) to (5-18c) for a'k = 0, it follows: 1 + a3 = P14 + P34; 1 + a2i =

Pl4 + OC12; a i 2 + OL32 = OC21 + a23; 0C3 + a23 = ^32 + P34. From symmetry

considerations ai2 = CX21 = a23 = CX32 = oc ; thus, P14 = P34 = a3 = a i = 1 while

taking Qi as reference flow.

From Eq.(5-la) the state space reads:

SS = [hin,i, hin,3, hi, h2, h3, h4]

and from Eq.(5-3a) the state vector reads:

S(n) = [hin,i, hin,3, hi(n), h2(n), h3(n), h4(n)]

(5.2.3(3a))

(5.2-3(3b))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

Page 579: Applications of Markov Chains in Chemical Engineering

573

hin,!

hin3

hi

h2

h3

h4

hin.l

1

0

0

0

0

0

hin3 0

1

0

0

0

0

hi

Pin,14i

0

P l U

P21J1

0

0

h2

0

0

Pl2Ji

P22Jh

P3241

0

h3

0

Pin3Jh

0

P23J1

P33J1

0

h4

0

0

Pl44i

0

P344i

P44ai

p =

(5.2-3(3c))

From Eqs.(5-28b), (5-29b) and (5-30b) for j = 1, i = a = 2, i = b = 3 and | =

4, assuming that the density of the fluid flowing in the system remains constant,

follows that:

Pin.lJi = ^ At p i u = 1 - (1 + ai2)mAt p2i4i = a2i|AiAt

Pl2Ji = a l2^2At P224i = 1 - (a2i + a23)^2At P32Ji = a32M-2At

Pin34i = «3^ At P334, = 1 - 034 + a32)^3At P234i = a23^3At

Pl4Ji=Pl4M4At p444,= l-M4At P34Ji = P34M4At (5.2-3(3d))

LfiMn) = l hi P.i Tj (n)At = li .Cp.; Ui.o - Ti(n)]At i = 1,2,3 {5.2-3(3e))

In addition:

hi(n+l) = l^,ipin,i ji + hi(n)pii J, + h2(n)p2i4i + Lfi j,(n)

h2(n+l) = hi(n)pi2ji + h2(n)p224i + h3(n)p32ji + Lf2ji(n)

h3(n+l) = lUn3Pin3Ji + h2(n)p234i + h3(n)p334i + LojiCn)

h4(n+l) = hi(n)pi4ji + h3(n)p344, + h4(n)p444i (5.2-3(30)

The following substitution is made for replacing enthalpes by temperatures,

.e. Ah = CpA; thus,

hi(n+l) = hin,i + Cp{Ti(n) - Ti„.i} i = 1,..., 4 (5.2-3(3g))

Page 580: Applications of Markov Chains in Chemical Engineering

574

Substitution of the above expression into Eq.(5.2-3(2f)), assuming that Cp,i = Cp,

hin,l = hin,3 and applying the probabilities in Eq.(5.2-3(3d)) as well as that ai2 =

0C21 = a23 = 0 32 = cx, yields:

Ti(n+1) = Ti(n)[l - { ihi + (1 +a)^i}At] + T2(n)a^iAt + [|LiiTin,i + JXhiTo,i]At

T2(n+1) = Ti(n)a^2At + T2(n)[l - { h2 + 2a|Li2}At] + T3(n)aji2At

+ ^h2To,2At

T3(n+1) = T2(n)a|Li3At + T3(n)[l - {|Xh3 + (1 +a)|i3}At] + [^3Tin,i + ^h3To,3]At

T4(n+1) = Ti(n)^4At + T3(n)H4At + T4(n)(l - |ii4)At (5.2-3(3h))

To obtain the last expression, one should take hin,i = CpTin,i stemming from

Eq.(5.2-3(3g)). Eqs.(5.2-3(3h)) make it possible to calculate the temperature

distributions versus time in reactors 1, 2, 3 and 4. In the numerical solution it was

assumed that |ii = |i2 = |13 = ^4 = M'. The initial state vector S(0) = [hinj, hi(0),

h2(0), h3(0), h4(0)], in terms of temperatures, reads S(0) = [Tinj, Ti(0), T2(0),

T3(0), T4(0)] = [SO^C, 250C, 25oC, 25oC, 25^Cl Other common parameters

were: Ti,o = T2,o = T3,o = lO^C and At = 0.0005. The transient response of

Ti(oC), T2(^C), T3(oC) and T4(oC), i.e. the temperatures in reactors 1, 2, 3 and 4,

is depicted in Fig.5.2-3(3a). The effect of |i = 0, 10, 100 is demonstrated in cases

b, c and d; the effect of a = 0, 10 in cases a and b and of |ihi = V^hi = ^h3 = 0» 10»

100 in cases b, e and f.

Page 581: Applications of Markov Chains in Chemical Engineering

575

60

50

G40

^ 3 0

20

10

-

-

a = 0, i = 10, \i^= 10

t i l l 4 . - • • •

i = l , 3

" " - • - . 2

1 1 1 1

(a^

H

-\

-

~

—. '

\~

r

a = 1

1

10, i = 10, \3i^ 10

1 1 1

4. . . - - - - • -

1,3

2

1 1 1

(bj

1

J

A H

0 0.1 0.2 0.3 0.4 0. t

5 0 0.1 0.2 0.3 0.4 0.5 t

30

25

^ 2 0

u o

H"l5

10

a = 1

1

10, i = 0, i =

1 1

i = 4

- . J^ ,3

1 1

10

1

L

(c)

1

-

r\£\ UU

80

60

40

on

-

- '

a = 10, |i = 100, j = 10

1 1 1 1

1 . 2 , 3

1 1 1 1

(ci)|

-

-

0.1 0.2 0.3 0.4 0.5 0 0.02 0.04 0.06 0.08 0. t t

100

^ 8 0

u o ^•"60

40

on

-

a = 10, ^ = 10, ji = 0

1 1 1 1 i - 4

1 , 2 , 3

1 1 1 1

(e2|

H

-

0 0.2 0.4 0.6 0.8 t

30

25

20

15

10 1 0

a = 10, ^ = 10, u = 100 h

r' Lv

1 1

4

1. 3

(f)

-1

1 1 2 , 0.1 0.2 0.3

Fig.5.2.3(3a). Ti(oC), T2(«C),T3(^C) and T4(«C) versus t, demonstrating the effect of |i, a and |ihi = |Lih2 = Hh3

Page 582: Applications of Markov Chains in Chemical Engineering

576

5,2-3(4) - Concentration of solutions The concentrator is a 4-stage system where evaporation of the solution takes

place in reactors 1, 2 and 3. The inlet concentration of the solution is Cijn,!-

Fig.5.2-3(4). The concentrator

The configuration in Fig.5-1 is reduced to that in Fig.5.2-3(4) by choosing

reactors j , a, b and ^, designated a s j = l , a = 2, b = 3 and ^ = 4, respectively.

From Eqs.(5-18a) to (5-18c), for a'k > 0, it follows:

1 = p34 + cx'i + oc'2 + a'3 1 = ai2 + oc'i ai2 = a23 + a 2 a23 = P34 + oc'3

where a'l = Q'i/Qi and taking Qi as reference flow. It assumed that a'i = a' (i =

1, 2, 3), thus:

a i2 = 1 - a' a23 = 1 - 2a' P34 = 1 - 3a'

From Eq.(5-la) the state space reads:

SS = [Ci,in,h Cii , C12,, Ci3, C14]

From Eq.(5-3a) the state vector reads:

S(n) = [Ci,in,l, Cii(n), Ci2(n), CnCn), Ci4(n)]

(5.2-3(4a))

(5.2.3(4b))

(5.2-3(4c))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

Page 583: Applications of Markov Chains in Chemical Engineering

577

l.in.l

Cu

Cl2

Cl3

Ci4

Cl,in,l

1

0

0

0

0

Cii

Pin.l

PU

0

0

0

Cl2

0

P12

P22

0

0

Cl3

0

0

P23

P33

0

Ci4

0

0

0

P34

P44

p =

(5.2-3(4d))

From Eqs.(5-28a) to (5-30a), assuming C'14 = C14, i.e. no accumulation in reactor

4, noting that Lfi(n) = 0, i = 1, 2, 3, it follows:

Cii(n+1) = Ci,injPinj + Cii(n)pii Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22

Ci3(n+1) = Ci2(n)p23 + Ci3(n)p33 Ci4(n+1) = Ci3(n)p34 + Ci4(n)p44

(5.2-3(4e))

Pin,i = m At pii = 1 - ai2 [i\M P12 = ai2|i2At

P22 = 1 - OC23 ^2At P23 = a23l 3At

P33 = 1 - P34 3At P34 = p34NAt P44 = 1 " P34 4At (5.2-3(4f))

Eqs.(5.2-3(4e)) and (5.2-3(4f)) make it possible to calculate the concentration

distributions of the salt versus time in reactors 1 to 4. In the numerical solution it

was assumed that 1x1 = 112 = ^3 = ^4 = M- The initial state vector S(0) = [Cijn,!,

Cii(O), Ci2(0), Ci3(0), Ci4(0)] = [1, 0, 0, 0, 0] in cases a to c in Fig.5.2-3(4a)

and [0, 1, 0, 0, 0] in case d; At = 0.005. The transient response of Cu (i = 1, 2, 3,

4), i.e. the salt concentration in reactors 1 to 4, is depicted in Fig.5.2-3(4a). The

effect |Li = 1, 10 is demonstrated in cases a and b; the effect of a = 0.1,0 in cases b

and c. The effect of the initial concentration in reactor 1 is demonstrated in case d.

Page 584: Applications of Markov Chains in Chemical Engineering

578

1= 1, a = 0.1, C =1 l.in.l

U

1.5r

1

0.5

0

C.(0) = 0(1 = 1,2,3,4) ii

1 1 1 1

.-'^ - \ 1 = 1 . - . - - ^ ' ' - " /

_ / , ' 2 / 3 .'4

/ / / / / / , ' L i i j i \ L

A

(a)

4 6 t

8 10

i = 10, a = 0, C =1 l.in.l

C (0) = 0 (i = 1,2,3,4)

U

1.5

1

0.5

0

li

1 1 1

i = l

1 /l/l.\

/ / • ' / • •

L - : 1 L

(c)

|x = 10, a = 0.1, C =1 l.in,l

C_.(0) =

1

= 0( i =

JL '

= 1,2,3,4)

J .

2 /• 1

h / ^ • / / ' / 3 / 4

n^// -' / / ' / . ' ' \:J- 1 1

1

1

-j (b)

A

0.5 1.5

i = 10, a = 0.1, C =0 l.in.l

C(0) = 1,C.(0) = 0(1 = 2,3,4) 11 ii

0 0.5 1 1.5 0 0.5 1 1.5 t t

Fig.5.2-3(4a). Cn (i = 1, 2, 3, 4) versus t, demonstrating the effect of |LI and a

5>2-3(5) - Electrolysis of solutions-model A [86] The following scheme was suggested as a possible network model to

describe a real electrolytic processes. Reactors 1 and 2 are continuous-flow stirred-tank electrolytic reactors (CSTER), reactor 3 is a reactor for the recycling electrolyte and reactor 4 is collector in which no electrolytic process takes place.

Page 585: Applications of Markov Chains in Chemical Engineering

579

Fig.5.2-3(5). The electrolyser flow system

The configuration in Fig.5-1 is reduced to that in Fig.5.2-3(5) by choosing

reactors j , a, b and ^, designated asj = l , a = 2, b = 3 and ^ = 4, respectively.

From Eqs.(5-18a)-(5-18c), for a'k = 0 and taking Qi as reference flow, it follows:

P34 = 1, a23 = a3i = a as well as that ai2 = 1 + OC23 (5.2-3(5a))

where a = Q23/Q1 is the recycle. From Eq.(5-la) the state space reads:

SS = [Ci4n,b Cii, C12,, Ci3, C14] (5.2-3(5b))

where Ci,in,l is the concentration at the inlet to reactor 1 of the species designated

by 1 undergoing electrolysis in reactors 1 and 2. From Eq.(5-3a) the state vector

reads:

S(n) = [Ci,in,l, Cii(n), Ci2(n), Ci3(n), Ci4(n)] (5.2-3(5c))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

P =

l.in,l

Ci i

C12

Cl3

Ci4

Cl,in,l

1

0

0

0

0

Cii

Pin.l

Pll

0

P31

0

C12

0

P12

P22

0

0

Cl3

0

0

P23

P33

0

Ci4

0

0

P24

0

P44 (5.2-3(5d))

Page 586: Applications of Markov Chains in Chemical Engineering

580

From Eqs.(5-28a)-(5-30a), assuming C'i4= C14, i.e. no accumulation in reactor 4 of species 1, it follows:

Cii(n+1) = Ci j[n,l(n)pin,l + Cii(n)pii + Ci3(n)p3i + Lii(n)

Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22 + Li2(n)

Ci3(n+1) = Ci2(n)p23 + Ci3(n)p33 Ci4(n+1) = Ci2(n)p24 + Ci4(n)p44

where (5.2-3(5e))

Lii(n) - ^iijACu i(n)At« finjtC*! - Cii(n)]At- ji^ [0 - Cii(n)]At

Li2(n) - ^i2.iACi2,i(n)At- |ii2.i[Ci2 " C^2^^)]M^ Hj .i O - Cj2(n)]At

mi. i = kMAiA i (112,1 = kMA2A 2 (5.2-3(50)

Wp.i (1/sec) are mass transfer coefficients for the transfer of solute f in process p (=

1 to designate an electrolytic process) from the bulk of the solution to the electrode

in reactor i. f = 1, p = 1 and i = 1 means the mass transfer coefficient for the

transfer of solute 1 in process 1 in reactor 1; f = 1, p = 1 and j = 2 stands for the

mass transfer coefficient for the transfer of solute 1 in process 1 in reactor 2. ku is

the mass transfer coefficient in m/sec, Ai and Vi are, respectively, the electrode area

and the effective electrolyser volume. It is assumed [86] that the electrolytic

process takes place under limiting current conditions, i.e. the solute concentrations

on the surface of the electrode, C*ii = C*i2 = 0. The probabilities are as follows:

Pin,l = JA At Pll = 1 - ai2 mAt pi2 = ai2M'2At

P22 = 1 - ai2fA2At P23 = a23^3At p24 = P24M4At = M4At

R33 = 1 - 031 wAt P31 = a3iM4At P44 = 1 - M4At (5.2-3(5g))

where 031 = 023 = Q23/Q1 = a is the recycle. Eqs.(5.2-3(5e)) to (5.2-3(5g)) make

it possible to calculate the concentration distributions of Cu versus time in reactors i

= 1, 2, 3 and 4. In the numerical solution, fully described by a, [ii, M-n.b ^11.2

and dt, it was assumed that m = ^2 = M3 = M4 = M** 1^^ ^ ^^ ^^ calculations

were based on ref.[86]. The initial state vector S(0) = [Ci^n,!, Cii(O), Ci2(0),

Page 587: Applications of Markov Chains in Chemical Engineering

581

Ci3(0), Ci4(0)] = [0, 0.04, 0, 0, 0] in cases a to e in Fig.5.2-3(5a), [0.04, 0, 0, 0,

0] in cases f and g and [0.04, 0.04, 0, 0, 0] in case h; At = 0.1. The transient

response of Cn (i = 1, 2, 3,4) in reactors 1 to 4, is depicted in Fig.5.2-3(5a). The

effect of the recycle a = 0.5, 5 is demonstrated in cases a and b ; the effect of |i =

0.002, 0.02, 0.2 in cases a, c and d; the effect of the mass transfer coefficient

Hi 1,1 = Hi 1,2 = 0.0094, 0.2 in cases b and e as well as in cases f and g in which

^11,1 = ^11,2 = 0.0094, 0, respectively. Note that cases g and h demonstrate

absence of an electrolytic process. The effect of the location of the introduction of

species Ai into the system is shown in cases g and f. In cases a to e, Ai was

introduced initially into reactor 1; in cases f and g it was introduced into the inlet of

reactor 1 where the initial concentration in this reactor as well as in the others was

zero. Only in case h, Ai was introduced both in reactor 1 and continuously at the

inlet to it.

u

o.osp

0.021

0.01

0|

0

p

)

k

Y

a = 0.5, i = 0.02

C =0,C (0) = 1.in.1 i r '

\ l i = ll

1 2 . \

14. - \ > S l iiL

., u = 0.0094 11.1

0.04,C .(0) = 0 (i = 2,3,4)

^^:-^_--:~.. ^ .

1

(aj

-

,—

(b^ a = 5, ^ = 0.02, i = 0.0094 42J 11.1 '

C =0,C (0) = 0.04,C (0) = 0 i.in,i i r ' i r '

0 = 2.3,4)

50 100 150 0

0.04

0.03

0.02

0.01

0

- \

-

-/*

a = 5,

C = 1.in.1

\ l i =

12

13_

^ =

= 0,C

11

_ —

0.002,

,(0) =

i = 0.0094 ^ J 11.1

0.04,C .(0) = 0

(1 = 2,3.4) 1

H

14 A

1 - 50 100 150

a = 5, ^ = 0.2, i = 0.0094 ^^1

(i=2,3,4)

Page 588: Applications of Markov Chains in Chemical Engineering

582

u

0.04

0.03

0.02

0.01

0

a = 5, n = 0.02, i = 0.2 ^q

\ C =0,C (0) = 0.04,C (0) = 0 \ i.in.i ' i r ' ' ir ' J \ (i = 2,3,4) 1

- \ i l = ll J

12 \v...,^^^ 14

'— . — . - - • 1 - |- • • • - _ 10

0.025,

0.02

0.015

0.01

0.005

^ 0

15 0

(f)

a = 5, ^ = 0.02, i = 0.0094 J y • 11.1

I; C =0.04, C (0) = 0 (i = 1^) I pi' 1,in,l ' r ' ^ ' - I

_J \ \ I 100 200 300 400 500 t

Op-

0.035 0.03

0.025

0.02

0.015

0.01 0.005

0 0 100 200 300 400 500 600 -100 0 100 200 300 400 500 600

t t

Fig.5.2-3(5a). Cii (i = 1, 2, 3, 4) versus t, demonstrating the effect of a, |LI, |LIII,I and the introduction location of species Ai

C . =0.04,C.(0) = 0 (i = 1-4) 1,in,1 1i

_L I \ \ I

(h)-^

a = 5, ^ = 0.02, Li = 0 n 11,1 '

C =C (0) = 0.04,C (0) = 0| i.in.i i r ' i r ' - j

(i = 2.3,4) I I I 1 r ^

5>2-3(6) - Electrolysis of solutions-model B [86] In this configuration, a CSTER is imbedded between two perfectly-mixed

reactors 1 and 3 in the forward loop. As in model A above, electrolyte recycling is represented by a perfectly-mixed reactor 4 in the feedback loop shown in Fig.5.2-3(6). Electrolysis takes place in reactor 2 and the collector is reactor 5.

Fig.5.2-3(6). The electrolyser flow system

Page 589: Applications of Markov Chains in Chemical Engineering

583

From Eqs.(5-18a) to (5-18c), for a'k = 0 and taking Qi as reference flow, it

follows:

P35 = 1, OC12 = a23, CX34 = (X41 = a and ai2 = 1 + QLU (5.2-3(6a))

where a = Q34/Q1 is the recycle. From Eq.(5-la) the state space read:

SS = [Ci,in,b Cii, C12,, Ci3, Ci4, C15] (5.2-3(6b))

where Ci^n,! is the concentration at the inlet to reactor 1 of the species designated

by 1 undergoing electrolysis in reactor 2. From Eq.(5-3a) the state vector reads:

S(n) = [Ci,in,l, Cii(n), Ci2(n), Ci3(n), CuCn), Ci5(n)] (5.2-3(6c))

where S(n+1) is given by Eq.(5-4). The probability matrix given by Eq.(5-27) is

reduced to:

Cl,in,l Cii C12 Ci3 Ci4 Ci5

Cl,in,l

Cll

C12

Cl3

Ci4

Cl5

1

0

0

0

0

0

Pin.l

Pll

0

0

P41

0

0

P12

P22

0

0

0

0

0

P23

P33

0

0

0

0

0

P34

P44

0

0

0

0

P35

0

P55

p =

(5.2-3(6d))

From Eqs.(5-28a)-(5-30a), assuming C'14 = C14, i.e. no accumulation in reactor 5

of species 1, it follows:

Cii(n+1) = Ci,i„,i(n)pi„,i + Cii(n)pii + Ci4(n)p4i

Ci2(n+1) = Cii(n)pi2 + Ci2(n)p22 + Li2(n)

Ci3(n+1) = Ci2(n)p23 + Ci3(n)p33 Ci4(n+1) = Ci3(n)p34 + Ci4(n)p44

Ci5(n+1) = Ci3(n)p35 + CisWpss (5.2-3(6e))

where

Li2(n) = jli2^iACj2 j(n)At = ^,2,i[Ci2 - Ci2(n)]At = fl,2_i[0 - Ci2(n)]At

Page 590: Applications of Markov Chains in Chemical Engineering

584

12,1 = kMA2A 2 (5.2-3(6f))

|Xii,2 (1/sec) is the mass transfer coefficient for the transfer of solute 1 in process 1

(electrolysis) in reactor 2. kM is the mass transfer coefficient in m/sec, Ai and Vi

are, respectively, the electrode area and the effective electrolyser volume. It is

assumed [86] that the electrolytic process takes place under limiting current

conditions, i.e. the solute concentrations on the surface of the electrode, C*i2 = 0.

The probabilities are as follows:

Pin,i = m At pii = 1 - ai2mAt P12 = ai2|ii2At

P22 = 1 - CXl2 2At P23 = OC23M'3At

P33 = 1 - 0Ci2H3At P34 = a34^4At P35 = p35^4At = ^sAt

P44 = 1 - cx4m4At P41 = a4im At P55 = 1 - ^sAt (5.2-3(6g))

noting that a i2 = a23 = 1 + OC34 and a4i = a34 = Q34/Q1 = a is the recycle.

Eqs.(5.2-3(6e)) to (5.2-3(6g)) make it possible to calculate the concentration

distributions of Cn versus time in reactors i = 1,..., 5. In the numerical solution,

fully described by a, m, 11,2 and At, it was assumed that m = |i. The initial state

vector S(0) = [Ci,in,b Cii(O), ..., Ci5(0)] = [0, 0.04, 0, 0, 0, 0] in cases a to d in

Fig.5.2-3(6a) and [0.04, 0, 0, 0, 0, 0] in case e and f; At =0.1, 0.5 in cases a to d

and e, f, respectively. The transient response of CH (i = 1, 2, 3) in reactors 1, 2,

3, is depicted in Fig.5.2-3(6a). The effect of the recycle a = 0.05, 5 is

demonstrated in cases a and b; the effect \i = 0.02,0.2 in cases b and c; the effect

of the mass transfer coefficient ii 1,2 = 0.0094 and 0.2 in cases c and d as well as e

and f. The effect of the introduction location of species Ai , i.e. the inlet

concentration to reactor 1, Cijn,! = 0.04, while the initial concentration in all

reactors is zero, is demonstrated in cases e and f for |i 11 2 = 0.0094, 0 which

demonstrate again the effect of |Lii 1,2. Note that in case f no electrolysis takes place

in reactor 2 indicated by |Lii 1,2 = 0.

Page 591: Applications of Markov Chains in Chemical Engineering

585

u

(a) a - 0.05, ^ - 0.02, ji - 0.0094

11,2

C = 0, C (0) = 0.04. C = 0 l.in.l 11 li

(i = 2-5)

h-

\ l

\ns^

h ^

» « (bj a - 5, ji - 0.02, \i - 0.0094 |

11,2

C = 0, C (0) = 0.04, C = 0 J i.iii.i i r ' li n

(i = 2-5)

1 1 H 150 0 50 100 150

0.04

0.03

^0.02

0.01

' ' (c)|

a - 5, fi - 0.2, \i - 0.0094 11,2

C = 0, C (0) = 0.04, C = 0 J l.in,l 11 li ^

(i = 2-5)

' ' (d)|

a - 5, ji - 0.02, n - 0.2 11,2

C = 0, C (0) = 0.04, C = 0 J i.in.i i r ' li '

(i = 2-5)

15 0

U

0.04

0.03 -0.02

0.01

-

"" F"

/ ^ jr a -

C

l,in.

1

5

i""

•• 1 1

^^ . ^

H m 0.02, n -11.2

0.04. C = 0 (i =

li

1 i

' (e)|

i = l j 2, 3"1

- j 0.0094

1-5) H

1

i ' 1 « ( f ^

^fn'"^ 1, 2, 3

ly

/ a - 5, ji - 0.02, M. - 0 11,2

C =0.04. C = 0 (i = l-5) 1 l.in.l li

i i i , i , . 0 200 400 J 600 800 1000 0 200 400 600 800 1000

Fig.5.2-3(6a). C n (i = 1, 2, 3) versus t, demonstrating the effect of

a, |i9 M*! 1,2 And the introduction location of species Ai

5.2-3(7) - Simultaneous dissolution, absorption and chemical reaction

In the following, simulation is carried out of a combined process

incorporating dissolution, absorption of species f = 1 takes place into the solution

as well as reacting with species 2 arriving from reactor 2 at flow rate Q12 according

to

Page 592: Applications of Markov Chains in Chemical Engineering

586

Ai + A2 ^ A3 for which - n = -12 = 13 = k2CiC2 (5.2.3(7a))

The feed to reactors 1 and 2 may contain species 1 and 2 at concentrations Cf,in,i

and Cf4n,2 where f = 1,2.

Qi

Q2

ri r\ 1

j = l absorption + reaction

i

1 ki db

a: dissol

= 2 ution

^13 do ^ - 3

Fig.5.2-3(7). Flow system for the simultaneous dissolution,

absorption and chemical reaction

It is assumed that the quantities dissolved and absorbed do not change the

flow rate Qi + Q2. From Eqs.(5-18a) to (5-18c), for a'k = 0 and taking Qi as

reference flow, it follows:

P i3= l and a 2 i = a 2 = Q2/Qi.

where a = Q2/Q1. From Eq.(5-la) the state space reads:

SS = [Cf,in,l, Cf,in,2, Cfi, Cf2, Cfs] f = 1, 2

From Eq.(5-3a) the state vector reads:

S(n) = [Cf,in,l, Cf,in,2, Cfi(n), Cf2(n), Cf3(n)] f = 1, 2

S(n) = [Cf,in,l, Cf,in,2, Cfi(n), Cf2(n), Cf3(n)] f = 1, 2

(5.2-3(7b))

(5.2-3(7c))

(5.2.3(7c))

where S(n+1) is given by Eq.(5-4). The probabiUty matrix for f = 1, 2, given by

Eq.(5-27), is reduced to:

Page 593: Applications of Markov Chains in Chemical Engineering

587

Cf,in,l Cf,in,2 Cfi Cf2 Co

Cf,in.l

Cf,in,2

P = Cfi

Cf2

Cl3

1

0

0

0

0

0

1

0

0

0

Pin,l

0

Pll

P21

0

0

Pin,2

0

P22

0

0

0

P13

0

P33 (5.2-3(7d))

From Eqs.(5-28a) to (5-30a), assuming C'o = Cf3, i.e. no accumulation of the

species in reactor 3, it follows for f = 1,2 that:

Cfi(n+1) = Cf,in,l(n)Pin,l + Cfi(n)pii + Cf2(n)p2i + Lfi(n)

Cf2(n+1) = Cf,in,2(n)pin,2 + Cf2(n)p22 + Lf2(n)

Cf3(n+1) = Cfi(n)pi3 + Cf3(n)p33 (5.2-3(7e))

where

Lfi(n) = [Hfj jACfi j(n) + rfj(n)]At

Lf2(n) = if2.2ACf2,2(n)At (5.2-3(7f))

In |Xfp,i the following designations are applicable: f = 1, 2 indicate species 1 and 2,

respectively; p = 1, 2 stand for processes of absorption and dissolution,

respectively; i = 1, 2 indicate reactors 1 and 2, respectively. Thus,

L,,(n) = [^ij jACji i(n) + ri,(n)]At = [^l,,,(C*, - CjjCn)) - kjCijCnKji (n)]At

LjiCn) = [r2i(n)]At = - kjC, j(n)C2i (n)]At

Li2(n) = 0

L22(n) = \l22^2^C22^2^TlW = 22,2 22 " C22(n)]At (5.2-3(7g))

The probabilities are as follows:

Pin,l=|ilAt Pin,2 = a2mAt pii = l-M,iAt p2i = a2imAt

P22 = 1 - a2m2At P13 = |X3At P33 = 1" m^i (5.2-3(7h))

Page 594: Applications of Markov Chains in Chemical Engineering

588

noting that a2i = a2. Eqs.(5.2-3(7e)) to (5.2-3(7h)) make it possible to calculate

the concentration distributions of Cfi versus time in reactors i. In the numerical

solution, fully described by a2, m, M-ii,!, 22,2* 2, C*ii, C*22 and At, it was

assumed that |Xi = p,, iin j = 1x22,2 = 1. In addition, C*ii = C*22 = 1 in cases a to

e and C*ii = C*22 = 0 in case f. The initial state vector S(0) = [Cfjn,b Cf,in,2»

Cfi(O), Cf2(0), Cf3(0)] = [0, 0, 0, 0, 0] for f = 1, 2 in cases a to e in Fig.5.2-3(7a)

and [0, 0, 1, 0, 0] for f = 1 and [0, 0, 0, 1, 0] for f = 2 in case f; At = 0.002. The

effect of |i = 0, 1, 10 is demonstrated in cases a, b and c; the effect of k2 = 1, 10 in

cases b and d; the effect of a2 = 1, 10 in cases b and e. In case f, no dissolution or

absorption take place; however chemical reaction occurs since the initial

concentrations of A1 in rector 1 and of A2 in reactor 2 were unity. The latter

species was transferred by the flow to reactor 1 undergoing there a chemical

reaction.

Page 595: Applications of Markov Chains in Chemical Engineering

589

C , =C , = 1, C (0) = 0 (i = 1-3, f= 1, 2) 11 22 f 1

U

1

0.8

0.6 a

0.4

0.2

0

Li = 0, a = 2

1 - -r

f//" /

r 1 1

= l , k = 1 2

1

' (^

-

0

u

1 4

1.2

1

0.8

0.6

0.4

0.2

0

-

1

/

1

/

---

1= 1, a = 10,k =1 ^ ' 2 2 1 i 1

2 V - ^ /

fi = l l

_12

1 1 1

_ ^\

-J

-

- • -=

0

C =C =1,C (0) = 0 0=1-3, f= 1,2) 11 22 fi ^ '

H = 0, a = 1, k =1 2 2 1 1

22

11 ^ —

" / / ^ ' ' 7 / / ' " J/ / r 1 1

' (b)

-

1

= C =1,C (0) = 0 ( i= l -3 , f= l ,2 )

^ = l .

1

2 2 _ _

" /

a = 2

1

I L

1, k =10 2

1

" ^

1

(d)

0 1 2 3 4 t

C* =C* =0,C (0) = C (0)=1,C(0) = 0 11 22 ir 22 fi

Fig.5.2-3(7a). Cn, C21 and C22 versus t, demonstrating the effect of |Li, k2, and a2

Page 596: Applications of Markov Chains in Chemical Engineering

590

NOMENCLATURE

a, b , ..., Z designation of peripheral reactors in Figs.4-1 and 4 - l a .

aj * stoichiometric coefficients of species j in the /th reaction,

aj stoichiometric coefficients of species j .

hi* ^hj heat transfer area to reactors i and j , m^.

api, apj mass transfer area for process p corresponding to conditions

prevailing in reactors i or j , m^.

A, B ZxZ square matrices given by Eq.(2-34).

A^ = fk designates a system with respect to its chemical formula f and

location k in the flow system. Aj = i designates the state of the system (a molecule), i.e. a specific

chemical formula (Chapter 3).

Ai(t) = Ai concentration of a chemical species i at time t (Chapter 2).

Ai(0) initial concentration of species i.

ajk» bjk elements of the matrices A, B.

CA(t), CAO concentration of A at time t and t = 0, respectively.

Cj Chapter 4: concentration of the stimulating input in reactor j in

moles/(m^ reactor); concentration of species j , moles of j/m^» also

designating the state of the system.

Cj(n), Cj(n+1) concentration of species j in the mixture at time interval t and t + At

or step n and n+1, respectively.

C(n+1) concentration vector at step n+1.

Cj concentration of species j in reaction i.

C*2 equilibrium concentration of species 1 absorbed on the surface of the

liquid in reactor 2 corresponding to its partial pressure in the gas

phase above the Uquid.

C* J as above, but for species 1 in reactor 1.

Cfi, Cfj concentration of species f in reactors i or j , kg or kg-mol/m^.

C'fi. C'fj concentration of species f leaving reactors i or j , kg or kg-mol/m^.

These concentrations are not, in general, equal to the concentrations

Cfi, Cfj in the reactor.

Page 597: Applications of Markov Chains in Chemical Engineering

591

Cfi(n)

C'fi(n)

C'f^(n)

C'4

Cf,in,j

Cf,in,i

CSTER

Di(%)

I-\ne

Dmax

E

eq.

exact

f

fi, fj

concentration of species f in reactor i at time t or at step n, kg or

kg-mol/m^.

concentration of species f leaving reactor i at time t or at step n, kg or

kg-mol/m^.

concentration of species f in reactor ^, kg or kg-mol/m^.

concentration of species f leaving reactor ^, kg or kg-mol/m^.

This concentration is not, in general, equal to the concentrations

concentration of species f leaving reactor ^ at time t or at step n, kg

or kg-mol/m^.

concentration of the tracer in reactor ^, kg or kg-mol/m^.

concentration of the tracer leaving reactor ^, kg or kg-mol/m^.

This concentration is not, in general, equal to the concentrations

concentration of species f at the inlet to reactor j , kg or kg-mol/m^.

concentration of species f at the inlet to reactor i, kg or kg-mol/m^.

specific heat of the fluid mixture in reactor j , kcal/(kg K); similarly

for reactor i.

continuous-flow stirred-tank electrolytic reactor

deviation in the concentration for the /th pair of data between the

exact solution and Markov chain solution , i.e.,

Di(%) = 100ICexact,i " CMarkov,il/Cexact,i-

mean deviation defined by

'mean )ZPi ^) = (1/H) / D (%) where H is the number of pairs.

i=l considered in the comparison.

maximum value of Di(%).

age distribution defined by Eq.(4-20).

at equilibrium.

exact solution

running index, 1, 2,..., F, for designating the different species.

subscript; designating species f in reactors i or j .

Page 598: Applications of Markov Chains in Chemical Engineering

592

fjj probability that, starting from state Sj, the system will ever pass

through Sj. Defined in Eqs.(2-95).

fjk probability that, starting from Sj, the system will ever pass through

Sk. Defined in Eqs.(2-98).

probability that state Sj is avoided at steps (times) 1, 2, . . . , n - 1

where re-occupied at step n. Defined in Eqs.(2-94).

fjk(n) indicates that Sk is avoided at steps (times) 1,..., n-1 and

occupied exactly at step n, given that state Sj is occupied initially.

Defined in Eqs.(2-97).

total number of species or reactants.

specific enthalpy of the fluid in the feed vessels to reactor i, kcal/kg.

specific enthalpies of the fluid in reactors i, j and in collector ^,

kcal/kg or kg-mole.

specific enthalpies of the fluid in reactors i and in collector ^ at time t

or step n, kcal/kg or kg-mole.

specific enthalpies of the fluid leaving reactors i, j and ^, kcal/kg or

kg-mole.

H the rate of supply of Ai in moles/sec from the vapor phase (Ai)gas

into the condensed phase,

j , k integers designating states j and k, respectively.

J, K total number of cities in the extemal circle and in the intemal city,

respectively,

kj reaction rate constant with respect to the conversion of species j in

the /th reaction (in consistent units),

k, kj reaction rate constant (Chapter 3).

kij reaction rate constant for the conversion from state i (species A[) to

state j (species Aj).

kfp,i, kfp j mass transfer coefficients for process p with respect to species f

corresponding to conditions in reactors i and j , m/s.

khi, khj heat transfer coefficient corresponding to the conditions in reactors i

andj, kcal/(sm2K).

kM mass transfer coefficient, m/s.

LHS left hand side.

fjj(")

fjk(n)

F

hin,i hi, hj, h^

hi(n), h^(n)

h'i, h'j, h'

Page 599: Applications of Markov Chains in Chemical Engineering

593

L(n) a mathematical expression corresponding to time t or step n.

n designates step n in discrete Markov chains.

NA(t), NAO number of moles of A at time t and t = 0, respectively.

No total number of inhabitants in the state.

Nj(t) number of inhabitants occupying state j (an external city j) at time t.

Nk(t) number of inhabitants occupying state k (an internal city k) at time t.

N® number of reacting species in reaction i.

p(y, T, X, t) probability density function, i.e. probability per unit length.

P(y, T, X, t) transition probability function defined by Eq.(2-185).

p, q constant one-step transition probabilities.

p subscript designating some transfer mechanism such as absorption,

dissolution, etc., or simultaneously several processes. It is assigned

arbitrarily numerical values such as absorption - 1 , dissolution - 2.

P total number of transfer mechanisms.

Pin4> Pin j single step transition probabilities from the state of the feed reactors

(to reactors i and j) to the state of reactors i and j .

Pini h single step transition probability with respect to enthalpy (or

temperature) from the state of the inlet reactor (to reactor i) to the

state of reactor i; similarly for reactor j , i.e., p • j .

p-j ij single step transition probability to remain in the state of reactor j

with respect to enthalpy (or temperature). Py jj single step transition probability with respect to enthalpy (or

temperature) from the state of reactor k to the state of reactor j .

Pj i jj single step transition probability with respect to enthalpy (or

temperature) from the state of reactor k to the state of reactor i. Pj • h single step transition probability with respect to enthalpy (or

temperature) from state of reactor j to state of reactor i.

Pjj Yi single step transition probaHlity to remain in the state of reactor i

with respect to enthalpy (or temperature),

pk, qk one-step transition probabilities which depend on the state k.

Pjk> Pkj one-step probability or the transition probability from state j to state k

(or the opposite) in one step (one time interval) for each j and k;

Page 600: Applications of Markov Chains in Chemical Engineering

594

probability of occupying state k after one step given that the system occupied state j before. Defined in Eqs.(2-13,13a) and (4-4,4-4a).

PJE' Pka transition probabilities from reactor j to , from k to , respectively.

Pjk the probability of the transition Aj -> Aj for the ith reaction.

Pjj, ptt probability of remaining in step j during one step; probability of

remaining in reactor .

Pjj(n) probability of occupying Sj after n steps (or at time n) while initially

occupying also this state.

pjk(n) n-step transition probability function designating the conditional

probability of occupying Sk at the nth step given that the system

initially occupied Sj.

Pjk(n,r) probability of occupying state k at step r given that state j was

occupied at step n.

Pjk('Cit) transition probability of a system to occupy state k at time t subjected

to the fact that the system occupied state j at time T.

Pi(t) probability that the number N(t) of events occurred (customers

arrived) is equal to i, given that the service time is t; defined in

Eq.(2-89). Probability of the system to occupy Sj at time t; defined

inEq.(2-112).

Px(t) defined in Eq.(2-119).

Po(t) probability that the system remains at x = 0, i.e. state SQ, until time t.

P one-step transition probability matrix defined by Eq.(2-16).

P(n) a probability matrix containing the elements pjk(n). Defined in

Eq.(2-31).

prob{ Sj} probability of observing event Sj. Probability of occupying state Sj.

prob{ Sk I Sj} conditional probability. Probability of observing an event Sk under

the condition that event Sj has already been observed. Probability

of occupying state Sk under the condition that state Sj has already

been occupied,

prob {SkSj} probability for the intersection of events Sk and Sj or probability

of observing, not simultaneously, both Sk and Sj.

qj(t) rate or intensity function indicating the rate at which inhabitants

leave state Sj (extemal city j), 1/s.

Page 601: Applications of Markov Chains in Chemical Engineering

595

Qjk(0 transition probability of the inhabitants from Sj to occupy Sk at

time t.

Qi» Qj volumetric flow rate of the fluid entering reactors i and j , Figs.4-1

and 5-1, m^/s.

Qi» Qj volumetric flow rate of the fluid leaving reactors i and j , Fig.5-1,

m % . This flows do not contain any dissolved material.

Qji and Qy interacting flows between reactors (states) j and i or i and j ,

respectively, Fig.4-1, m^/s. Similarly for Qki and Qik.

Qp external flow into reactor P , m^/s.

Or external flows into the reactors, r = j , a, b , . . . , Z, m^/s.

qi^ volumetric flow rate from reactor (state) i to ^ , Fig.4-1, m^/s.

Similarly, qj^, qk^ and qp^.

Tj = dCj/dt rate of change by reaction of the concentration of species j per unit

volume of fluid in reactor.

rj ^ rate of change by reaction at time t of the concentration of species j in

the ith reaction per unit volume of fluid in reactor,

rj (n) rate of change by reaction at step n of the concentration of species j

in the /th reaction per unit volume of fluid in reactor,

r . reaction rate of species f by reaction m per unit volume of reactor (or

fluid in reactor) j corresponding to the conditions in this reactor;

similarly for reactor i. (xri)

r . (n) as above, at time t or step n; similarly for reactor i.

rfj(n) rate of change at step n by all reaction of the concentration of species

f in reactor j which equals Z^^'f (^) where m = 1, ..., R.

R total number of reactions.

RHS right hand side

RTD residence time distribution.

Ri recycle ratio defined in Eq.(4-22).

Sj, Sk designate events or states j and k, respectively. S stands for state

and the subscript] designates the number of the state.

SS state space. Set of all states a system can occupy.

Page 602: Applications of Markov Chains in Chemical Engineering

596

Si(n) occupation probability of state i at time n by the system. Defined in

Eq.(2-21a).

Si(0) initial occupation probability of state i by the system.

S(n) state vector of the system at time n (step n). Defined in Eq.(2-22a).

S(0) initial state vector. Defined in Eq.(2-22).

t, T designates generally time where in a discrete process t designates the

number of steps from time zero, s.

tin mean residence time of the fluid in the reactor, s. Defined in Eq.(4-

26).

mean residence time of the fluid in reactor j , s.

residence time of the fluid in a plug-flow reactor, s.

residence time of the fluid in a plug-flow reactor j , s.

source temperature from which (to which) heat is transferred into

(from) reactor j , K. Similarly for TQJ

temperatures in reactors i and j , K.

temperature at inlet to reactor j , K.

volume of reactor, m .

volume of reactor (or fluid in reactor) j or i, m .

X a prescribed value in Eq.(2-119) indicating the number of events

occurring during the time interval (0, t) = t; x indicates also a

numerical value corresponding to the state of a system, i.e. x = 0, 1,

2, ...

xo initial magnitude of the state.

X(t) a random variable describing the states of the system with respect to

time and referring to Eq.(2-8). It also designates the fact that the

system has occupied some state at time or step t. Referring to

Eq.(2-119) and the following ones, X(t) which is a random

variable, designates the number of events occurring during the time

interval (0, t).

X(n) the position at time or step n of a moving particle (n = 0,1,2, . . . ) .

Xn number of customers in the queue inmiediately after the nth customer

has completed his service.

tmj

tp

tpj

To.i

Ti,Tj

Tinj V

\ ^ i

Page 603: Applications of Markov Chains in Chemical Engineering

597

Yn the number of customers arriving during the service time of the nth

customer.

Z total number of events, states, chemical species or reactors that a

system can occupy.

Z(n) size of the jump of the particle at the nth step.

Greek letters

ttji ratio between the flow from reactor j to reactor i and the total flow.

rate Qj. Similarly, ay, aki, aik defined in Eqs.(4-2), (4-6) as well

as Oji, ttkj, ocjki similarly defined.

tti, ak ratio between the flow entering reactor i or k to the total flow rate Qj.

Defined in Eq.(4-12d),

a'i, a'j defined in Eqs.(5-18), (5-18d).

P the mean number of customers being serviced per unit time,

pj^ defined in Eqs.(4-2), (4-6). Similarly, pr^ r = i, j and k. ACf I driving force for the transfer process p with respect to species f at

conditions prevailing in reactor j ; similarly for reactor i replacing

subscript]; kg or kg-mol/m^.

AC^ j(n) as above at time t or step n, kg or kg-mol/m^.

ATi = To,i - T- on reactor i; similarly for reactor j , K. ATj(n) = To,i - T.(n) at time t or step n, K.

AH^^^ heat of reaction m at conditions in reactor j , kcal/kg or kg-mol.

AHrj heat of reaction at conditions in reactor j , kcal/kg or kg-mol.

X rate factor; rate of arrival of customers or the rates events/time or

births/time.

A.i birth rate which is a function of the state Si.

X^ mean occurrence rate of the events which is a function of the actual

state x; mean birth rate.

At time interval, s.

|i , |X volumetric heat transfer coefficients defined in Eq.(5-22) for

reactors i and j , 1/s.

|lj mean recurrence time defined in Eqs.(2-96). |ij (1/sec) is a measure

of the transition rate of the system defined in Eqs.(4-3).

Page 604: Applications of Markov Chains in Chemical Engineering

598

|Li defined in Eq.(4-11).

|ix death rate in Eq.(2-158). Mean occurrence rate of the events which

is a function of the actual state x. U. o tt. . volumetric mass transfer coefficient for species f, for mass transfer ^fp,i ^fpo

process of type p (for example: absorption, p = 1; desorption, p = 2; dissolution, p =3; etc.) corresponding to reactors i or j ; defined in

Eq.(5-14), 1/s.

V period.

^ subscript designating the collection reactor for the tracer. The final

reactor in the flow system symbolized as a "dead" or an "absorbing

state" for the tracer for which p^^ = 1.

limiting probabilities. Defined in Eqs.(2-105).

stationary distribution of the limiting state vector. Defined in Eqs.(2-

105a).

density of the content of reactors i, j or ^, kg/m^.

density of the streams leaving reactors i, j or ^, kg/m^.

density of the stream entering reactors i or j , kg/m^.

TCk

7C

Pi.

P'i

Pj'P^

. P'j' P'

Pin,i» Pinj

Page 605: Applications of Markov Chains in Chemical Engineering

599

REFERENCES

1. A.A.Markov, Extension of the Law of Large numbers to Dependent Events,

Bulletin of the Society of the Physics Mathematics, Kazan, 15(1906)155-156.

2. A.T.Bharucha-Reid, Elements of the Theory of Markov Processes and their

Applications, McGraw-Hill Book Company, Inc., 1960.

3. R.A.Howard, Dynamic Programming and Markov Processes, The M.I.T.

Press, 1960.

4. D.R.Cox and H.D.Miller, The Theory of Stochastic Processes, Chapman

and Hall Ltd, 1965.

5. Y.A.Rozanov, Introductory Probability Theory, Prentice-Hall Inc., 1969,

(Revised English Edition Translated and Edited by R.A.Silverman).

6. G.G.Lowry (editor), Markov chains and Monte Carlo Calculations in

Polymer Science, Marcel Dekker, Inc., New York, 1970.

7. J.G.Kemeny and J.L.Snell, Finite Markov Chains, D.Van Nostrand

Company, Inc., 1960.

8. M.F.Norman, Markov Processes and Learning Models, Academic Press,

1972.

9. S.T.Chou, L.T.Fan and R.Nassar, Modeling of Complex Chemical

Reactions in a Continuous-Flow Reactor: A Markov Chain Approach,

Chemical Engineering Science, 43, 2807-2815(1988).

10. F.H.Bool, B.Emst, J.R.Kist, J.L.Locher and F.Wierda, Escher-The

Complete Graphic Work, Thames and Hudson, 1995.

11. U.M.Schneede, Rene Magritte Life and Work, Barron's Educational Series,

Inc., 1982.

12. M.Paquet, Rene Magritte 1898-1967, Benedikt Taschen Verlag GmbH,

1994.

13. I.F.Walther, Vincent van Gogh, Benedikt Taschen Verlag GmbH & Co.

KG, Colonge, 1987.

14. A.M.Hammacher, Magritte, Thames and Hudson, 1986.

15. W.Feller, An Introduction to Probability Theory and Its Applications, Vol. 1,

John Wiley & Sons, Inc., 1968.

16. E.Parzen, Modem Probability Theory and Its Applications, John Wiley &

Sons, Inc., 1960.

Page 606: Applications of Markov Chains in Chemical Engineering

600

17. E.Parzen, Stochastic Processes, Holden-Day, Inc., 1962.

18. E.Cinlar, Introduction to Stochastic Processes, Prentice-Hall, Inc., 1975.

19. Moshik, Painted on Weekdays, Zmora, Bitan, Modan - Publisher, 1981.

20. S.Whitfield, Magritte, The South Bank Center, London, 1992.

21. O.Levenspiel, Chemical Reaction Engineering, 2nd.ed., John Wiley and

Sons, Inc., New York, 1972.

22. N.M.Rodiguin and E.N.Rodiguina, Consecutive Chemical Reactions-

Mathematical Analysis and Development, D.Van Nostrand Company, Inc.,

Princeton, New Jersey, 1964.

23. A.Apelblat, Private Communication, 1996.

24. R.B.Bird, W.E.Stewart and E.N.Lightfoot, Transport Phenomena, John

Wiley & Sons, Inc., 1960.

25. Y.Aharoni, Carta's Atlas of the Bible, 2nd.ed., Carta, Jerusalem, 1974.

26. P.J.Miller, Markov Chains and Chemical Processes, Journal of Chemical

Education, 9, 222-224(1972).

27. S.J.Formosinho and Maria da Graca M.Miguel, Markov Chains for Plotting

the Course of Complex Reactions, Journal of Chemical Education, 56, 582-

585(1979).

28. F.D.Anita and S.Lee, The Effect of Stoichiometry on Markov chain Models

for Chemical Reaction Kinetics, Chemical Engineering Science, 40, 1969-

1971(1985).

29. S.T.Chou, L.T.Fan and R.Nassar, Modeling of Complex Chemical

Reactions in a Continuous-Flow Reactor: a Markov Chain Approach,

Chemical Engineering Science, 43, 2807-2815(1988).

30. T.Z.Fahidy, Solving Chemical Kinetics Problems by the Markov-Chain

Approach, Chemical Engineering Education, 27,42-43(1993).

31. A.A.Frost and R.G.Pearson, Kinetics and Mechanism, John Wiley & Sons,

Inc., 1960.

32. C.H.Bamford and C.F.H.Tipper (Editors), Chemical Kinetics: Volume 1-

The Practice of Kinetics, Volume 2-The Theory of Kinetics, Elsevier

Publishing Company, 1969.

33. C.Capellos and B.H.J.Bielski, Kinetic Systems, Wiley-Interscience, 1972.

34. R.H.Perry and C.H.Chilton, Chemical Engineers' Handbook, McGraw-

Hill Kogakusha, 5th Ed., 1973.

Page 607: Applications of Markov Chains in Chemical Engineering

601

35. G.M.Fleck, Chemical Reaction Mechanism, Holt, Rinehart and Winston,

Inc., 1971.

36. Chien, J.Y., Kinetic Analysis of Irreversible Consecutive Reactions, Journal

of the American Chemical Society, 70, 2256-2261(1948).

37. McKeithan, T.W., Kinetic Proofreading in T-Cell Receptor Signal

Transduction, Proc. Natl. Acad. Sci. USA, 92, 5042-5046(1995)

3 8. O.Levenspiel, The Chemical Reactor OmnibooK Distributed by OSU Book

Stores, Inc., Corvallis, OR 97330, 1979.

39. S.W.Benson, The Foundation of Chemical Kinetics, McGraw-Hill Book

Company, Inc., 1960.

40. C.T.Chen, The Kinetic System Consisting of Two Pairs of Competitive

Consecutive Second-Order Reactions, The Joumal of Physical Chemistry,

62, 639-640(1958).

41. G.Natta and E.Mantica, The Distribution of Products in Series of Consecutive

Competitive Reactions, Joumal of the American Chemical Society, 74, 3152-

3156(1952).

42. A.G.Fredrickson, Stochastic Triangular Reactions, Chemical Engineering

Science, 21,687-691(1966).

43. W.C.Schwemer and A.A.Frost, A Numerical Method for the Kinetic Analysis

of Two Consecutive Second Order Reactions, Joumal of the American

Chemical Society, 73, 4541-4542(1952).

44. M.C.Tobin, A simple Kinetic Method for Some Second-Order Reactions,

The Joumal of Physical Chemistry, 59, 799-800(1955).

45. W.J.Svirbely, The Kinetics of Three-step Competitive Consecutive Second-

order Reactions, Joumal of the American Chemical Society, 81, 255-

257(1959).

46. W.J.Svirbely and J.A.Blauer, The Kinetics of Three-step Competitive

Consecutive Second-order Reactions. II, Joumal of the American Chemical

Society, 83, 4115-4118(1961).

47. A.A.Frost and W.C.Schwemer, The Kinetics of Consecutive Competitive

Second-order Reactions: The Saponification of Ethyl Adipate and Ethyl

Succinate, Joumal of the American Chemical Society, 74,1268-1273(1952).

48. K.J.Laider, Reaction Kinetics: Volume I-Homogeneous Gas Reactions,

Pergamon Press, The Macmillan Company, 1963.

Page 608: Applications of Markov Chains in Chemical Engineering

602

49. E.S.Swinboume, Analysis of Kinetic Data, Appleton-Century-Crofts, New

York, Meredith Corporation, 1971.

50. V.K.Weller and H.Berg, Polarographische Reaktiongeschwindigkeitsmessungen,

Berichte der Bunsengesellschaft, 68, 33-36(1964).

51. S.W.Benson, The Induction Period in Chain Reactions, The Journal of Chemical

Physics, 20, 1605-1612.

52. F. A.Masten and J.L.Franklin, A General Theory of Coupled Sets of First Order

Reactions, Journal of the American Chemical Society, 72, 3337-3341(1950).

53. O.Levenspiel, Chemical Reaction Engineering, John Wiley and Sons, Inc.,

New York, 1962.

54. C.G.Hill, An Introduction to Chemical Engineering Kinetics & Reactor

Design, John Wiley and Sons, Inc., New York, 1977.

55. A.J.Lotka, Contribution to the Theory of Periodic Reactions, The Journal of

Physical Chemistry, 14, 271-274(1910).

56. R.J.Field, E.Koros and R.M.Noyes, Oscillations in Chemical Systems. IL

Through Analysis of Temporal Oscillations in the Bromate-Cerium-Malonic

Acid System, Journal of the American Chemical Society, 94, 8649-

8664(1972).

57. K.Showalter, R.M.Noyes and K.Bar-Eli, A Modified Oregonator Model

Exhibiting Complicated Limit Cycle Behavior In a Flow System, Journal of

Chemical Physics, 69, 2514-2524(1978).

5 8. D.T.Gillespie and M.Mangel, Conditioned Averages in Chemical Kinetics,

Journal of Chemical Physics, 75, 704-709(1981).

59. P.De Kepper and K.Bar-Eli, Dynamical Properties of the Belousov-

Zhabotinski Reaction in a Flow System. Theoretical and Experimental

Analysis, The Journal of Physical Chemistry, 87, 480-488(1983).

60. K.Bar-Eli, On the Stability of Coupled Chemical Oscillators, Physica, 14D,

242-252(1985).

61. V.Gaspar and K.Showalter, The Oscillatory Reaction. Empirical Rate Law

Model and Detailed Mechanism, Journal of the American Chemical Society,

109, 4869-4876(1987).

62. E.C.Edblom, L.Gyorgyi, M.Orban and I.R.Epstein, A Mechanism for

Dynamical Behavior in the Landolt Reaction with Ferrocyanide, Joumal of

the American Chemical Society, 109,4876-4880(1987).

Page 609: Applications of Markov Chains in Chemical Engineering

603

63. V.Gaspar and K.Showalter, Period Lengthening and Associated Bifurcations

in a Two-Variable, Flow Oregonator, The Journal of Physical Chemistry,

88, 778-791(1988).

64. K.Bar-Eli and R.J.Field, Simulation of the Minimal BrOs' Continuous-Flow

Stirred Tank Reactor Oscillator on the Basis of a Revised Set of Rate

Constants, The Journal of Physical Chemistry, 94, 3660-3663(1990).

65. V.Gaspar and K.Showalter, A Simple Model for the Oscillatory lodate

Oxidation of sulfite and Ferrocyanide, The Journal of Physical Chemistry,

94, 4973-4979(1990).

66. M.Brons and K.Bar-Eli, Asymptotic Analysis of Canards in the BOB

Bquations and the Role of the Inflection Line, Proceedings of the Royal

Society, London, A445, 305-322(1994).

67. CLaszlo and R.J.Field, Simple Models of Deterministic Chaos in the

Belousov-Zhabotinski Reaction, The Journal of Physical Chemistry, 95,

6594-6602(1991).

68. C.G.Steinmetz, T.Geest and R.Larter, Universality in the Peroxidase-

Oxidase Reaction: Period Doubling, Chaos, Period Three, and Unstable

Limit Cycles, The Journal of Physical Chemistry, 97, 5649-5653(1993).

69. C.G.Steinmetz and R.Larter, The Quasiperiodic Route to Chaos in a Model

of the Peroxidase-Oxidase Reaction, Journal of Chemical Physics, 94,

1388-1396(1991).

70. M.Marek and LSchreiber, Chaotic Behavior of Deterministic Dissipative

Systems, Cambridge University Press, 1995.

71. C.Y.Wen and L.T.Fan, Models for Flow Systems and Chemical reactors.

Marcel Dekker, inc. New York, 1975.

72. A.Petho and R.D.Noble, Residence Time Distribution Theory in Chemical

Bngineering, Verlag Chemie, 1982.

73. A.Tamir, Impinging-Stream Reactors, Fundamentals and Applications,

Elsevier, Amsterdam, 1994. Also translated to Chinese and published by the

Chemical Industry Press of China, 1996.

74 J.G.van de Vusse, A new Model for the Stirred Tank reactor. Chemical

Engineering Science, 17, 507-521(1962).

Page 610: Applications of Markov Chains in Chemical Engineering

604

75. L.G.Gibilaro, H.W.KrophoUer and D.Spikis, Solution of a Mixing Model

due to van de Vusse by a Simple Probability Method, Chemical Engineering

Science, 22, 517-523(1967).

76. B.A., Buffham, L.G.Gibilaro and H.W.KrophoUer, Network Combing of

Complex Flow-Mixing Models, Chemical Engineering Science, 24,7-10

(1969).

77. W.Pippel and G.Philipp, Utilization of Markov Chains for Simulating of

Dynamics of Chemical Systems, Chemical Engineering Science, 32, 543-

549(1977).

7 8. D.M.Himmelblau and K.B .Bischoff, Process Analysis and Simulation:

Deterministic systems, John Wiley and Sons, Inc., New York.London.

Sydney, 1968.

79. J.CMerchuk and R.Yunger, The Role of the Gas-Liquid Separator of Airlift

Reactors in the Mixing Process, Chemical Engineering Science, 45, 2973-

2975(1990).

80. H.S.Fogler, Elements of Chemical Reaction Engineering, Prentice-Hall Inc.,

1992.

81. A.Cholette and L.Cloutier, Mixing Efficiency Determinations for Continuous

Flow Systems, The Canadian Journal of Chemical Engineering, June, 105-

112(1959).

82. T.Bar and A.Tamir, An Impinging-stream Reactor with Two Pairs of Air

Feed, The Canadian Journal of Chemical Engineering, 68(1990)541-552.

83. S.I.Sandler, Chemical and Engineering Thermodynamics, John Wiley and

Sons Inc., 1977.

84. I.N.Kovalenko, N.Yu.Kuznetsov and V.M.Shurenkov, Models of Random

Processes, CRC Press, Inc., Boca Raton New York London Tokyo, 1996.

85. H.Petigen, H.Jurgens and D.Saupe, Chaos and Fractals-New Frontiers of

Science, Springer-Verlag New York, Inc., 1992.

86. T.Z.Fahidy, Modeling of Tank Electrolysers via Markov Chains, Journal of

Applied Electrochemistry, 17(1987)841-848