Optimal Control of Stochastic Service SystemsAuthor(s): Shaler Stidham, Jr.Source: Advances in Applied Probability, Vol. 10, No. 2 (Jun., 1978), pp. 277-278Published by: Applied Probability TrustStable URL: http://www.jstor.org/stable/1426882 .

Accessed: 14/06/2014 04:09

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances inApplied Probability.

http://www.jstor.org

This content downloaded from 195.78.108.185 on Sat, 14 Jun 2014 04:09:23 AMAll use subject to JSTOR Terms and Conditions

Twente University of Technology, Enschede, The Netherlands, 15-19 August 1977 277

f u-1/2 dF(u) = 00, we have

(17) Bn, Vn 4

(2-mro2)-1/2X, with

n 1(n)f1

(18) B.= I (k+l1)-1/2 dF(u)uk(1- u)"-, k=O kk

(1 (19) B. --~n-1/2 u-3/2 dF(u).

References

[1] BILLINGSLEY, P. (1968) Convergence of Probability Measures. Wiley, New York. [2] BREIMAN, L. (1968) Probability. Addison-Wesley, Reading, Mass. [3] ORNSTEIN, D. (1966) A limit theorem for independent random variables. Proc. 5th Berkeley

Symp. Math. Statist. Prob. 2(2), 213-216. [4] SHEPP, L. A. (1964) A local limit theorem. Ann. Math. Statist. 35, 419-423. [5] STAM, A. J. Limit theorems with infinite limiting measures for certain sums of exchangeable

random variables. Stoch. Proc. Appl. (submitted). [6] STONE, C. (1966) On local and ratio limit theorems. Proc. 5th Berkeley Symp. Math. Statist.

Prob. 2(2), 217-224. [7] STONE, C. (1966) Ratio limit theorems for random walks on groups. Trans. Amer. Math.

Soc. 125, 86-100.

Optimal control of stochastic service systems

SHALER STIDHAM, JR., Technical University of Denmark, Lyngby

Traditional queueing theory has focussed almost entirely on descriptive models of service systems, in which the transient and limiting distributions are derived for the associated stochastic processes, such as queue length and waiting time. Recently, however, there has been increasing interest in models for the optimal design and control of stochastic service systems. It is the aim of this paper to review some of the work that has been done in this area, to try to present a unified theory and indicate common problems and common methodologies for solving them, and to suggest the directions in which future research might proceed.

Just as descriptive queueing theory is most often concerned with Markov processes, so the theory of control of stochastic service systems usually involves Markov decision processes, and the related solution techniques of dynamic

This content downloaded from 195.78.108.185 on Sat, 14 Jun 2014 04:09:23 AMAll use subject to JSTOR Terms and Conditions

278 7TH CONFERENCE ON STOCHASTIC PROCESSES AND APPLICATIONS

programming. Queueing systems have special structure, which leads both to special problems (such as unbounded costs) requiring refinements of the standard dynamic-programming theory, and to special advantages, such as the optimality of structured (e.g., monotonic) policies. Both these aspects are discussed at length in a rather general context.

Records and random record processes

MARK WESTCOTT, CSIRO Division of Mathematics and Statistics, Canberra

Let Xo1, X1,"" be a sequence of independent and identically distributed

random variables with continuous distribution function G. The (upper) records in this sequence are the successive maxima. We define the sequence of record times Vr inductively by

Vo= 0, Vr, = min (i:Xi > Xv,_,) (r= 1, 2, .

),

and the inter-record times by A~ = V,- Vr,_(r = 1, 2, - ). Also, let v, be the

number of records in (X1, X, ... , X)(n = 1, 2, * ). Because of the continuity of G, none of {V,}, {A,}, {v,} depend on G, unlike

the record heights Xv,; for this reason we do not discuss the latter in any detail. Many properties of these variables have been discovered over the years by a variety of authors; surveys may be found in Resnick (1973) and Westcott (1977b). A neat unified approach to the problem is made possible by the following result of Williams (1973); if W1, W2," ?

are i.i.d. unit exponential variates then the sequence

(1) Vo=0, Vr,= [(Vr,_l + 1)ew] (r= 1, 2, ... ),

[ ] denoting integer part, is distributed identically with the record time se- quence. From this explicit construction most known results, and several new ones, follow readily, as shown in Williams (1973), Westcott (1977a, b). Heuris- tically, they are all consequences of the fact, clearly implicit in (1), that {log V,} is approximately a Poisson process of unit rate (as noted long ago by Armitage).

The principal object of this paper is to investigate the closely related random record process recently introduced by Gaver (1976). Suppose P is a point process on [0, oo) with a point at 0, independent of the X,, and associate Xj with the jth point of P (j= 0, 1,

. . - ). The point process P* of the points of P

associated with records in X, is the random record process. Gaver gives examples of how this process might occur in practice.

This content downloaded from 195.78.108.185 on Sat, 14 Jun 2014 04:09:23 AMAll use subject to JSTOR Terms and Conditions