3
BOOK REVIEWS 547 singled out as resulting from a steepening wave, but this seems to go beyond our present resources. In Part II it is demonstrated that the subtleties of Part I are by no means purely the playthings of the mathematician. The analysis of one-dimensional wave propagation in a magnetofluid taxes.the methods of Part I to their limit. At that, one is left with the feeling in some cases that the final word has yet to be said (see Section 7-1 on the application of evolutionarity to the piston prob- lem). The usual scattering of typographical errors and petty slips are present. I found the following particularly jarring. On page 27 there is the assertion for Lipschitz continuous function that the modulus of the derivative is bounded (mathematical conscience murmurs, "If the derivative exists"), and immediately thereafter on page 28 one finds, "A trivial example of a Lipschitz continuous function is given by the solitary wave (not defined in the text) illustrated in Fig. 1.6." Unfortunately, the wave form appears to the untutored eye to have vertical tangents. On pages 96-100 the arcs rl and P. of Figure 2.9 are repeatedly referred to as "space-like" although for sure they are time-like. Irritation at isolations is a small price to pay for such a generally worthwhile work. ALBERT A. BLANK New York University Dynanic Programming and the Calculus of Variations. By STUAT E. DREYFUS. Academic Press, Inc., New York, 1965. xix + 248 pp. Before going into the review of this book I must give two possible interpreta- tions for the term "Dynamic Programming." (1) Dynamic Programming describes the subject matter of the mathematical theory of multi-stage decision processes. This first interpretation is due to Richard Belhnan who coined the term. (2) Dynamic Programming is a particular approach to (1) which is based on three main ingredients: the Principle of Optimality, the Optimal Value Function and the (Bellman) Functional Equation. With this interpretation Dynamic Programming can be considered as a systematic presentation, coordination, and generalization of the ideas of Mass and Carathodory. In the book under review and in this review of the book the term Dynamic Programming should be understood according to the second interpretation. The book has seven chapters. Chapter I (Discrete Dynamic Programming) and Chapter VII (Stochastic and Adaptive Optimization Problems) are very well done but outside the central purpose of the book. This review will be devoted entirely to Chapters II through VI. In Chapter II (The Classical Variation Theory) the author gives a very clear survey of the results of the classical calculus of variations. In Chapter III (The Simplest Problem) the aim of the author is to derive the results of the classical calculus of variations with the help of Dynamic Pro- gramming. This derivation, similar to the derivation given by Carathodory in Downloaded 12/04/14 to 129.120.242.61. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Dynamic Programming and the Calculus of Variations (Stuart E. Dreyfus)

  • Upload
    huber

  • View
    213

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Dynamic Programming and the Calculus of Variations (Stuart E. Dreyfus)

BOOK REVIEWS 547

singled out as resulting from a steepening wave, but this seems to go beyond ourpresent resources.In Part II it is demonstrated that the subtleties of Part I are by no means

purely the playthings of the mathematician. The analysis of one-dimensionalwave propagation in a magnetofluid taxes.the methods of Part I to their limit.At that, one is left with the feeling in some cases that the final word has yet tobe said (see Section 7-1 on the application of evolutionarity to the piston prob-lem).The usual scattering of typographical errors and petty slips are present. I

found the following particularly jarring. On page 27 there is the assertion forLipschitz continuous function that the modulus of the derivative is bounded(mathematical conscience murmurs, "If the derivative exists"), and immediatelythereafter on page 28 one finds, "A trivial example of a Lipschitz continuousfunction is given by the solitary wave (not defined in the text) illustrated in Fig.1.6." Unfortunately, the wave form appears to the untutored eye to have verticaltangents. On pages 96-100 the arcs rl and P. of Figure 2.9 are repeatedly referredto as "space-like" although for sure they are time-like. Irritation at isolationsis a small price to pay for such a generally worthwhile work.

ALBERT A. BLANKNew York University

Dynanic Programming and the Calculus of Variations. By STUAT E. DREYFUS.Academic Press, Inc., New York, 1965. xix + 248 pp.Before going into the review of this book I must give two possible interpreta-

tions for the term "Dynamic Programming."(1) Dynamic Programming describes the subject matter of the mathematical

theory of multi-stage decision processes. This first interpretation is due toRichard Belhnan who coined the term.

(2) Dynamic Programming is a particular approach to (1) which is based onthree main ingredients: the Principle of Optimality, the Optimal ValueFunction and the (Bellman) Functional Equation. With this interpretationDynamic Programming can be considered as a systematic presentation,coordination, and generalization of the ideas of Mass and Carathodory.

In the book under review and in this review of the book the term DynamicProgramming should be understood according to the second interpretation.The book has seven chapters. Chapter I (Discrete Dynamic Programming)

and Chapter VII (Stochastic and Adaptive Optimization Problems) are verywell done but outside the central purpose of the book. This review will be devotedentirely to Chapters II through VI.

In Chapter II (The Classical Variation Theory) the author gives a very clearsurvey of the results of the classical calculus of variations.

In Chapter III (The Simplest Problem) the aim of the author is to derive theresults of the classical calculus of variations with the help of Dynamic Pro-gramming. This derivation, similar to the derivation given by Carathodory in

Dow

nloa

ded

12/0

4/14

to 1

29.1

20.2

42.6

1. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 2: Dynamic Programming and the Calculus of Variations (Stuart E. Dreyfus)

54S BOOK REVIEWS

the fifth chapter of the book Die Differential-und Integralgleichungen der Mechanikund Physitc, edited by Frank and von Mises, Braunschweig, 1925, proceeds fromthe following assumption: the Hamilton-Jacobi partial differential equationassociated with the variational problem has a solution, or equivalently, usingDynamic Programming terminology, the optimal value function exists and issmooth enough. This derivation illuminates the geometric aspects of the calculusof variations and is a useful mnemonic device to check the correctness of finalformulas but cannot be considered as an alternative approach to the calculus ofvariations. On this point the reviewer respects the author’s opinion but does notshare his optimism when he states in the introduction: "Indeed, a completelyrigorous presentation would necessarily be at odds with the primary goal ofintuitive clarity. The question is left open as to whether the dynamic program-ruing approach to problems of the calculus of variations, if pursued for severalhundred years (as has been the classical theory) and if rendered completelyrigorous, might yield results that are either slightly sharper or slightly weakerthan those of the classical theory. At this moment of its infancy, it is fairly safeto say that the dynamic programming results, at a rather subtle level, areweaker."

Chapter IV (The Problem of Mayer) is a technical extension of Chapter III.Chapter V (Inequality Constraints) considers the specific type of problems

which have been at the origin of the theory of optimal control. In this chapterthe author applies again the formal methods of dynamic programming and statesPontryagin’s Maximum Principle. Whereas, in Chapter III the Optimal ValueFunction is in general smooth enough to justify the application of these formalmethods (a proof of that fact is found in the theory of the Hamilton-Jacobi partialdifferential equation), this is not the case for the problems treated in this chapter.On page 171 the author states: "The above result was derived by Valentine inthe 1930’s by modifying classical arguments and recently was rederived by theeminent Russian mathematician, L. S. Pontryagin, who termed it ’the MaximumPrinciple’." Such a remark, unfortunately common, is very misleading. L. D.Berkovitz has shown that some fairly general forms of Pontryagin’s MaximumPrinciple can be obtained from the following three elements: (1) the classicalcalculus of variations (with the treatise of Bliss as a standard); (2) Valentine’sdissertation; (3) the multiplier rule for the abnormal problem of Mayer, a resultdue to McShane. The fundamental paper of L. D. Berkovitz is often misunder-stood as based on elements (1) and (2) only. This is unfortunate since of thesethree elements, McShane’s result is the only one which is strictly necessary, asPontryagin and his associates have shown in their proof of the Maximum Prin-ciple, which is based on a very useful device introduced by McShane: the con-sideration of some convex variations and separation properties of some convexsets. The Maximum Principle is the most important necessary condition in thetheory of optimal control" it characterizes the separation of some convex sets.There is no hope to ever derive this result with any method which does not refer,directly or indirectly, to the concept of convex sets and their separation.

Chapter VI (Problems with Special Linear Structures) presents a real challenge

Dow

nloa

ded

12/0

4/14

to 1

29.1

20.2

42.6

1. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 3: Dynamic Programming and the Calculus of Variations (Stuart E. Dreyfus)

BOOK REVIEWS 549

to the devoted dynamic programmer, page 190: "The troublesome types ofproblems that we shall investigate have what seem to be particularly simpleforms and appear frequently in the literature. While it is certainly advisable tostudy simplified problems before attacking the complicated situations found innatures, the properties of the simplified problems that are treated in this chapterturn out to be atypical of the properties of most practical problems. For thisreason, we feel that these special types of problems have been receiving undueemphasis and that a short chapter near the end of this book is their appropriatehabitat." The only trouble with these simple problems is that they put in fullview those shortcomings of Dynamic Programming which are so easily hidden,but still present, in more complex problems. The interested reader should compareChapter VI with the fundamental papers:R. E. BELLMAN, I. GLICKSBERG, AND O. A. GROSS, On the "bang bang" control

problem, Quart. Appl. Math., 14(1956), pp. 11-18.J. P. LA SALLE, Time optimal control systems, Proc. Nat. Acad. Sci. U.S.A.,

45(1959), pp. 573-577.The book under review is well organized and clearly written. The topics of

Chapters I and VII are the ideal ground to apply the methods of dynamic pro-gramming. I hope that the author will expand these two chapters into a full bookin the near future.

HUBER HALKINUniversity of California at La Jolla

Computer Software. Programming Systems for Digital Computers. By IVAN FLORES.Prentice-Hall, Englewood Cliffs, New Jersey. x + 493 pp. $16.00.In the preface this book bills itself as "an advanced programming book", one

which "will broaden the horizons of the systems programmer who is knowledge-able about specific systems by presenting general principles applicable to allprogramming systems". The book itself justifies neither claim; most advancedtopics in computer software are not mentioned, and there is little or no attemptto generalize upon any principles set forth. Important topics such as dynamicrelocation, multiprogramming, real time processing, and problem or procedureoriented languages are not discussed despite the book’s claim that one of itspurposes is to "observe design principles employed in current and future sys-tems". What the author does present is a system containing a restricted FAP-type macro assembler, an IOCS with an interrupt supervisor, a library (which isnot described), a loader, and a job supervisor. Only the assembler and the IOCSare described in any detail. The loader and iob supervisor receive a superficialhandling with quite general descriptions. For example, the problem of relocationduring loading is essentially ignored, and there is an overemphasis on the dis-tinction between programs, subprograms, and subroutines (a distinction whichshould be minimized). There is also some confusion between subroutines assem-bled with a program and those loaded from the system library. Discussion ofassembler and loader functions sometimes gets fused so that it is unclear whichis doing what.

Dow

nloa

ded

12/0

4/14

to 1

29.1

20.2

42.6

1. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p