The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed Opportunities

The Maximum Principle of Optimal Control:A History of Ingenious Idea and Missed Opportunities

Hans Josef Pesch 1, Michael Plail 2

1 University of Bayreuth, Germany2 Steinebach, Wörthsee, Germany

Optimization Day, University of Southern Australia,Adelaide, Australia,

January 29, 2011

Outline

• Carathéodory‘s Royal Road of the Calculus of Variations

• Hestenes‘ secret report and first formulation

• Bellman‘s and Isaacs‘ regrets

Hans Josef Pesch, Roland Bulirsch: The Maximum Principle, Bellman‘s Equation, and Carathéodory‘s WorkJ. of Optimization Theory and Applications, Vol. 80, No. 2, Feb. 1994

Hans Josef Pesch, Michael Plail:The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed OpportunitiesControl and Cybernetics, Vol. 38, No. 4A, 973-995, 2009.

• Pontryagin and his students: adoration and embitterment

Missed Opportunities to the Maximum Principle of Optimal Control

Carathéodory‘s Royal Road in the Calculus of Variations

Relationship between

and

allows the reduction of

to

Hilbert‘s Independence Theorem

Hamilton-Jacobi Equations

Problems of the Calculus of Variations

Problems of Finite Optimization


subject to implicit differential equations

Search for -curves that extremize

Lagrangian problems: precursors of optimal control

for line elements of curves

with

DOF: n - pcontrols


Stage 1: Definition: extremal (minimal or maximal)

Different from today‘s terminology: weak extremum / minimum / maximum

Stage 2: Proof of necessary Legendre-Clebsch condition

or

in today‘s terminology for minimization

has a positive definit Hessian

for fixed

closer neighborhood


Stage 3: Caratheodory‘s equivalent variational problems

Let

then

independent of

Let

Then: integration along two curves yields

Thus

andand therefore any line element where

can be passed by one and only one minimal curve


adding a null Lagrangian


Stage 4: Caratheodory‘s existence result for a special problem

with

for all and all with

then the solutions of are extremals of

If there exists


Stage 5: Caratheodory‘s sufficient conditionIf there exists

for which there hold

and

for sufficiently small , then the solutions of

yield

Hence we have to determine the functions

such that

(as function of ) possesses a minimum for

with value

(Carathéodory, 1935)

That is the so-called Bellman EquationNo imbedding or extremal fields on Carathéodory‘s Royal Road

or

C‘s fundamental equations:


Substituting the fundamental equations and replacing by yields

Hence we obtain the necessary condition of Weierstraß


Stage 6: Caratheodory‘s formulation of Weierstraß‘ condition

Similarly

Introducing the Lagrange function

the fundamental equations take the form

(Carathéodory: 1926)


Stage 7: Lagrangian variational problems

Exit to the Maximum Principle?

Introducing canonical variables

and solving these equation for yieldswith

Defining the Hamiltonian in canonical coordinates

the Weierstraß necessary condition takes the form

and

Recall Caratheodory‘s Hamiltonian

Carathéodory‘s closed approach to optimal control (from 1935)

Today‘s Hamiltonian

degree of freedom: control

degree of freedom: control?

Exit to the Maximum Principle from C‘s Royal Road

call them controls

canonical equations


With the maximizing Hamiltonian for

and the costate

we obtain as long as

By means of the Euler-Lagrange equation

and because of

Furthermore

Hence, must have a maximum with respect to along a curve

From here it is still a big step to

Missed Carathéodory the exit?


1904

1932

Constantin Carathéodory (1873 - 1950)

• Born in Berlin to Greek parents, grew up in Brussels (father was the Ottoman ambassador) to Belgium • The Carathéodory family was well-respected in Constantinople (many important governmental positions)

• Formal schooling at a private school in Vanderstock (1881-83); travelling with is father to Berlin, Italian Riviera; grammar school in Brussels (1985); high school Athénée Royal d'Ixelles, graduation in 1891 • Twice winning of a prize as the best mathematics student in Belgium• Trelingual (Greek, French, German), later: English, Italian, Turkish, and the ancient languages

• École Militaire de Belgique (1891-95), École d'Application (1893-1896): military engineer

• War between Turkey and Greece (break out 1897); British colonial service: construction of the Assiut dam (until 1900); Studied mathematics: Jordan's Cours d'Analyse a.o.; Measurements of Cheops pyramid (published in 1901)



• Graduate studies at the University of Göttingen (1902-04) (supervision of Hermann Minkowski: dissertation in 1904 (Oct.) on Diskontinuierliche Lösungen der Variationsrechnung• In March 1905: venia legendi (Felix Klein)

• Various lecturing positions in Hannover, Breslau, Göttingen and Berlin (1909-20)• Prussian Academy of Sciences (1919, together with Albert Einstein)

• Plan for the creation of a new University in Greece (Ionian University) (1919, not realized due to the War in Asia Minor in 1922); the present day University of the Aegean claims to be the continuation• University of Smyrna (Izmir), invited by the Greek Prime Minister (1920); (major part in establishing the institution, ends in 1922 due to war• University on Athens (until 1924)• University of Munich (1924-38/50); Bavarian Academy of Sciences (1925)

• C. played a remarkable opposing role together with the Munich „Dreigestirn“ (triumvirate) (Perron, Tietze) within the Bavarian Academy of Science during the Nazi terror in Germany

Magnus Rudolph Hestenes (1906 – May 31, 1991)

Thus, has a maximum value with respect to along a minimizing curve .

Research Memorandum RM-100, Rand Corporation, 1950

I became interested in control theory in 1948.At that time I formulated the general controlproblem of Bolza …, and observed the maximum principle … is equivalent to the conditions of Euler-Lagrange and Weierstrass …

It turns out that I had formulated what is now known as the general optimal control problem.

The Maximum Principle (first formulation, controls, 1950)

Missed opportunity

Richard Ernest Bellman (Aug. 26, 1920 – March 19, 1984)

Rufus Philip Isaacs (1914 – 1981)

The Maximum Principle (Bellman‘s & Isaacs‘ Equation, 1951+)

Isaacs in 1973 about his Tenet of Transition of 1951

Once I felt that here was the heart of the subject ….. Later I felt that it … was a mere truism. Thus in (my book) Differential Games it is mentioned only by title. This I regret. I had no idea, that Pontryagin‘s principle and Bellman‘s maximal principle (a special case of the tenet, appearing a little later in the Rand seminars) would enjoy such a widespread citation.

Missed opportunities

Lev Semenovich Pontryagin (Лев Семёнович Понтрягин) (Sept. 3, 1908 – May 3. 1988)

The Maximum Principle (1956)

This fact is a special case of the following general principlewhich we call maximum principle

Doklady Akademii Nauk SSSR, Vol. 10, 1956

The Maximum Principle (1956)

Vladimir G. Boltyanski Revaz V. Gamkrelidze

proved the Maximum Principle

Boltyanski in 1991 about the Maximum Principle of 1956

By the way, the first statement of the maximum principle was given by Gamkrelidze, who has established (generalizing the famous Legendre Theorem) a sufficient condition for a sort of weak optimality problem. Then, Pontryagin proposed to name Gamkrelidze‘s condition Maximum Principle. … Finally, I understood that the maximum principle is not a sufficient, but only a necessarycondition of optimality.

Pontryagin was the Chairman of our department at the SteklovMathematical Institute, and he could insist on his interests.So, I had to use the title Pontryagin‘s Maximum Principle in my paper. This is why all investigators in region of mathematicsand engineering know the main optimization criterium as thePontryagin‘s Maximum Principle.

Gamkrelidze in 2008 about Pontryagin

My life was a series of missed opportunities, but one opportunity, I have not missed, to have met Pontryagin.*

* at the Banach Center Conference on 50 Years of Optimal Control in Bedlewo, Poland, on September 15, 2008

Plail, M.: Die Entwicklung der optimalen Steuerungen. Vandenhoeck & Ruprecht, Göttingen, Germany, 1998

Carathéodory‘s words:

Constantin Carathéodory (Κωνσταντίνος Καραθεοδωρή)* Sept. 13, 1873 in Berlin; † Feb. 2, 1950, Munich

I will be glad if I have succeeded in impressing the idea that it is not only pleasant and entertainingto read at times the works of the old mathematicialauthors, but that this may occasionally be of usefor the actual advancement of science.

Besides this there is a great lesson we can derive from the facts which I have just referred to. We haveseen that even under conditions which seem mostfavorable very important results can be discardedfor a long time and whirled away from the main streamwhich is carrying the vessel science. …

If their ideas are too far in advance of their time, andif the general public is not prepared to accept them, these ideas may sleep for centuries on the shelvesof our libraries … awaiting the arrival of the prince charming who will take them home. (C.C. 1937)

Thank you for your attention!

Both papers and a third forthcoming onecan be downloaded from

www.ingmath.uni-bayreuth.de/

Email: [email protected]

http://www.ingmath.uni-bayreuth.de/

Documents

The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed Opportunities