Ingenieur- Mathematik Constantin Carathéodory, Bellman‘s Equation and the Maximum Principle Hans Josef Pesch University of Bayreuth, Germany 50 Years of

Ingenieur-Mathematik

Constantin Carathéodory,

Bellman‘s Equation and the Maximum Principle

Hans Josef Pesch

University of Bayreuth, Germany

50 Years of Optimal Control, Bedlewo, Poland, Sept. 15-19, 2008


Outline

• Carathéodory‘s Royal Road of the Calculus of Variations,

• „his“ Bellman‘s Equation,

• and his Precursor of the Maximum Principle

Hans Josef Pesch, Roland Bulirsch: The Maximum Principle, Bellman‘s Equation, and Carathéodory‘s WorkJ. of Optimization Theory and Applications, Vol. 80, No. 2, Feb. 1994


Carathéodry‘s Royal Way in the Calculus of Variations

Relationship between

and

allows the reduction of

to

Hilbert‘s Independence Theorem

Hamilton-Jacobi Equation

Problems of the Calculus of Variations

Problems of Finite Optimization


Simple variational problem in the small

variation


Special variation

Variation of the integral


Formulation of an equaivalent variational problem (1)

Let

then

independent of


Formulation of an equaivalent variational problem (2)

Let

Then: integration along two curves yields

Thus

and therefore any line element where

can be passed by one and only one extremal curve

and

will be needed for the Legendre-Clebsch condition


Existence of extremals for a special variational problem

If there exists

with

for all and all with

then there holds: The solutions of are extremals of


Theorem: sufficient condition (Crathéodory, 1931)

If there exists

for which there hold

and

for sufficiently small , then the solutions of

yield


Crathéodory‘s Fundamental Equations (1931)

Hence we have to determine the functions

such that

(as function of ) possesses a minimum for

with value

(Carathéodory, 1935)

That is the so-called Bellman Equation

or

Thus

No imbedding or extremal fields on Carathéodory‘s Royal Road


Carathéodory‘s formulation of Weierstraß‘ Excess Function

Substituting the fundamental equations and replacing by yields

Hence we obtain the necessary condition of Weierstraß


Carathéodory‘s precursor of the Maximum Principle (1926)

Introducing canonical variables

and solving this equation for yields

Defining the Hamiltonian

yields


Lagrangian variational problems

Side conditions (Lagrangian problems)

Similarly

Introducing

the fundamental equations takes the form

(Carathéodory: 1926)

with

Defining

the Weierstraß necessary condition takes the form


Carathéodory‘s precursor of the Maximum Principle (1926)

From

there is only a little step via

to

with

s.t.


The Maximum Principle (precursor, 1926)

Constantin Carathéodory (Κωνσταντίνος Καραθεοδωρή)* Sept. 13, 1873 in Berlin; † Feb. 2, 1950, Munich

I will be glad if I have succeeded in impressing the idea that it is not only pleasant and entertainingto read at times the works of the old mathematicialauthors, but that this may occasionally be of usefor the actual advancement of science.

Besides this there is a great lesson we can derive from the facts which I have just referred to. We haveseen that even under conditions which seem mostfavorable very important results can be discardedfor a long time and whirled away from the main streamwhich is carrying the vessel science. …

If their ideas are too far in advance of their time, andif the general public is not prepared to accept them, these ideas may sleep for centuries on the shelvesof our libraries … awaiting the arrival of the prince charming who will take them home. (C.C. 1937)


• Born in Berlin to Greek parents, grew up in Brussels (father was the Ottoman ambassador) to Belgium • The Carathéodory family was well-respected in Constantinople (many important governmental positions)

• Formal schooling at a private school in Vanderstock (1881-83); travelling with is father to Berlin, Italian Riviera; grammar school in Brussels (1985); high school Athénée Royal d'Ixelles, graduation in 1891 • Twice winning of a prize as the best mathematics student in Belgium• Trelingual (Greek, French, German), later: English, Italian, Turkish, and the ancient languages

• École Militaire de Belgique (1891-95), École d'Application (1893-1896): military engineer

• War between Turkey and Greece (break out 1897); British colonial service: construction of the Assiut dam (until 1900); Studied mathematics: Jordan's Cours d'Analyse a.o.; Measurements of Cheops pyramid (published in 1901)

Constantin Carathéodory (1873 - 1950)


Constantin Carathéodory (1873 - 1950)

• Graduate studies at the University of Göttingen (1902-04) (supervision of Hermann Minkowski: dissertation in 1904 (Oct.) on Diskontinuierliche Lösungen der Variationsrechnung• In March 1905: venia legendi (Felix Klein)

• Various lecturing positions in Hannover, Breslau,Göttingen and Berlin (1909-20)• Prussian Academy of Sciences (1919, together with Albert Einstein)

• Plan for the creation of a new University in Greece (Ionian University) (1919, not realized due to the War in Asia Minor in 1922); the present day University of the Aegean claims to be the continuation• University of Smyrna (Izmir), invited by the Greek Prime Minister (1920); (major part in establishing the institution, ends in 1922 due to war• University on Athens (until 1924)• University of Munich (1924-38/50); Bavarian Academy of Sciences (1925)


Magnus Rudolph Hestenes (1906 – May 31, 1991)

Thus, has a maximum value

with respect to along

a minimizing curve .

Research Memorandum RM-100,

Rand Corporation, 1950

I became interested in control theory in 1948.

At that time I formulated the general control

problem of Bolza …, and observed the maximum

principle … is equivalent to the conditions of

Euler-Lagrange and Weierstrass …

It turns out that I had formulated what is now

known as the general optimal control problem.

The Maximum Principle (first formulation, 1950)


Richard Ernest Bellman (Aug. 26, 1920 – March 19, 1984)

Rufus Philip Isaacs (1914 – 1981)

The Maximum Principle (Bellman‘s & Isaacs‘ Equation, 1951+)


Isaacs in 1973 about his Tenet of Transition of 1951

Once I felt that here was the heart of the subject …..

Later I felt that it … was a mere truism.

Thus in (my book) Differential Games

it is mentioned only by title. This I regret.

I had no idea, that Pontryagin‘s principle

and Bellman‘s maximal principle

(a special case of the tenet, appearing a little later

in the Rand seminars) would enjoy such

a widespread citation.


Lev Semenovich Pontryagin (Лев Семёнович Понтрягин) (Sept. 3, 1908 – May 3. 1988)

The Maximum Principle (1956)

This fact is a special case

of the following general principle

which we call maximum principle

Doklady Akademii Nauk SSSR,

Vol. 10, 1956


The Maximum Principle (1956)

Vladimir G. Boltyanski Revaz V. Gamkrelidze


Boltyanski in 1991 about the Maximum Principle of 1956

By the way, the first statement of the maximum principle was given

by Gamkrelidze, who has established (generalizing the famous

Legendre Theorem) a sufficient condition for a sort of weak

optimality problem. Then, Pontryagin proposed to name

Gamkrelidze‘s condition Maximum Principle. … Finally, I understood

that the maximum principle is not a sufficient, but only a necessary

condition of optimality.

Pontryagin was the Chairman of our department at the Steklov

Mathematical Institute, and he could insist on his interests.

So, I had to use the title Pontryagin‘s Maximum Principle

in my paper. This is why all investigators in region of mathematics

and engineering know the main optimization criterium as the

Pontryagin‘s Maximum Principle.


Thank you for your attention!

The referenced paper can be downloaded fromwww.uni-bayreuth.de/departments/ingenieurmathematik

Email: [email protected]


Documents

Ingenieur- Mathematik Constantin Carathéodory, Bellman‘s Equation and the Maximum Principle Hans Josef Pesch University of Bayreuth, Germany 50 Years of