17
“Tony” Runzhe Yang Feb. 14, 2020 https://runzhe-yang.science On the Global Convergence of Gradient-Based Learning in Continuous Zero-Sum Games

On the Global Convergence of Gradient-Based Learning in

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: On the Global Convergence of Gradient-Based Learning in

“Tony” Runzhe Yang

Feb. 14, 2020

https://runzhe-yang.science

On the Global Convergence of Gradient-Based Learning in Continuous Zero-Sum Games

Page 2: On the Global Convergence of Gradient-Based Learning in

Minimax Problems in Machine Learning

Generative Adversarial Networks (GANs) [Goodfellow et al., 2014]

GANs Illustration: https://medium.freecodecamp.org/an-intuitive-introduction-to-generative-adversarial-networks-gans-7a2264a81394

minG

maxD

V (D,G) = Ex⇠pdata(x)[logD(x)] + Ez⇠pz(z)[log(1�D(G(z)))]

<latexit sha1_base64="74lJQW1s2utWiTzyEl4HfJbZGFg=">AAAD4XicfZNbb9MwFMfdhssotw4eecCimpRAVyVoXF6QJuhUXhBFWrtJTRQ5rptZy022W6Wz/AF4Q/DIJ+Oj8Iadhks3hSPl6O9zfsdHtk+iIqFcuO6PVtu6dv3GzZ1bndt37t673919MOX5kmEywXmSs9MIcZLQjEwEFQk5LRhBaZSQk+j8ncmfrAjjNM+OxbogQYrijC4oRkKHwu43P6VZKEfKT1EZyqGCU3vYhyMHvoE6JM6iCB6F0o9SWEKf0xQWeiVIKeQcCaSUXTpq5id5DId2RTkBfHa59OJPpVmoCrz4XWd7+0N7VMccJ+iE3Z47cCuDV4VXix6obRzutl/68xwvU5IJnCDOZ55biEAiJihOiOr4S04KhM9RTGZaZiglPJDV7Sm4pyNzuMiZ/jIBq+i/FRKl3BynD7UwCK8UX6dRH0ZptciLTG9kqK3KctNiu79YvA4kzYqlIBnetF8sEyhyaB4IzikjWCRr2PGHRJ+GkQ96248FYUjk7Kn0EYv1Wyl9uhj6fWj0/1Ca/UW1bkJXiClpXBOA85WSxjUBPNadjGsCCNMtjGsChM4fN6cLna4GSwg5Zqq5T1mQmowieaTBzp6+VoQZ1fMA8RliCAv9U5hJ8y7P1VUxfT7wDgYvPh30Dt/WM7cDHoEnwAYeeAUOwXswBhOAwc/W45bdcixsfba+WF83aLtV1zwEW2Z9/wUCyEVE</latexit>

Page 3: On the Global Convergence of Gradient-Based Learning in

Minimax Problems in Machine Learning

Maximum Entropy Inverse Reinforcement Learning (MaxEnt IRL) [Ziebart et al., 2008]&[Finn et

al., 2016 - slightly different here]

IRL Illustration: https://runzhe-yang.science/demo/Inverse_Reinforcement_Learning.pdf

min⇡

maxC

V (⇡, C) = E⌧⇠⇡[�C(⌧)] + log (E⌧⇠⇡[⇡(⌧)/ exp(�C(⌧))])

<latexit sha1_base64="yyDZppKyrzZ7EDyAYjkHjnvBvk8=">AAAD1nicfZNbixMxFMfTjpe13rrqmyjBstBqra2slxdhsS74IlbYdgvNUDJpOg2bmYQkU1rC+Cb66NfzS/gZTKZVty7jgTn8c87v5JDkTCQ506bb/VGpBpcuX7m6d612/cbNW7fr+3dGWmSK0CERXKhxhDXlLKVDwwynY6koTiJOT6Ozvs+fLqnSTKQnZi1pmOA4ZXNGsHGhaf0bSlg6tUiyHCV4NbX9HI6abtmG/RZ8A13QLKIIHjvG4AxplkAPT572mz7QCuGTPxDiIkaczk3zHF7QvmbDP0N0JZu/q1shUixemFZtWm90O93C4EXR24oG2Npgul99iWaCZAlNDeFY60mvK01osTKMcJrXUKapxOQMx3TiZIoTqkNbXFkOD1xkBudCuS81sIier7A40f5UbeiER3Sh9DqJ2jBKioWQqdvIUzuVq02L3f5m/jq0LJWZoSnZtJ9nHBoB/avAGVOUGL6GNfSOutMo+sFt+1FShY1Qjy3CKnbPk7vTxRC1odf/Q1n6F3W6DF1ilVvvygAilrn1rgzQsevkXRlAlWvhXRlgXP6kPC1dupgvY+xA5eV9VpJuySiyxw6sHbhrxUQxNw+QLLDCxLg/wU9a79+5uihGzzu9w86LT4eNo7fbmdsD98Ej0AQ98AocgfdgAIaAgJ+Ve5UHlYfBOPgcfAm+btBqZVtzF+xY8P0XBp9C9w==</latexit>

Experts

Policy networks Trials

Demonstrations

Cost Fn.

Good/Bad

Page 4: On the Global Convergence of Gradient-Based Learning in

Minimax Problems in Game Theory

Agent 1

minx

maxy

S(x,y)

<latexit sha1_base64="NsgQ8OVYc7bq9u0xnxC3odXMzq8=">AAADd3icfZJbixMxFMezHS/reOvqow8Gy0pXSpmR9fK4oIIIYsXt7sJOKZk0nQ07uZCkpSXMZ/DT+Kqfw4/imyfTAa3LeCCHf875nRxyOLkuuXVJ8nOnE127fuPm7q349p279+539x6cWLUwlI2pKpU5y4llJZds7Lgr2Zk2jIi8ZKf55ZuQP10yY7mSx26t2USQQvI5p8RBaNo9yASXU5/lAq+qTJDVRq8r/KVfBwe4vh/gabeXDJPa8FWRNqKHGhtN9zovs5miC8GkoyWx9jxNtJt4YhynJavibGGZJvSSFOwcpCSC2Ymv/1ThfYjM8FwZONLhOvp3hSfCCuIuBhhEQGyt7FrkA5yL+qK0hIcCtVW52rTY7u/mryeeS71wTNJN+/mixE7hMDY844ZRV65xnL1l8BvDPsKznzQzxCnzzGfEFDC9Cn5X4GyAg/4fyuUfFHQbuiSm8sG1AVQtKx9cG2AL6BRcG8AMtAiuDXCQP25Pa0hnYcbO+ZGp2vusNGvIPPfvAIz3YayEGg77gOkFMYQ6WNUYNi39d6+uipPnw/Rw+OLzYe/oQ7Nzu+gReoL6KEWv0BF6j0ZojCj6ir6h7+hH51f0OHoa9TdoZ6epeYi2LEp/A6dFIj8=</latexit>

x 2 Rn

<latexit sha1_base64="SsxVG42yrDyJVeXMFYi8CjifX64=">AAADY3icfZLtahQxFIbTHT/q1I+26i8RgktBZFlmpLb+LKgggrhKty101pJks9PQSTIkmWWXMBfjX70iL8D78GR2QNcyBubwzjnPyUnCS8tCWJckPzd60Y2bt25v3om37t67/2B7Z/fE6sowPma60OaMEssLofjYCVfws9JwImnBT+nVm1A/nXNjhVbHblnyiSS5EjPBiIPUxfbjjEq8wJlQOJPEXVKKv3yFfD8ZJs3C10Xaij5q1+hip3eQTTWrJFeOFcTa8zQp3cQT4wQreB1nleUlYVck5+cgFZHcTnxz/hrvQWaKZ9rApxxusn93eCJtONwAgwiIbZRdSjrAVDY/ulSwUaDWOherEevz3ez1xAtVVo4rtho/qwrsNA5PhKfCcOaKJY6ztxxuY/hH2PZTyQ1x2rzwGTG5JIsabpfjbICD/h8q1B8UdBc6J6b2IXQBTM9rH0IXYHOYFEIXwA2MCKELcFA/7i6XUG5s4pwfmbp7zqLkLUmpfwdgvAfPSpgR4AfMLokhzIEtY3Ba+q+vrouTl8N0f/jq837/6EPruU30BD1Dz1GKDtEReo9GaIwY8ugb+o5+9H5FW9Fu9GiF9jbanodobUVPfwOEMBqH</latexit>

y 2 Rm

<latexit sha1_base64="rA7DLmYL/WQ3JFyXAU9hSzcKJvw=">AAADY3icfZLtahQxFIbTHT/q1I9t1V8iBJeCyLLMSG39WVBBBHGVblvorEuSPTsNnXyQZJZdhrkY/+oVeQHeh8nsgq5lPDCHd3KekzcJh+qCW5ckP7c60Y2bt25v34l37t67/6C7u3dqVWkYjJgqlDmnxELBJYwcdwWcawNE0ALO6NWbUD+bg7FcyRO31DAWJJd8xhlxfmnSfZxRgZc44xJngrhLSvGXr2LS7SWDpAl8XaRr0UPrGE52O4fZVLFSgHSsINZepIl244oYx1kBdZyVFjRhVySHCy8lEWDHVXP+Gu/7lSmeKeM/6XCz+ndHRYQNh+tjLwJiG2WXgvYxFc2P0tJvFKiNzsXKYtPfzV6PKy516UCylf2sLLBTODwRnnIDzBVLHGdvwd/GwEe/7ScNhjhlXlQZMbkgi9rfLsdZHwf9P5TLP6jXbeicmLoKqQ1gal5XIbUBNvdOIbUBYLxFSG2A8/WT9rL25WZMnKuGpm73WWhYk5RW7zwY7/tnJcxwPw+YXRJDmPNjGftJS/+dq+vi9OUgPRi8+nzQO/6wnrlt9AQ9Q89Rio7QMXqPhmiEGKrQN/Qd/ej8inaivejRCu1srXseoo2Inv4GhEAahw==</latexit>

Agent 2

Action space:

Payoff function: �S(x,y)

<latexit sha1_base64="FxIIFphVESxosT7D/CU0SQ01q8k=">AAADXnicfZLfa9RAEMe3F601WnutL4Ivi0ehynkkpWofC1YQQTyx1xaa49jszaVLsz/Y3Rx3hPwnvtr/yTf/FHdzAT1LHMjkm5nP7GSHSVXOjI2inxud4N79zQdbD8NHj7ef7HR3986NLDSFEZW51JcpMZAzASPLbA6XSgPhaQ4X6c17n7+YgzZMijO7VDDmJBNsxiixLjTpdl9/O0hSjhd97F/Ll5NuLxpEteG7Im5EDzU2nOx23iZTSQsOwtKcGHMVR8qOS6ItozlUYVIYUITekAyunBSEgxmX9a9XeN9FpngmtXuExXX074qScMOJve5jJzxiamWWPO3jlNcfUgl3kKfWKherFuv97ex4XDKhCguCrtrPihxbif108JRpoDZf4jA5BXcbDZ/dsV8UaGKlflUmRGecLCp3uwwnfez1/1Am/qBOt6FzoqvSuzaAynlVetcGmMx18q4NAO1aeNcGWJc/a08rl078jK0th7pq77NQ0JBpWn5wYLjvxkqoZm4fML0mmlDrNjJ0mxb/u1d3xfnhID4avPl61Dv51OzcFnqOXqADFKN36AR9REM0QhTN0Xf0A912fgWbwXaws0I7G03NU7RmwbPfC5cXuw==</latexit>

S(x,y)

<latexit sha1_base64="9S7dmACgbsjDBLHBaN3n+h0j3IU=">AAADXXicfZLfa9RAEMe3F6s11vZaH3zwZfEoVDmOpFTtY0GFUhBP7LWF5jg2e3Pp0uwPdjfHHSF/ia/6R/nkv+JuLqBniQOZfDPzmZ3sMKnKmbFR9HOjEzzYfPho63H4ZPvpzm53b//SyEJTGFGZS32dEgM5EzCyzOZwrTQQnuZwld699/mrOWjDpLiwSwVjTjLBZowS60KT7u7XwyTleNHH/rV8Nen2okFUG74v4kb0UGPDyV7nbTKVtOAgLM2JMTdxpOy4JNoymkMVJoUBRegdyeDGSUE4mHFZ/3mFD1xkimdSu0dYXEf/rigJN5zY2z52wiOmVmbJ0z5Oef0hlXAHeWqtcrFqsd7fzk7GJROqsCDoqv2syLGV2A8HT5kGavMlDpMP4G6j4ZM79rMCTazUr8uE6IyTReVul+Gkj73+H8rEH9TpNnROdFV61wZQOa9K79oAk7lO3rUBoF0L79oA6/IX7Wnl0omfsbXlUFftfRYKGjJNy48ODA/cWAnVzO0DprdEE2rdQoZu0+J/9+q+uDwaxMeDN1+Oe6fnzc5toRfoJTpEMXqHTtEZGqIRoqhA39B39KPzK9gMtoOdFdrZaGqeoTULnv8GX2wXhA==</latexit>

sequential game - agent 1 moves first (slower)

Page 5: On the Global Convergence of Gradient-Based Learning in

Gradient Descent-Ascent Dynamics of GANs

Daskalakis et al., Training GANs with Optimism, ICLR 2018

minG

maxD

V (D,G) = Ex⇠pdata(x)[logD(x)] + Ez⇠pz(z)[log(1�D(G(z)))]

<latexit sha1_base64="74lJQW1s2utWiTzyEl4HfJbZGFg=">AAAD4XicfZNbb9MwFMfdhssotw4eecCimpRAVyVoXF6QJuhUXhBFWrtJTRQ5rptZy022W6Wz/AF4Q/DIJ+Oj8Iadhks3hSPl6O9zfsdHtk+iIqFcuO6PVtu6dv3GzZ1bndt37t673919MOX5kmEywXmSs9MIcZLQjEwEFQk5LRhBaZSQk+j8ncmfrAjjNM+OxbogQYrijC4oRkKHwu43P6VZKEfKT1EZyqGCU3vYhyMHvoE6JM6iCB6F0o9SWEKf0xQWeiVIKeQcCaSUXTpq5id5DId2RTkBfHa59OJPpVmoCrz4XWd7+0N7VMccJ+iE3Z47cCuDV4VXix6obRzutl/68xwvU5IJnCDOZ55biEAiJihOiOr4S04KhM9RTGZaZiglPJDV7Sm4pyNzuMiZ/jIBq+i/FRKl3BynD7UwCK8UX6dRH0ZptciLTG9kqK3KctNiu79YvA4kzYqlIBnetF8sEyhyaB4IzikjWCRr2PGHRJ+GkQ96248FYUjk7Kn0EYv1Wyl9uhj6fWj0/1Ca/UW1bkJXiClpXBOA85WSxjUBPNadjGsCCNMtjGsChM4fN6cLna4GSwg5Zqq5T1mQmowieaTBzp6+VoQZ1fMA8RliCAv9U5hJ8y7P1VUxfT7wDgYvPh30Dt/WM7cDHoEnwAYeeAUOwXswBhOAwc/W45bdcixsfba+WF83aLtV1zwEW2Z9/wUCyEVE</latexit>

✓G = ✓G � ⌘r✓GV (D,G)

<latexit sha1_base64="JvApDbPVabXOK1x7rljsEJlrqHg=">AAADhXicfVJtaxNBEN7m1NbzLa0f/bIYClXSkEhs/VIsWKkIYoQmLfRC2NtMLkvv9o7dSUg47qf4a/yqP8B/42xyYGM5B3Z4dp5n9mGWCbNYWWy3f2/VvHv3H2zvPPQfPX7y9Fl9d29g05mR0JdpnJqrUFiIlYY+KozhKjMgkjCGy/Dmg+Mv52CsSvUFLjMYJiLSaqKkQCqN6scBTgHFKD8v+Am/dTnkAUEeaBHGVCkZIgYHZ83zV6N6o91qr4LfBZ0SNFgZvdFu7SgYp3KWgEYZC2uvO+0Mh7kwqGQMhR/MLGRC3ogIrglqkYAd5qsJC75PlTGfpIaORr6q3u7IRWITgdMmJ+AkdoXsMgmbPExWlzTT9JBTbXQu1hab/jh5N8yVzmYIWq7tJ7OYY8rdJ/KxMiAxXnI/OAOaxsAXevZrBkZgal7ngTBRIhYFTRfxoMkd/p9U6b9SwlXSuTBF7lKVQKbzInepSmAjcnKpSgCGLFyqEiDxF9V0RnTg/hgx75mi2meRQakMw/wjCf19+lYhjaJ94HIqjJBIi+vTpnX+3au7YPCm1em23n7rNk4/lzu3w16wl+yAddgxO2WfWI/1mWTf2Q/2k/3ytr1Dr+sdraW1rbLnOdsI7/0fneAmpQ==</latexit>

✓D = ✓D + ⌘r✓GV (D,G)

<latexit sha1_base64="zdnWAcarHJ3Fuyqrojf8dqAAvEY=">AAADhXicfZLfaxNBEMe3ObU1/kr10ZfFUKgaw53E1hexYKQiiBGatNALYW8zuSy92zt250LCcX+Kf42v+gf43ziXHNhYzoEdvjvzmR1mmSCNlEXX/b3TcG7dvrO7d7d57/6Dh49a+49HNsmMhKFMosRcBMJCpDQMUWEEF6kBEQcRnAdXH8r8+QKMVYk+w1UK41iEWs2UFEihSevYxzmgmOT9gr/j1y4vuU+S+1oEEUWqzGnBR4f9zunzSavtdt218ZvCq0SbVTaY7DeO/Gkisxg0ykhYe+m5KY5zYVDJCIqmn1lIhbwSIVyS1CIGO87XExb8gCJTPksMHY18Hb1ekYvYxgLnHU6iROxa2VUcdHgQry9JqumhktqqXG5abPfH2dtxrnSaIWi5aT/LIo4JLz+RT5UBidGKN/0+0DQGvtCzX1MwAhPzIveFCWOxLGi6kPsdXur/oUr/RUnXoQthirx0dYBMFkVeujrAhtSpdHUAGGpRujoAKX9Wn04p7Zd/jJgPTFHfZ5lCRQZB/pHA5gF9q5BG0T5wORdGSKTFbdKmef/u1U0xet31et0333rtk8/Vzu2xp+wZO2QeO2Yn7BMbsCGT7Dv7wX6yX86u88rpOUcbtLFT1TxhW+a8/wOFbyad</latexit>

Page 6: On the Global Convergence of Gradient-Based Learning in

Gradient Descent-Ascent Dynamics

minx

maxy

S(x,y)

<latexit sha1_base64="NsgQ8OVYc7bq9u0xnxC3odXMzq8=">AAADd3icfZJbixMxFMezHS/reOvqow8Gy0pXSpmR9fK4oIIIYsXt7sJOKZk0nQ07uZCkpSXMZ/DT+Kqfw4/imyfTAa3LeCCHf875nRxyOLkuuXVJ8nOnE127fuPm7q349p279+539x6cWLUwlI2pKpU5y4llJZds7Lgr2Zk2jIi8ZKf55ZuQP10yY7mSx26t2USQQvI5p8RBaNo9yASXU5/lAq+qTJDVRq8r/KVfBwe4vh/gabeXDJPa8FWRNqKHGhtN9zovs5miC8GkoyWx9jxNtJt4YhynJavibGGZJvSSFOwcpCSC2Ymv/1ThfYjM8FwZONLhOvp3hSfCCuIuBhhEQGyt7FrkA5yL+qK0hIcCtVW52rTY7u/mryeeS71wTNJN+/mixE7hMDY844ZRV65xnL1l8BvDPsKznzQzxCnzzGfEFDC9Cn5X4GyAg/4fyuUfFHQbuiSm8sG1AVQtKx9cG2AL6BRcG8AMtAiuDXCQP25Pa0hnYcbO+ZGp2vusNGvIPPfvAIz3YayEGg77gOkFMYQ6WNUYNi39d6+uipPnw/Rw+OLzYe/oQ7Nzu+gReoL6KEWv0BF6j0ZojCj6ir6h7+hH51f0OHoa9TdoZ6epeYi2LEp/A6dFIj8=</latexit>

(⌧xx = �rxS(x,y)

⌧yy = ryS(x,y)

<latexit sha1_base64="lHH0gyrUqk1i4hWAuwiLq5PAO8s=">AAAD0nicfVNda9swFFXifXTeV7o9bXsRC4VupCEZ3cfLoLANxmAsY0lbqEKQFMUVtWUjySGe8MPYHvcH9xf2K3blGNq0eAJfHd97zj1IvmZZLI0dDP602sG16zdubt0Kb9+5e+9+Z/vBoUlzzcWEp3Gqjxk1IpZKTKy0sTjOtKAJi8URO3vn60dLoY1M1dgWmZgmNFJyITm1kJp1fhEmIqkchx6mDIml+cytSjJPrSMswasSv8V7RFEW01md+bZb7T3st+IZIbWsOJcVXnZRVVxWhUSoeW0763QH/UG18FUwrEEX1Ws0226/AiueJ0JZHlNjToaDzE4d1VbyWMA5ciMyys9oJE4AKpoIM3XVdZV4BzJzvEg1PMriKntR4WhiEmpPexiAp5gKmSJhPcyS6iXNFDTyrA3lam2x6W8Xb6ZOqiy3QvG1/SKPsU2x/yJ4LrXgNi5wSN4LOI0Wn6Htl0xoalP93BGqo4SuSjhdhEkPe/w/qlTnVMBN1CXVpfOhicDTZel8aCKYCJx8aCIIDRY+NBEs1MfN5QzKxN+xtW6ky2afVSZqJmPuAxDDHbhWyrWEecD8lGrKLfwFIUza8PJcXQWHL/rD/f7Lr/vdg0/1zG2hJ+gp2kVD9BodoI9ohCaIo7+tTutR63EwDr4HP4Kfa2q7VWseoo0V/P4HCSZDIQ==</latexit>

Continuous-time system:

Two questions:

1. When does the system converge?

2. Where does the system converge, if it converges?

Page 7: On the Global Convergence of Gradient-Based Learning in

Optimality in Minimax Optimization

Where does the system converge, if it converges?

Jin et al., What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? ArXiv, 2019

S(x⇤,y) S(x⇤,y⇤) maxy0

S(x,y0)

<latexit sha1_base64="QlJbK7FQoKn9mSr71RmUjS80JUI=">AAADnHicfZJta9swEMfVeA+d95RuLwdDLJS2IQR7dA8vC+tgbHTLWJMW6jTIysUVtSVNkkOC8bfal9nebh9kkuOxpcU78PH33e903HGxTJk2QfB9o+XduHnr9uYd/+69+w8etrcejbTIFYUhFalQpzHRkDIOQ8NMCqdSAcniFE7iyzcufzIHpZngx2YpYZyRhLMZo8TY0KT98ctuFGd4cd7tYSeWezhK4Su+Gj7v1okoI4tJUcV2yj9YDe3sTdqdoB9Uhq+LsBYdVNtgstV6GU0FzTPghqZE67MwkGZcEGUYTaH0o1yDJPSSJHBmJScZ6HFRDV7ibRuZ4plQ9uMGV9F/KwqS6YyYix62wiG6UnqZxT0cZ9WPkNw+5Ki1ysWqxXp/M3s9LhiXuQFOV+1neYqNwG63eMoUUJMusR8dgp1GwZF99pMERYxQ3SIiKrHrK+10CY562On/oYz/Ra1uQudElYVzTQAV87JwrgnQie3kXBMAyrZwrgkwNn/cnJY2HbkdG1MMVNncZyGhJuO4eGtBf9uulVDF7D1gekEUocbes28vLbx6V9fF6Hk/3O+/+LzfOXhf39wmeoKeoV0UolfoAL1DAzREFH1DP9BP9Mt76h16H7yjFdraqGseozXzRr8Bwh0saA==</latexit>

Page 8: On the Global Convergence of Gradient-Based Learning in

(⌧xx = �rxS(x,y)

⌧yy = ryS(x,y)

<latexit sha1_base64="lHH0gyrUqk1i4hWAuwiLq5PAO8s=">AAAD0nicfVNda9swFFXifXTeV7o9bXsRC4VupCEZ3cfLoLANxmAsY0lbqEKQFMUVtWUjySGe8MPYHvcH9xf2K3blGNq0eAJfHd97zj1IvmZZLI0dDP602sG16zdubt0Kb9+5e+9+Z/vBoUlzzcWEp3Gqjxk1IpZKTKy0sTjOtKAJi8URO3vn60dLoY1M1dgWmZgmNFJyITm1kJp1fhEmIqkchx6mDIml+cytSjJPrSMswasSv8V7RFEW01md+bZb7T3st+IZIbWsOJcVXnZRVVxWhUSoeW0763QH/UG18FUwrEEX1Ws0226/AiueJ0JZHlNjToaDzE4d1VbyWMA5ciMyys9oJE4AKpoIM3XVdZV4BzJzvEg1PMriKntR4WhiEmpPexiAp5gKmSJhPcyS6iXNFDTyrA3lam2x6W8Xb6ZOqiy3QvG1/SKPsU2x/yJ4LrXgNi5wSN4LOI0Wn6Htl0xoalP93BGqo4SuSjhdhEkPe/w/qlTnVMBN1CXVpfOhicDTZel8aCKYCJx8aCIIDRY+NBEs1MfN5QzKxN+xtW6ky2afVSZqJmPuAxDDHbhWyrWEecD8lGrKLfwFIUza8PJcXQWHL/rD/f7Lr/vdg0/1zG2hJ+gp2kVD9BodoI9ohCaIo7+tTutR63EwDr4HP4Kfa2q7VWseoo0V/P4HCSZDIQ==</latexit>

Continuous-time system:

Kose-Uzawa Theorem: convex-concave payoff fn.

T. Kose. Solutions of Saddle Value Problems by Differential Equations. Econometrica, 24:59–70, 1956. H. Uzawa. Gradient method for concave programming, II global stability in the strictly concave case. Stanford Univ., 1958.

“If S(x, y) is strictly convex in x and strictly concave in y, then gradient descent-ascent converges a saddle point.”

S(x, y) = (x2 � y2)/2

<latexit sha1_base64="IraZSyfxFAdF4+1DMowVqExxN1I=">AAADYnicfZJtb9MwEMe9hocRnlrGO3hhUU3qUFeSajy8QZoESAgJUcS6TVq6ynGvmbXEtmynSojyXXgL34j3fBCcNBKUKZyU0993v/PFpwtlzLTxvJ9bHefa9Rs3t2+5t+/cvXe/23twrEWqKEypiIU6DYmGmHGYGmZiOJUKSBLGcBJevqnyJytQmgl+ZHIJs4REnC0ZJcaG5t2HXwbZMN/Dr/EgOx/v5+fjvWfjebfvjbza8FXhN6KPGpvMe50XwULQNAFuaEy0PvM9aWYFUYbRGEo3SDVIQi9JBGdWcpKAnhX175d410YWeCmU/bjBdfTvioIkOiHmYoitqBBdK50n4RCHSX0QktuLKmqjMlu32Oxvlq9mBeMyNcDpuv0yjbERuJoQXjAF1MQ5doO3YF+j4KO99pMERYxQT4uAqCghWWlfF+FgiCv9P5TxP6jVbeiKqLKoXBtAxaosKtcG6Mh2qlwbAMq2qFwbYGz+qD0tbTqoZmxMMVFle59MQkOGYfHOgu6uHSuhitl9wPSCKEKN3UrXbpr/715dFcfjkX8wev75oH/4odm5bfQIPUED5KOX6BC9RxM0RRR9Rd/Qd/Sj88txnZ6zs0Y7W03NDtow5/FvSEAYbQ==</latexit>

E.g.,

Page 9: On the Global Convergence of Gradient-Based Learning in

New Theorem: globally less convex in y(⌧xx = �rxS(x,y)

⌧yy = ryS(x,y)

<latexit sha1_base64="lHH0gyrUqk1i4hWAuwiLq5PAO8s=">AAAD0nicfVNda9swFFXifXTeV7o9bXsRC4VupCEZ3cfLoLANxmAsY0lbqEKQFMUVtWUjySGe8MPYHvcH9xf2K3blGNq0eAJfHd97zj1IvmZZLI0dDP602sG16zdubt0Kb9+5e+9+Z/vBoUlzzcWEp3Gqjxk1IpZKTKy0sTjOtKAJi8URO3vn60dLoY1M1dgWmZgmNFJyITm1kJp1fhEmIqkchx6mDIml+cytSjJPrSMswasSv8V7RFEW01md+bZb7T3st+IZIbWsOJcVXnZRVVxWhUSoeW0763QH/UG18FUwrEEX1Ws0226/AiueJ0JZHlNjToaDzE4d1VbyWMA5ciMyys9oJE4AKpoIM3XVdZV4BzJzvEg1PMriKntR4WhiEmpPexiAp5gKmSJhPcyS6iXNFDTyrA3lam2x6W8Xb6ZOqiy3QvG1/SKPsU2x/yJ4LrXgNi5wSN4LOI0Wn6Htl0xoalP93BGqo4SuSjhdhEkPe/w/qlTnVMBN1CXVpfOhicDTZel8aCKYCJx8aCIIDRY+NBEs1MfN5QzKxN+xtW6ky2afVSZqJmPuAxDDHbhWyrWEecD8lGrKLfwFIUza8PJcXQWHL/rD/f7Lr/vdg0/1zG2hJ+gp2kVD9BodoI9ohCaIo7+tTutR63EwDr4HP4Kfa2q7VWseoo0V/P4HCSZDIQ==</latexit>

Continuous-time system:

S is twice differentiable

�inf(Sxx) := infx,y

�min(Sxx(x,y))

<latexit sha1_base64="CYWiRjOblOOMItxeP8u4/MgezoE=">AAADsXicfZJta9swEMfVeA+d95RuL/dGLBSSEULcde0YDArbYAzGMta0HXXIZEV2RSxZSHJIMP6A+wj7FHu7vdvJMWuz4h349Pfd73TouEil3Njh8MdWy7tx89bt7Tv+3Xv3Hzxs7zw6MVmuKRvTLM30WUQMS7lkY8ttys6UZkREKTuN5m9c/nTBtOGZPLYrxSaCJJLHnBILoWmbhinAMzItQi7jsvsFRCTwsnJlD796jV2ijvaxO1YlvqwSXF5W4XVZ9yrc603bneFgWBm+LoJadFBto+lO6yCcZTQXTFqaEmPOg6Gyk4Joy2nKSj/MDVOEzknCzkFKIpiZFNU0SrwLkRmOMw2ftLiKXq0oiDCC2Is+BuEQUymzElEfR6L6yZSEixy1Ublct9jsb+OXk4JLlVsm6bp9nKfYZtgNHM+4ZtSmK+yHbxm8RrOPcO0nxTSxmX5WhEQngixLeF2Cwz52+n8ozPsvCroJXRBdFs41ATRblIVzTYBJoJNzTQDT0MK5JsBC/rg5rSAduhlbW4x02dxnqVhNRlHxDkB/F8ZKqOawD5heEE2ohSX3YdOCf/fqujjZGwT7gxef9ztHH+qd20ZP0FPURQE6REfoPRqhMaLoO/qJfqHf3nPvq/fNi9Zoa6uueYw2zJv/AVH+N5c=</latexit>

If

then all bounded trajectories converge.

and steady states exists,⌧�1x �inf(Sxx) > ⌧�1

y �sup(Syy)

<latexit sha1_base64="r94Pw7Tshrjs/eIu+SEpJMr4MAA=">AAADqXicfZLdbtMwFMe9ho8Rvjq45MaimjSgVAkaG1doEiAhJEQn1q2iKZHjOpm1xLFsp2pk5eF4DJ6AW3gDTtII6KZwpBz94/M7/stHJ5Ip18bzvm/1nGvXb9zcvuXevnP33v3+zoNTnReKsgnN01xNI6JZygWbGG5SNpWKkSxK2Vl08aauny2Z0jwXJ6aUbJ6RRPCYU2LgKOzPAkOK0K6qr/a5XwUpdC5IaAMu4mrvM4gow6smVU8wfo3XeHkZ14X8g5dNAjzsD7yR1wS+KvxWDFAb43CndxAsclpkTBiaEq1nvifN3BJlOE1Z5QaFZpLQC5KwGUhBMqbntplChXfhZIHjXMEnDG5O/+2wJNMZMedDDKJGdKN0mUVDHGXNTy4FXFRTG52rtcWmv4lfzS0XsjBM0LV9XKTY5LgeNF5wxahJS+wGbxm8RrGPcO0nyRQxuXpqA6KSjKwqeF2CgyGu9f9QLv6ioLvQJVGVrVMXQPNlZevUBegEnOrUBTAFFnXqAgzUT7rLEspBPWNj7FhV3T4ryVoyiuw7AN1dGCuhisM+YHpOFKEGltuFTfMv79VVcfpi5O+PXh7vD44+tDu3jR6hx2gP+egQHaH3aIwmiKJv6Af6iX45z5xjZ+p8WaO9rbbnIdoIh/4GRSE2kA==</latexit>

�sup(Syy) := supx,y

�max(Syy(x,y))

<latexit sha1_base64="2jbfE9Ia95PHCORplZsqvx8ZLnI=">AAADsXicfZJdi9NAFIZnGz/W+NXVS28Gy0IrpSS6u4ogLKggglhxu7uyKXVmOs0OzSTDzKS0hPxAf4K/wlu980wa2K0lDuTw5pznzMscDlWJMDYIfu60vBs3b93evePfvXf/wcP23qNTk+Wa8RHLkkyfU2J4IlI+ssIm/FxpTiRN+Bmdv3X1swXXRmTpiV0pPpYkTsVMMGIhNWmzKAF4SiZFZHJVdr+CoBKvcBXLHn79BrvKOr3s1/mrLkmWW13d62yvN2l3gkFQHbwtwlp0UH2Gk73WUTTNWC55allCjLkIA2XHBdFWsISXfpQbrgibk5hfgEyJ5GZcVNMo8T5kpniWafhSi6vs9Y6CSCOJvexjEA4xlTIrSfuYyuonUylc5KiNzuXaYtPfzl6NC5Gq3PKUre1neYJtht3A8VRozmyywn70jsNrNP8E135WXBOb6WdFRHTshgivi3HUx07/DxXpFQq6CV0QXRYuNAEsW5SFC02AicHJhSaAa7BwoQmwUD9pLisoR27G1hZDXTb7LBWvSUqL9wD6+zBWwrSAfcDskmjCLCy5D5sW/rtX2+L0+SA8GBx+Oegcf6x3bhc9QU9RF4XoJTpGH9AQjRBDP9Av9Bv98V5437zvHl2jrZ265zHaON78LwsSN9M=</latexit>

Page 10: On the Global Convergence of Gradient-Based Learning in

LaSalle’s Invariance Principle

System: z = f(z), z 2 Rk, f(z⇤) = 0

<latexit sha1_base64="4BASSXckLKgpp7vbsyUb7Rv+tzE=">AAADjnicfVJda9swFFXjfXTeV7o97kUsFNpigl26bi9hgW1QBmPZaNpCnQZJkV0RWxaSHJIJ/539mr1usH8zyfHYsuJd8PXxvefq+IqDRcaUDsOfWx3v1u07d7fv+fcfPHz0uLvz5EwVpSR0TIqskBcYKZoxTsea6YxeCElRjjN6judvXP98QaViBT/VK0EnOUo5SxhB2pam3WE8K7SJcQ6/VHAAk70a7gewfseMwzhH+hpj+PlqHvzuX5mDan/gYDjt9sJ+WAe8CaIG9EATo+lO59hqkjKnXJMMKXUZhUJPDJKakYxWflwqKhCZo5ReWshRTtXE1KtWcNdWZjAppH24hnX17wmDcuV+OIAWOIqqkVrlOIA4rz8Kwe1BjrUxuVxLbOrr5NXEMC5KTTlZyydlBnUB3W3CGZOU6GwF/fgttdtI+sEe+1FQiXQhD0yMZJqjZWW3S2EcQIf/R2X8D9XiNuoCycq41EYgxaIyLrURVGqVXGojUGklXGojaNs/bW8L266to7UZyapdZylow8TYvLNEf9deKyKSWT9Aco0kIto62LdOi/711U1wdtiPjvovPh31hu8bz22DZ+A52AMReAmG4ASMwBgQ8BV8A9/BD6/rHXsD7/Wa2tlqZp6CjfBOfgGNvCi6</latexit>

L : Rk ! R

<latexit sha1_base64="b9PVrOMsZ/OsTtAZUvwwaYmp16U=">AAADcXicfZL/btMwEMe9hh8j/NgGfyH+sVZNQlBVCdoG4q9JgIQQiILWbdJSJtu9plYT27Kd0irKA/A0/AuPwnPwAthpBJQpnJTT13ef8+WsoyrjxkbRj41OcOXqteubN8Kbt27f2dreuXtiZKEZDJnMpD6jxEDGBQwttxmcKQ0kpxmc0tkLnz+dgzZcimO7VDDKSSr4hDNiXehiu/v2eZITO6UUf/w0w4nm6dQSreVn/DvuqKgf1YYvi7gRXdTY4GKnc5iMJStyEJZlxJjzOFJ2VBJtOcugCpPCgCJsRlI4d1KQHMyorKep8J6LjPFEavcJi+vo3xUlyY3/tR52wiOmVmaZ0x6meX2QSriLPLVWuVi1WO9vJ89GJReqsCDYqv2kyLCV2D8YHnMNzGZLHCYvwU2j4Z279r0CTazUj8qE6DQni8pNl+Kkh73+H8rFH9TpNnROdFV61wYwOa9K79oAk7pO3rUBoF0L79oA6/LH7Wnl0vWSWFsOdNXeZ6GgISktXzkw3HPPSpjmbh8wmxJNmHVLGrpNi//dq8vi5Ek/3u8ffNjvHr1pdm4TPUC76CGK0VN0hF6jARoihr6gr+gb+t75GdwPcLC7QjsbTc09tGbB41/9QiCB</latexit>

continuously differentiable.

<latexit sha1_base64="g3/oBsGTZ08JMoSSJSokMeFhJ+M=">AAADUnicfZJLa9tAEMfXUh+pmrZJeuxlqQmUYowU0scx0BRKocSF2AlEJqzWY2Ub7YPdlbER+g69Np8rl36VnrorC1o3qAMa/jvzmx3tMJkqmLFx/LMXhPfuP3i49Sh6vP3k6bOd3b2JkaWmMKaykPo8IwYKJmBsmS3gXGkgPCvgLLv+4PNnC9CGSXFqVwqmnOSCzRkl1oUm6QmHnFzu9ONh3Bi+K5JW9FFro8vd4G06k7TkICwtiDEXSazstCLaMlpAHaWlAUXoNcnhwklBOJhp1fxujfddZIbnUrtPWNxE/66oCDec2KsBdsIjplFmxbMBznhzkEq4izy1Ublct9jsb+fvpxUTqrQg6Lr9vCywldhPBM+YBmqLFY7SY3Cv0fDFXXuiQBMr9esqJTrnZFm71+U4HWCv/4cy8Qd1ugtdEF1X3nUBVC7qyrsuwOSuk3ddAGjXwrsuwLr8aXdauXTqZ2xtNdJ1d5+lgpbMsuqjA6N9N1ZCNXP7gOkV0YRat4WR27Tk3726KyYHw+Rw+ObrYf/oc7tzW+gFeoleoQS9Q0foExqhMaLoG/qOfqCb4Db4FfbCcI0GvbbmOdqwcPs3CicVxg==</latexit>

compact positive invariant set w.r.t. the system dynamics

M ✓ E

<latexit sha1_base64="7BPDlX36tW28eToayim2jOw0Izw=">AAADWHicfZLfa9swEMfVeFtb71fbPe5FLBTGCMEu3drHwlYYg7IMmrZQhyIrF1fUljTpHBKM/429bv/W9tdMcgxbVrwDH1/dfU5nHZfqXFiMop8bveDBw0ebW9vh4ydPnz3f2d27sKo0HMZc5cpcpcxCLiSMUWAOV9oAK9IcLtO79z5/OQdjhZLnuNQwKVgmxUxwhi6UnCW2TC0gfKWnNzv9aBg1Ru+LuBV90troZrf3LpkqXhYgkefM2us40jipmEHBc6jDpLSgGb9jGVw7KVkBdlI1P13TfReZ0pky7pNIm+jfFRUrbMHwdkCd8IhtlF0W6YCmRXNQWrqLPLVWuVi1WO+Ps+NJJaQuESRftZ+VOUVF/VzoVBjgmC9pmHwA9xoDZ+7azxoMQ2XeVAkzWcEWtXtdRpMB9fp/qJB/UKe70DkzdeVdF8DVvK686wJs5jp51wWAcS286wLQ5c+709qlEz9jxGpk6u4+Cw0tmabVqQPDfTdWxo1w+0D5LTOMo9vF0G1a/O9e3RcXB8P4cPj2y2H/5FO7c1vkJXlFXpOYHJET8pGMyJhwosk38p386P0KSLAZbK/Q3kZb84KsWbD3G0x3Fzg=</latexit>

⌦ ⇢ Rk

<latexit sha1_base64="EWSez0n75IibVmq7kY51bYAMHGc=">AAADaHicfVJdaxQxFE13/KjjR7f1QdSX0KUgsiwzUq2PBRVEkK7SbQuddUmyd6dhJ5mQZJZdhgF/ja/6d/wL/gqT2QFdy3hhLif3npszNxyqMm5sFP3c6gQ3bt66vX0nvHvv/oOd7u7emckLzWDE8izXF5QYyLiEkeU2gwulgQiawTmdv/H98wVow3N5alcKxoKkks84I9aVJt0nyYmAlODEFNSAxYkg9opS/PnLfNLtRYOoDnwdxA3ooSaGk93Oq2Sas0KAtCwjxlzGkbLjkmjLWQZVmBQGFGFzksKlg5IIMOOyXqLCB64yxbNcu09aXFf/niiJMP7n+tgBTzE1MitB+5iK+pAr6S7yrI3J5VpiU9/OXo9LLlVhQbK1/KzIsM2xfyc85RqYzVY4TN6C20bDR3ftiQJNbK6flwnRqSDLym2X4qSPPf4flcs/VIfbqAuiq9KnNgLLF1XpUxvBpE7JpzYCaCfhUxvBuv5pe1u5dm0Ta8uhrtp1lgoaJqXlO0cMD9yzEqa58wNmV0QTZp03Q+e0+F9fXQdnLwbx4eDlp8Pe8YfGc9voKdpHz1CMjtAxeo+GaIQY+oq+oe/oR+dX0A0eBY/X1M5WM/MQbUSw/xv8Dhz3</latexit>

L(z(t)) 0

<latexit sha1_base64="TPORk+LIkIlBgF47jpPqW3CAuuk=">AAADaXicfVJtaxNBEN7mfKnnSxP9IvbLYiikEsKdtOrHggoiihGattALYW8zuS69273uzoXE4z74a/yqP8ff4J9w9xLQWM6BHZ6deWYeZpg4T4XBIPi51fJu3Lx1e/uOf/fe/Qc77c7DE6MKzWHEVar0WcwMpELCCAWmcJZrYFmcwml8+drlT+egjVDyGJc5jDOWSDETnKENTdq70VQh/dCL4ox+6eH+Po1SuKLuG0za3WAQ1Eavg3ANumRtw0mn9cK240UGEnnKjDkPgxzHJdMoeAqVHxUGcsYvWQLnFkqWgRmX9RQV3bORKZ0pbZ9EWkf/rihZZjKGF31qgaOYGpllFvdpnNUflUvbyLE2KhcriU19nL0al0LmBYLkK/lZkVJU1C2KToUGjumS+tEbsNNo+GjbfspBM1T6WRkxnWRsUdnpEhr1qcP/owr5h2pxE3XOdFU610Tgal6VzjURTGKVnGsigLYSzjUR0OaPm9O5TUdux4jlUFfNOosc1sw4Lt9aor9n18q4FvYeKL9gmnG0x+nbSwv/vavr4OT5IDwYHH4+6B69X9/cNtklT0mPhOQlOSLvyJCMCCdfyTfynfxo/fI63mPvyYra2lrXPCIb5nV/A4tsG8g=</latexit>

inE = {z 2 Rk : L(z) = 0}

<latexit sha1_base64="xW1UZ5Hxk4EnbsW/aK0zq2X8S9A=">AAADgnicfZJtixMxEMdz3VPP+tTTl74JloPzKGVX6wOicKAHIopVrncHl1qSNN0L3c0uSba0hv0ifhrf6jfw2zjZLmg91sAO/8z8ZmYnDMsTaWwY/tpqBdtXrl7bud6+cfPW7Tud3bsnJis0FyOeJZk+Y9SIRCoxstIm4izXgqYsEads/trHTxdCG5mpY7vKxTilsZIzyakF16QzOMKvMHGEpfgrkQqTlNoLxvDnL25eviDTzOL3+1X0oQdBhKScdLphP6wOviyiWnRRfYaT3dZTKMWLVCjLE2rMeRTmduyotpInomyTwoic8jmNxTlIRVNhxq4ar8R74JniWabhUxZX3r8zHE2N/+0eBuERUymzSlkPs7S6ZLmCQp7ayFyuW2z2t7PnYydVXlih+Lr9rEiwzbB/QTyVWnCbrHCbvBEwjRYfoOzHXGhqM33gCNVxSpclTBdj0sNe/w+V6g8KugldUF06b5oAni1K500TYGLo5E0TIDS08KYJsBA/bg7nEK4WyFo31GVzn2UuapIxdwRgew+elXItYR8wv6Cacgtb24ZNi/7dq8vi5FE/GvSffBp0D9/VO7eD7qMHaB9F6Bk6RG/REI0QR9/Qd/QD/Qy2g4MgCh6v0dZWnXMPbZzg5W9+uyRe</latexit>

the largest invariant set in E

<latexit sha1_base64="QmtJVpq96IK3IETEDzA3pO7swk0=">AAADTXicfZLNatwwEMcVp21S9ytJj72ILoFSlsUO6ccx0AZKoXQD2SQQL0HWzjoitiSk8bKL8RPk2j5XznmQ3kqp5DW02+AOePhr5jcaa5hU58JiFN2uBev37j/Y2HwYPnr85Omzre2dE6tKw2HEVa7MWcos5ELCCAXmcKYNsCLN4TS9+uDzpzMwVih5jAsN44JlUkwFZ+hCR4cXW71oEDVG74q4FT3S2vBiO3ibTBQvC5DIc2bteRxpHFfMoOA51GFSWtCMX7EMzp2UrAA7rpo/remui0zoVBn3SaRN9O+KihW2YHjZp054xDbKLoq0T9OiOSgt3UWeWqmcL1us9sfp+3ElpC4RJF+2n5Y5RUX9MOhEGOCYL2iYfAT3GgNf3LVfNRiGyryuEmaygs1r97qMJn3q9f9QIf+gTnehM2bqyrsugKtZXXnXBdjMdfKuCwDjWnjXBaDLH3entUsnfsaI1dDU3X3mGloyTatDB4a7bqyMG+H2gfJLZhhHt4Ch27T43726K072BvH+4M3Rfu/gc7tzm+QFeUlekZi8IwfkExmSEeEEyDX5Rr4HN8GP4Gfwa4kGa23Nc7Ji6xu/AbZ3FZI=</latexit>

limt!1

✓infp2M

kz(t)� pk◆

= 0

<latexit sha1_base64="j/aa+LeT8FCUn1sS7JRh+13E1LE=">AAADpHicfVLbbhMxEHWzXEq4pfDIi0VU1KIQbVC5vCBVAiSEqAiiSSvVIfI63o3V9dqyJyHB7J/xI7zyCh+BvYkEoVpG2tnjM2d85NEkOhcW4vj7ViO6dPnK1e1rzes3bt663dq5M7RqZhgfMJUrc5pQy3NR8AEIyPmpNpzKJOcnyfnLUD+Zc2OFKo5hqflI0qwQqWAUPDVuDUku5NgBMSKbAjVGfcZEFCksS0xynsJeOI0dSSTWHuKjsqLJ18B82YP9R3rV65nqv49f4HjcasfduAp8EfTWoI3W0R/vNJ6SiWIzyQtgObX2rBdrGDlqQLCcl00ys1xTdk4zfuZhQSW3I1cNoMS7npngVBn/FYAr9u8OR6WVFKYd7EGQ2ArZpUw6OJHVQenCXxRUG52LlcWmP6TPR04Uega8YCv7dJZjUDjMGE+E4QzyJW6SV9y/xvAjf+17zQ0FZR46Qk0m6aL0r8sw6eCA/ycVxR+px3XSOTWlC6lOwNS8dCHVCWzmnUKqE3DjLUKqE4CvH9eXtS+TMGMA1zdlvc9C87UySdxrL2zu+rFSZoTfB8ym1FAGfq+bftN6/+7VRTB83O0ddJ98OGgfvl3v3Da6h+6jPdRDz9AheoP6aIAY+oZ+oJ/oV/Qgehd9jAYraWNr3XMXbUT06TduYjTK</latexit>

E ⇢ ⌦

<latexit sha1_base64="/JOdtAd7QegM0TMBua5A5NQ5Mdo=">AAADXXicfZLfaxNBEMe3Oav1rG2qDz74shgKIiHkpNo+FrQggjRC0xZ6IextJteltz/YnQsJx/0lvuof5ZP/inuXA43lHLjhuzOf2bkdJjGZcDgc/tzqBA+2Hz7aeRw+2X26t989eHbpdG45jLnOtL1OmINMKBijwAyujQUmkwyukrsPVf5qAdYJrS5wZWAiWarEXHCGPjTt7p/FLk8cII3PJaRs2u0NB8Pa6H0RNaJHGhtNDzrv45nmuQSFPGPO3URDg5OCWRQ8gzKMcweG8TuWwo2Xiklwk6L+85Ie+siMzrX1n0JaR/+uKJh0kuFtn3pRIa5WbiWTPk1kfdBG+YsqaqNyuW6x2R/nJ5NCKJMjKL5uP88zippWw6EzYYFjtqJh/BH8ayx88deeG7AMtX1TxMymki1L/7qUxn1a6f+hQv1BvW5DF8yWReXaAK4XZVG5NsClvlPl2gCwvkXl2gD0+Yv2tPHpuJoxYjGyZXufpYGGTJLizIPhoR8r41b4faD8llnG0S9k6Dct+nev7ovLt4PoaPDu61Hv9HOzczvkJXlFXpOIHJNT8omMyJhwkpNv5Dv50fkVbAe7wd4a7Ww1Nc/JhgUvfgO8FBip</latexit>

z(0) 2 ⌦ ) z(t) 2 ⌦, 8t � 0

<latexit sha1_base64="eheqzo5IiG8CTlGRyoSiJ6wJFMw=">AAADknicfVLbbhMxEHWzXMpyS4E3XiyiSi2Kog0qF/FU1CIhBGpATVupjiKvM9lYXduLPQkJq/0hvobX8jV4k5VKqBZLOzqec8ZnZzRxlkqHUXS50Qhu3Lx1e/NOePfe/QcPm1uPTpyZWgF9YVJjz2LuIJUa+igxhbPMAldxCqfxxUHJn87AOmn0MS4yGCieaDmWgqNPDZuHLFb0x060S5nUlB0pSDhlX2UyQW6t+U5XPO5e0W3KxsbyNKVIWQLfaDRstqJOtDz0OuhWoEWq0xtuNV6xkRFTBRpFyp0770YZDnJuUYoUipBNHWRcXPAEzj3UXIEb5Mt2C7rtMyPq/8F/Guky+3dFzpVTHCdt6kEpcUvkFipu01gtLybT/qFStVY5X1ms++P4zSCXOpsiaLGyH09994aWE6UjaUFguqAhOwTfjYXP/tmjDCxHY5/njNtE8Xnhu0soa9MS/08q9ZXU4zrpjNsiL0OdQJhZkZehTuAS71SGOgFYb1GGOgF6/riezjzNyhkj5j1b1PvMM6iUcZy/98Jw24+VCyv9PlAx4ZYL9Fsc+k3r/rtX18HJi053r/Pyy15r/2O1c5vkKXlGdkiXvCb75APpkT4R5Cf5RS7J7+BJ8DZ4FxyspI2NquYxWTvBpz8KSyrK</latexit>

z(0) 2 M ) z(t) 2 M, 8t 2 R

<latexit sha1_base64="lXQDWzbQ49FS5XdWY2xItwq4Vog=">AAADj3icfZJtaxNBEMe3OR9qfEr0pW8WQ6GVEC6l2r4qERVUKMbStIVeCLubzWXp3e6xOxcTj/s8fhrfKn4b5y4HGss5cMP/Zn6zs7MMTyLlwPd/bTW8W7fv3N2+17z/4OGjx632k3NnUivkSJjI2EvOnIyUliNQEMnLxEoW80he8Os3Rf5iIa1TRp/BKpHjmIVazZRggKFJ63XAY/p119+jgdL0hAanKpwDs9Z8oesU7JWZLg1mxrIoolCiQcxgzjk9nbQ6fs8vjd4U/Up0SGXDSbvxKpgakcZSg4iYc1d9P4FxxiwoEcm8GaROJkxcs1BeodQslm6clbPmdAcjU4pXwU8DLaN/V2QsdsXVuhRFgbhSuVXMu5TH5Y9JNB5UUBuVy3WLzf4wOxpnSicpSC3W7WcpPoKhxXPSqbJSQLSizeCtxGmsPMFjPyXSMjD2RRYwG8ZsmeN0IQ26tND/Q5X+g6KuQxfM5lnh6gBhFnlWuDrAhdipcHWAtNiicHUAYP6sPp1gulwSgGxo8/o+y0RWJOfZOwSbO/isTFiF+0DFnFkmAFe4iZvW/3evborz/V7/oPfy80Fn8LHauW3yjDwnu6RPDsmAvCdDMiKCfCPfyQ/y02t7h96xN1ijja2q5inZMO/Db4CHKWI=</latexit>

Page 11: On the Global Convergence of Gradient-Based Learning in

Construction of a Lyapunov Function

Continuous-time system:

(⌧xx = �Sx

⌧yy = Sy

<latexit sha1_base64="ynei9vOVJcdIfJrKG1jSeNAUnmU=">AAADqnicfZLdihMxFMezHT/W8aurl94Ey4JILe2yftwICyqIIFbd7q40pWQyp7NhZzJDkikdwrycb+EbeKtP4Ek7sNZlDOTwT84v/0MOJypSaexw+GOnE1y7fuPm7q3w9p279+539x6cmLzUAiYiT3N9FnEDqVQwsdKmcFZo4FmUwml08cbnT5egjczVsa0KmGU8UXIhBbd4Ne8yFkEilRPoYeqQWV7O3apmcW4dizK6qulr+uzrvDkw1iDVJVJ5pCEqtAAVN3bzbm84GK4XvSpGjeiRZo3ne50XaCvKDJQVKTdmOhoWdua4tlKkgOalgYKLC57AFKXiGZiZW7ehpvt4E9NFrnErS9e3f79wPDMZt+d9isIjZq1MlUV9GmXrQ14oNPLU1svVpsR2fbt4NXNSFaUFJTblF2VKbU59p2ksNQibVjRkbwF/o+Ej2n4qQHOb66eOcZ1kfFXj7xLK+tTr/6FSXaKo29Al17XzoQ0Q+bJ2PrQBJsFKPrQBoLGED22Axfxxe7rANPM9ttaNdd1eZ1VAQ0aRe4dguI9t5UJLnAcqzrnmwuJ0hzhpo3/n6qo4ORiMDgfPPx/2jj40M7dLHpHH5AkZkZfkiLwnYzIhgnwnP8kv8jvoB1+Cb8F0g3Z2mjcPydYK4j9p9zeS</latexit>

)(⌧xx = �Sxxx� Sxyy

⌧yy = Syxx+ Syyy

<latexit sha1_base64="x/gsOxaeZ/4u2vJ8sfKqrtfduwg=">AAAEEXicfVPbjtMwEHUaLku4deGRF4tqJS7dqkXL5QVpJUBCSGgLbHeL1lXluG5qbWJHtlMaWfkKvoY3BI98AX/DpI3YlipYyuh4zpkzsTMJ01gY2+3+9hr+pctXru5cC67fuHnrdnP3zolRmWZ8wFSs9DCkhsdC8oEVNubDVHOahDE/Dc9flfzpnGsjlDy2ecpHCY2kmApGLaTGzZ/ko4hmlmqtvmAS8khIx8DPFAGxNBu7RUEmE2UdCRO8KPBLvP9pvNqsMmSN3MfrXH7BASSVX77ml5d+VUm+bff4L4e3/ALC5aR603Gz1e10lwtvg14FWqha/fFu4xlYsSzh0rKYGnPW66Z25Ki2gsUczDPDU8rOacTPAEqacDNyy9su8B5kJniqNDzS4mV2vcLRxCTUztoYQCkxS2TyJGzjMFluVCrBqFRtVC5WLTb72+mLkRMyzSyXbNV+msXYKlx+UDwRmjMb5zggrzmcRvP3YHuUck2t0o8coTpK6KKA00WYtHGJ/ycV8kIKuE46p7pwZagTMDUvXBnqBCaCTmWoE3ANLcpQJ7DAH9fTKdCkvGNrXV8X9X0WKa+UYejegDDYg2ulTAuYB8xmVFNm4ScKYNJ6/87VNjh50ukddJ5+OGgdvqtmbgfdQ/fRA9RDz9Eheov6aICY99A78obeZ/+r/83/7v9YSRteVXMXbSz/1x8AOV1k</latexit>

a =p⌧xx

<latexit sha1_base64="3gd3276LXzmackVUo9ih/6a5Z5Y=">AAADaHicfZJta9RAEMe3Fx9qfLrqC1HfLD0KIseRSK2+EQoqiCCe0GsLzXFs9vbSpcnudndy5FgCfhrf6tfxK/gp3M0F9CxxIMM/M7/ZYYZJVc4NRNHPrV5w7fqNm9u3wtt37t673995cGxkqSmbUJlLfZoSw3Iu2AQ45OxUaUaKNGcn6cVbnz9ZMm24FEewUmxakEzwBacEXGjWf5KkBSb4DU7MpQabAClnVe2D1aw/iEZRY/iqiFsxQK2NZzu9g2QuaVkwATQnxpzFkYKpJRo4zVkdJqVhitALkrEzJwUpmJnaZoga77nIHC+kdp8A3ET/rrCkMAWB8yF2wiOmUWZVpEOcFs2PVMI95KmNymrdYrM/LF5PLReqBCbouv2izDFI7PeE51wzCvkKh8k75qbR7JN79rNimoDUz21CdFaQqnbTZTgZYq//h3LxB3W6C10SXVvvugAql7X1rgswmevkXRfAtGvhXRcALn/UnVYunfgdA9ixrrv7VIq1ZJra9w4M99xaCdXc3QOm50QTCu42Q3dp8b93dVUcvxjF+6OXX/YHhx/bm9tGT9EueoZi9Aodog9ojCaIoq/oG/qOfvR+Bf3gUfB4jfa22pqHaMOC3d8VTB0C</latexit>

b =p⌧yy

<latexit sha1_base64="Rm3oNZ7x2Y2R8NkbPQzdZWblHz4=">AAADaHicfZJta9RAEMe3Fx9qfLrqC1HfLD0KIseRSK2+EQoqiCCe0GsLzXHs7u2lS7O76e7kuBACfhrf6tfxK/gp3M0F9CxxIMM/M7/ZYYaheSYsRNHPrV5w7fqNm9u3wtt37t673995cGx1YRifMJ1pc0qJ5ZlQfAICMn6aG04kzfgJvXjr8ydLbqzQ6gjKnE8lSZVYCEbAhWb9JwmVmOI3OLGXBqoESDErax8sZ/1BNIoaw1dF3IoBam082+kdJHPNCskVsIxYexZHOUwrYkCwjNdhUlieE3ZBUn7mpCKS22nVDFHjPReZ44U27lOAm+jfFRWRVhI4H2InPGIbZUtJh5jK5kfnyj3kqY3K1brFZn9YvJ5WQuUFcMXW7RdFhkFjvyc8F4YzyEocJu+4m8bwT+7Zzzk3BLR5XiXEpJKsajddipMh9vp/qFB/UKe70CUxdeVdF8D0sq686wJs6jp51wVw41p41wWAyx91p3OXTvyOAaqxqbv7rHLekpRW7x0Y7rm1EmaEuwfMzokhDNxthu7S4n/v6qo4fjGK90cvv+wPDj+2N7eNnqJd9AzF6BU6RB/QGE0QQ1/RN/Qd/ej9CvrBo+DxGu1ttTUP0YYFu78BHiIdBQ==</latexit>

Scale transformation: ,

✓ab

◆= �

✓Sxx/⌧x Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y �Syy/⌧y

◆✓ab

<latexit sha1_base64="pDcRubuQGqNl6b0ahxJ23xbLFME=">AAAEn3icfZPfb9MwEMedEmCEH1vhkRdDNTRQ17Vo/HhBmgQIhAQUsW5Dc1U5rptaS5xgO1UjK38XfwsPvMK/wSWNGNmWWcrlG9/n7pzLxU9CoU2//9NpXXGvXru+dsO7eev2nfWN9t0DHaeK8RGLw1gd+VTzUEg+MsKE/ChRnEZ+yA/9k9eF/3DBlRax3DdZwscRDaSYCUYNbE3azpCEfGa2POLzQEgbUaPEMvcwJtNpbCzxI0xzQmobfu4RLqf/YKJEMDeP8SvsbTel+zopQ5e4tDnewcTQdGJBPqo7s3yH6O8KSlVAec9Wh9iu0KzKczEKKetgdlowu/jwXnMfzrbh8i5MNjr9Xr9c+LwYVKKDqjWctFvPISdLIy4NC6nWx4N+YsaWKiNYyCFxqnlC2QkN+DFISSOux7b89jnehJ0pnsUKLmlwuft/hKWRhhPOuxhEgehS6Szyu9iPyoc4kZCooGqRy1WJen0zezm2Qiap4ZKtys/SEJsYF+OFp0JxZsIMe+QNh7dR/COk/ZxwRU2snlhCVRDRZQ5vF2DSxYW+DBXyFAXdhC6oym1hmgAWL3JbmCZAB1CpME0AV1CiME2AAf9+szsBNyl6bIwdqry5zjLhFen79i2A3ia0lTIlYB4wm1NFmYFf2oNJG5ydq/Pi4GlvsNt79mW3s/ehmrk1dB89RFtogF6gPfQeDdEIMeeH88v57fxxH7jv3E/ucIW2nCrmHqot99tfIrKL4Q==</latexit>

Page 12: On the Global Convergence of Gradient-Based Learning in

Construction of a Lyapunov Function

✓0 Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y 0

<latexit sha1_base64="68xbnCbel7SSAr7uR4yMKfhNmDk=">AAAD23icfZNbb9MwFMfdhssItw6eEC/WqqGBupKicXmcBEgICVHEuk2aq8px3dRa4gT7pEpk5Yk3BI98OD4HXwA7jYAyBUs5+eec3zkndk7CLBYaguBHp+tdunzl6tY1//qNm7du97bvHOs0V4xPWBqn6jSkmsdC8gkIiPlppjhNwpifhOcvXfxkxZUWqTyCMuPThEZSLASjYF2z3jcS8wXs+STkkZAmoaBEUfkYkzDBAX6AP86Mk0XtKKvHRH9SYAjQfGaKqr6XFSE2Y79ByxotWlBbsi7tEy7nv/sRJaIlPJz1+sEwqBe+KEaN6KNmjWfb3WdknrI84RJYTLU+GwUZTA1VIFjMbeFc84yycxrxMyslTbiemvrcKrxrPXO8SJW9JODa+3eGoYm2b7gcYCscomulyyQc4DCpH9JM2kKO2sgs1i02+8PixdQImeXAJVu3X+QxhhS7T4PnQnEGcYl98orb3Sj+zpZ9n3FFIVWPDKEqSmhR2d1FmAyw0/9DhfyDWt2GrqiqjDNtAEtXlXGmDdCR7eRMG8CVbeFMGwA2ftQezmyYuDMGMGNVtfcpMt6QYWheW9DftcdKmRJ2HjBbUkUZ2N/Bt5M2+neuLorjJ8PRwfDph4P+4dtm5rbQfbSD9tAIPUeH6A0aowli6GfnXmen0/em3mfvi/d1jXY7Tc5dtLG8778A8thGWA==</latexit>

✓rIn 00 �rIm

<latexit sha1_base64="0XE11Q5hKoCxXFftdivKOrS6jjQ=">AAADmHicfZJtaxNBEMe3OR/q+ZTqO32zGCpVarhIfXgZUNEWxChNW+iGsLeZXJbe7h27k5Bw3Hfy0wi+0o/iXBLUWM6FG/4785uZm2XiPNUeo+j7ViO4cvXa9e0b4c1bt+/cbe7cO/HZ1CnoqyzN3FksPaTaQh81pnCWO5AmTuE0vnhTxU9n4LzO7DEuchgYmVg91koiuYbNI5HCGPdCEUOibWEkOj0vQ+4Oh5Y/5iI2PBIi5CtFnmcUMaEAO/oNC6eTCT4ZNltRO1oefll01qLF1qc33Gm8FKNMTQ1YVKn0/rwT5TgopEOtUqDCUw+5VBcygXOSVhrwg2I5dMl3yTPi48zRZ5EvvX9nFNJ4+sPJPidRIX6p/MLE+zw2y0uWWypUURuZ81WLzf44fj0otM2nCFat2o+nKceMV+/KR9qBwnTBQ/EWaBoHH6nspxycxMw9LYR0iZHzkqZLuNjnlf4fqu0flHQdOpOuLCpTB6hsVhaVqQN8Qp0qUweAoxaVqQOQ4sf14ZzConpjxKLnyvo+8xzWZBwX7wgMd+lZpXKa9oGriXRSIe1ySJvW+XevLouT5+3OQfvF54NW92i9c9vsIXvE9liHvWJd9oH1WJ8p9pV9Yz/Yz+BB0A3eB4crtLG1zrnPNk7w5RfJ3CyE</latexit>

+

<latexit sha1_base64="/x2wuocnqTXG6zs2Gp3nIoHjnYU=">AAADTXicfZLfa9RAEMe3qdoaf7X20ZfFoyB6HEmpPx4LWhBBvEKvLTRH2ezNpUuT3WV3c9wR8hf4av+uPvuH+CbibC6gZ4kDGb4785md7DCpzoV1UfR9LVi/c/fexub98MHDR4+fbG0/PbGqNBxGXOXKnKXMQi4kjJxwOZxpA6xIczhNr977/OkMjBVKHruFhnHBMimmgjOHoaNXF1u9aBA1Rm+LuBU90trwYjt4k0wULwuQjufM2vM40m5cMeMEz6EOk9KCZvyKZXCOUrIC7Lhq/rSmuxiZ0Kky+ElHm+jfFRUrbMHcZZ+i8IhtlF0UaZ+mRXNQWuJFnlqpnC9brPZ303fjSkhdOpB82X5a5tQp6odBJ8IAd/mChskHwNcY+IzXftFgmFPmZZUwkxVsXuPrMpr0qdf/Q4X8g6LuQmfM1JV3XQBXs7ryrguwGXbyrgsAgy286wIc5o+70xrTiZ+xc9XQ1N195hpaMk2rQwTDXRwr40bgPlB+yQzjDhcwxE2L/92r2+JkbxDvD14f7fcOPrU7t0mekefkBYnJW3JAPpIhGRFOgHwl38h1cBP8CH4Gv5ZosNbW7JAVW9/4DWrNFXg=</latexit>

+

<latexit sha1_base64="/x2wuocnqTXG6zs2Gp3nIoHjnYU=">AAADTXicfZLfa9RAEMe3qdoaf7X20ZfFoyB6HEmpPx4LWhBBvEKvLTRH2ezNpUuT3WV3c9wR8hf4av+uPvuH+CbibC6gZ4kDGb4785md7DCpzoV1UfR9LVi/c/fexub98MHDR4+fbG0/PbGqNBxGXOXKnKXMQi4kjJxwOZxpA6xIczhNr977/OkMjBVKHruFhnHBMimmgjOHoaNXF1u9aBA1Rm+LuBU90trwYjt4k0wULwuQjufM2vM40m5cMeMEz6EOk9KCZvyKZXCOUrIC7Lhq/rSmuxiZ0Kky+ElHm+jfFRUrbMHcZZ+i8IhtlF0UaZ+mRXNQWuJFnlqpnC9brPZ303fjSkhdOpB82X5a5tQp6odBJ8IAd/mChskHwNcY+IzXftFgmFPmZZUwkxVsXuPrMpr0qdf/Q4X8g6LuQmfM1JV3XQBXs7ryrguwGXbyrgsAgy286wIc5o+70xrTiZ+xc9XQ1N195hpaMk2rQwTDXRwr40bgPlB+yQzjDhcwxE2L/92r2+JkbxDvD14f7fcOPrU7t0mekefkBYnJW3JAPpIhGRFOgHwl38h1cBP8CH4Gv5ZosNbW7JAVW9/4DWrNFXg=</latexit>

=<latexit sha1_base64="BKvtt0en63nzwqHfqRarfg9y8g8=">AAADTXicfZLfa9RAEMe3qdoaf7X66MviURA5jqTUHy9CQQsiiFfotYXmKJu9uXRpsrvsTo47Qv4CX+3f1ef+Ib6JuJsL6FniQIbvznxmJztMqnNhMYpu1oL1O3fvbWzeDx88fPT4ydb202OrSsNhxFWuzGnKLORCwggF5nCqDbAizeEkvfzg8yczMFYoeYQLDeOCZVJMBWfoQofvz7d60SBqjN4WcSt6pLXh+XbwJpkoXhYgkefM2rM40jiumEHBc6jDpLSgGb9kGZw5KVkBdlw1f1rTHReZ0Kky7pNIm+jfFRUrbMHwok+d8IhtlF0UaZ+mRXNQWrqLPLVSOV+2WO2P03fjSkhdIki+bD8tc4qK+mHQiTDAMV/QMPkI7jUGvrhrv2owDJV5VSXMZAWb1+51GU361Ov/oUL+QZ3uQmfM1JV3XQBXs7ryrguwmevkXRcAxrXwrgtAlz/qTmuXTvyMEauhqbv7zDW0ZJpWBw4Md9xYGTfC7QPlF8wwjm4BQ7dp8b97dVsc7w7ivcHrw73e/ud25zbJc/KCvCQxeUv2yScyJCPCCZBv5Du5Cq6DH8HP4NcSDdbammdkxdY3fgOfLxWK</latexit>

KA =

<latexit sha1_base64="sgIBdebyyRTbPVw5umFyFMmrwIc=">AAADVXicfZJtaxNBEMe3l1rr+dBWX/pmMRREQriUqn1TqKggihihaSu9EPY2k+vS2wd290LCcp/Ct/q5xA8juHs50FjOgRv+O/ObndthMlUwY5Pk50bU2by1dXv7Tnz33v0HO7t7D8+MLDWFEZWF1BcZMVAwASPLbAEXSgPhWQHn2fXrkD+fgzZMilO7VDDmJBdsxiixPvTlw8S9qjA+xpPdbtJPasM3xaARXdTYcLIXvUinkpYchKUFMeZykCg7dkRbRguo4rQ0oAi9JjlceikIBzN29R9XeN9Hpngmtf+ExXX07wpHuOHEXvWwFwExtTJLnvVwxuuDVMJfFKi1ysWqxXp/OzsaOyZUaUHQVftZWWArcRgKnjIN1BZLHKdvwL9Gw0d/7ScFmlipn7mU6JyTReVfl+O0h4P+H8rEH9TrNnROdOWCawOonFcuuDbA5L5TcG0AaN8iuDbA+vxpe1r5dBpmbK0b6qq9z0JBQ2aZe+vBeN+PlVDN/D5gekU0odYvYuw3bfDvXt0UZwf9wWH/+efD7sn7Zue20WP0BD1FA/QSnaB3aIhGiCKOvqJv6Hv0I/rV2exsrdBoo6l5hNass/MbL1EWHw==</latexit>

KS =

<latexit sha1_base64="hdReeMhKZkYHrRzeUh3elYIZOY0=">AAADVXicfZJtaxNBEMe3l1rr+dBWX/pmMRREQriUqn1TKKggihixaSu9EPY2k+vS2wd290LCcp/Ct/q5xA8juHs50FjOgRv+O/ObndthMlUwY5Pk50bU2by1dXv7Tnz33v0HO7t7D8+MLDWFEZWF1BcZMVAwASPLbAEXSgPhWQHn2fWrkD+fgzZMilO7VDDmJBdsxiixPvTl/cR9rjA+xpPdbtJPasM3xaARXdTYcLIXvUinkpYchKUFMeZykCg7dkRbRguo4rQ0oAi9JjlceikIBzN29R9XeN9Hpngmtf+ExXX07wpHuOHEXvWwFwExtTJLnvVwxuuDVMJfFKi1ysWqxXp/OzsaOyZUaUHQVftZWWArcRgKnjIN1BZLHKevwb9Gwwd/7UcFmlipn7mU6JyTReVfl+O0h4P+H8rEH9TrNnROdOWCawOonFcuuDbA5L5TcG0AaN8iuDbA+vxpe1r5dBpmbK0b6qq9z0JBQ2aZe+PBeN+PlVDN/D5gekU0odYvYuw3bfDvXt0UZwf9wWH/+afD7sm7Zue20WP0BD1FA/QSnaC3aIhGiCKOvqJv6Hv0I/rV2exsrdBoo6l5hNass/MbZA0WMQ==</latexit>

Kr =

<latexit sha1_base64="MZ3K8MQ+jSmAURhrcETRkhSAF5w=">AAADVXicfZJtaxQxEMfTvVrr+tBWX/omeBREjmNX6sMboaCCKOIJvbbSPY5sbm4bunkgyR53hP0UvtXPJX4YwWRvQc+yDuzwz8xvMpthclUyY5Pk51bU276xc3P3Vnz7zt17e/sH90+NrDSFMZWl1Oc5MVAyAWPLbAnnSgPheQln+dXrkD9bgDZMihO7UjDhpBBsziixPvTlw9TpGuNXeLrfT4ZJY/i6SFvRR62NpgfR82wmacVBWFoSYy7SRNmJI9oyWkIdZ5UBRegVKeDCS0E4mIlr/rjGhz4yw3Op/ScsbqJ/VzjCDSf2coC9CIhplFnxfIBz3hykEv6iQG1ULtctNvvb+cuJY0JVFgRdt59XJbYSh6HgGdNAbbnCcfYG/Gs0fPTXflKgiZX6icuILjhZ1v51Bc4GOOj/oUz8Qb3uQhdE1y64LoDKRe2C6wJM4TsF1wWA9i2C6wKsz590p5VPZ2HG1rqRrrv7LBW0ZJ67tx6MD/1YCdXM7wOml0QTav0ixn7T0n/36ro4fTpMj4bPPh/1j9+3O7eLHqJH6DFK0Qt0jN6hERojijj6ir6h79GP6Fdvu7ezRqOttuYB2rDe3m++3xZQ</latexit>

“electric”

“magnetic”

“frictional”

✓ab

◆= �

✓Sxx/⌧x Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y �Syy/⌧y

◆✓ab

<latexit sha1_base64="pDcRubuQGqNl6b0ahxJ23xbLFME=">AAAEn3icfZPfb9MwEMedEmCEH1vhkRdDNTRQ17Vo/HhBmgQIhAQUsW5Dc1U5rptaS5xgO1UjK38XfwsPvMK/wSWNGNmWWcrlG9/n7pzLxU9CoU2//9NpXXGvXru+dsO7eev2nfWN9t0DHaeK8RGLw1gd+VTzUEg+MsKE/ChRnEZ+yA/9k9eF/3DBlRax3DdZwscRDaSYCUYNbE3azpCEfGa2POLzQEgbUaPEMvcwJtNpbCzxI0xzQmobfu4RLqf/YKJEMDeP8SvsbTel+zopQ5e4tDnewcTQdGJBPqo7s3yH6O8KSlVAec9Wh9iu0KzKczEKKetgdlowu/jwXnMfzrbh8i5MNjr9Xr9c+LwYVKKDqjWctFvPISdLIy4NC6nWx4N+YsaWKiNYyCFxqnlC2QkN+DFISSOux7b89jnehJ0pnsUKLmlwuft/hKWRhhPOuxhEgehS6Szyu9iPyoc4kZCooGqRy1WJen0zezm2Qiap4ZKtys/SEJsYF+OFp0JxZsIMe+QNh7dR/COk/ZxwRU2snlhCVRDRZQ5vF2DSxYW+DBXyFAXdhC6oym1hmgAWL3JbmCZAB1CpME0AV1CiME2AAf9+szsBNyl6bIwdqry5zjLhFen79i2A3ia0lTIlYB4wm1NFmYFf2oNJG5ydq/Pi4GlvsNt79mW3s/ehmrk1dB89RFtogF6gPfQeDdEIMeeH88v57fxxH7jv3E/ucIW2nCrmHqot99tfIrKL4Q==</latexit>

✓Sxx/⌧x � rIn 0

0 �(Syy/⌧y � rIm)

<latexit sha1_base64="HnJRN60PzVuJk4rDHr01L3rjZxo=">AAADznicfZNdb9MwFIbdho8RButAcMONRTXUoa6kaHxcTgIkQEIUsW6T5ipyXDe1ljiR7VSJrAgu+Yv8AX4HJ2n4KFOwlKM35zzHb+KcBGkktPG8752uc+XqtetbN9yb27du7/R275zoJFOMT1kSJeosoJpHQvKpESbiZ6niNA4ifhpcvKrqpyuutEjksSlSPotpKMVCMGog5fe+kogvzMAlAQ+FtDE1SuSli/Fn35IgxjmuY/mEGJr5Ni/xgXrnS/yoznuEAForyBwMmqaiThW/moqy6on3XcLl/LcFUSJcmn2/1/dGXr3wZTFuRB81a+Lvdp+TecKymEvDIqr1+dhLzcxSZQSLOGycaZ5SdkFDfg5S0pjrma2PqsR7kJnjRaLgkgbX2b87LI01POFyiEFUiK6VLuJgiIO4vklSCRtV1EZnvrbY9DeLlzMrZJoZLtnafpFF2CS4+hp4LhRnJiqwS15zeBvFP8C2H1OuqEnUY0uoCmOal/B2ISZDXOn/oUL+QUG3oSuqSluFNoAlq9JWoQ3QIThVoQ3gCiyq0AYYqB+3l1Mok+qMjbETVbb75ClvyCCwbwB09+BYKVMC5gGzJVWUGfgDXJi08b9zdVmcPB2ND0fPPh32j943M7eFHqCHaIDG6AU6Qm/RBE0RQz862517nfvOxFk5pfNljXY7Tc9dtLGcbz8BijE+eQ==</latexit>

Page 13: On the Global Convergence of Gradient-Based Learning in

Construction of a Lyapunov Function

✓rIn 00 �rIm

<latexit sha1_base64="0XE11Q5hKoCxXFftdivKOrS6jjQ=">AAADmHicfZJtaxNBEMe3OR/q+ZTqO32zGCpVarhIfXgZUNEWxChNW+iGsLeZXJbe7h27k5Bw3Hfy0wi+0o/iXBLUWM6FG/4785uZm2XiPNUeo+j7ViO4cvXa9e0b4c1bt+/cbe7cO/HZ1CnoqyzN3FksPaTaQh81pnCWO5AmTuE0vnhTxU9n4LzO7DEuchgYmVg91koiuYbNI5HCGPdCEUOibWEkOj0vQ+4Oh5Y/5iI2PBIi5CtFnmcUMaEAO/oNC6eTCT4ZNltRO1oefll01qLF1qc33Gm8FKNMTQ1YVKn0/rwT5TgopEOtUqDCUw+5VBcygXOSVhrwg2I5dMl3yTPi48zRZ5EvvX9nFNJ4+sPJPidRIX6p/MLE+zw2y0uWWypUURuZ81WLzf44fj0otM2nCFat2o+nKceMV+/KR9qBwnTBQ/EWaBoHH6nspxycxMw9LYR0iZHzkqZLuNjnlf4fqu0flHQdOpOuLCpTB6hsVhaVqQN8Qp0qUweAoxaVqQOQ4sf14ZzConpjxKLnyvo+8xzWZBwX7wgMd+lZpXKa9oGriXRSIe1ySJvW+XevLouT5+3OQfvF54NW92i9c9vsIXvE9liHvWJd9oH1WJ8p9pV9Yz/Yz+BB0A3eB4crtLG1zrnPNk7w5RfJ3CyE</latexit>

Kr =

<latexit sha1_base64="MZ3K8MQ+jSmAURhrcETRkhSAF5w=">AAADVXicfZJtaxQxEMfTvVrr+tBWX/omeBREjmNX6sMboaCCKOIJvbbSPY5sbm4bunkgyR53hP0UvtXPJX4YwWRvQc+yDuzwz8xvMpthclUyY5Pk51bU276xc3P3Vnz7zt17e/sH90+NrDSFMZWl1Oc5MVAyAWPLbAnnSgPheQln+dXrkD9bgDZMihO7UjDhpBBsziixPvTlw9TpGuNXeLrfT4ZJY/i6SFvRR62NpgfR82wmacVBWFoSYy7SRNmJI9oyWkIdZ5UBRegVKeDCS0E4mIlr/rjGhz4yw3Op/ScsbqJ/VzjCDSf2coC9CIhplFnxfIBz3hykEv6iQG1ULtctNvvb+cuJY0JVFgRdt59XJbYSh6HgGdNAbbnCcfYG/Gs0fPTXflKgiZX6icuILjhZ1v51Bc4GOOj/oUz8Qb3uQhdE1y64LoDKRe2C6wJM4TsF1wWA9i2C6wKsz590p5VPZ2HG1rqRrrv7LBW0ZJ67tx6MD/1YCdXM7wOml0QTav0ixn7T0n/36ro4fTpMj4bPPh/1j9+3O7eLHqJH6DFK0Qt0jN6hERojijj6ir6h79GP6Fdvu7ezRqOttuYB2rDe3m++3xZQ</latexit>

“electric”

z =

✓ab

<latexit sha1_base64="s0PR+MI2O2fP7rHh4SJaqG+omWo=">AAADmHicfZJtaxNBEMe3OR/q+ZTqO32zGApVQkiktr4RAipqQYzStIVuCLubyWXp3d6xuwmJy30nP43gK/0oziahGss5cMN/Z367czOMKFJlXbv9Y6sWXbt+4+b2rfj2nbv37td3HpzYfGok9GWe5uZMcAup0tB3yqVwVhjgmUjhVFy8DvnTGRircn3sFgUMMp5oNVaSOwwN60dMZPQrfUVZCmO3FzMBidI+486oeRlTSn0geMnY5UGUMQM9uoSYUcnEPR3WG+1We2n0quisRYOsrTfcqR2wUS6nGWgnU27teadduIHnximZAj48tVBwecETOEepeQZ24JdNl3QXIyM6zg1+2tFl9O8bnmcW/3DSpCgCYpfKLjLRpCJbHvJC40OB2rg5X5XYrO/GLwde6WLqQMtV+fE0pS6nYa50pAxIly5ozN4AdmPgIz77qQDDXW6eecZNkvF5id0llDVp0P9Dlf6Doq5CZ9yUPrgqQOaz0gdXBdgEKwVXBYDBEsFVAQ7zx9XpAtMszNg53zNldZ15AWtSCP8WwXgXx8qlUbgPVE644dLhLse4aZ1/9+qqOHne6uy3Xnzeb3SP1ju3TR6TJ2SPdMgh6ZL3pEf6RJJv5Dv5SX5Fj6Ju9C76sEJrW+s7D8mGRV9+A4iALXE=</latexit>

�(x,y) = �rS(x,y)

<latexit sha1_base64="+lwHgld/s2z32CH5Zo9ZEB8rgV0=">AAADdXicfZJti9NAEMf3Gh/O+NTTlyIs1tNTak3kfHgjHKgggljxendwKWWznabLZR/Y3ZSWkI/gp/GtfhA/iW/dTQNaj7iQ2X9mfrPDDJOqnBkbRT+3OsGFi5cub18Jr167fuNmd+fWkZGFpjCiMpf6JCUGciZgZJnN4URpIDzN4Tg9e+PjxwvQhklxaFcKxpxkgs0YJda5Jt2HyXDO9pKU42Uf+2v1CL/GTzT+sumcdHvRIKoPPi/iRvRQc4aTnc6LZCppwUFYmhNjTuNI2XFJtGU0hypMCgOK0DOSwamTgnAw47LuqMK7zjPFM6ndJyyuvX9nlIQbTuy8j53wiKmVWfG0j1Ne/0gl3EOe2shcrkts1rezV+OSCVVYEHRdflbk2Ersh4anTAO1+QqHyVtw3Wj46J79pEATK/XjMiE642RZue4ynPSx1/9DmfiDOt2GLoiuSm/aACoXVelNG2AyV8mbNgC0K+FNG2Bd/LA9rFw48TO2thzqqr3OUkFDpmn5zoHhrhsroZq5fcB0TjSh1i1q6DYt/nevzoujZ4N4f/D8837v4EOzc9voDrqH9lCMXqID9B4N0QhR9BV9Q9/Rj86v4G5wP3iwRjtbTc5ttHGCp78BteQfUQ==</latexit>

Let“potential function”

✓ab

◆= �

✓Sxx/⌧x Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y �Syy/⌧y

◆✓ab

<latexit sha1_base64="pDcRubuQGqNl6b0ahxJ23xbLFME=">AAAEn3icfZPfb9MwEMedEmCEH1vhkRdDNTRQ17Vo/HhBmgQIhAQUsW5Dc1U5rptaS5xgO1UjK38XfwsPvMK/wSWNGNmWWcrlG9/n7pzLxU9CoU2//9NpXXGvXru+dsO7eev2nfWN9t0DHaeK8RGLw1gd+VTzUEg+MsKE/ChRnEZ+yA/9k9eF/3DBlRax3DdZwscRDaSYCUYNbE3azpCEfGa2POLzQEgbUaPEMvcwJtNpbCzxI0xzQmobfu4RLqf/YKJEMDeP8SvsbTel+zopQ5e4tDnewcTQdGJBPqo7s3yH6O8KSlVAec9Wh9iu0KzKczEKKetgdlowu/jwXnMfzrbh8i5MNjr9Xr9c+LwYVKKDqjWctFvPISdLIy4NC6nWx4N+YsaWKiNYyCFxqnlC2QkN+DFISSOux7b89jnehJ0pnsUKLmlwuft/hKWRhhPOuxhEgehS6Szyu9iPyoc4kZCooGqRy1WJen0zezm2Qiap4ZKtys/SEJsYF+OFp0JxZsIMe+QNh7dR/COk/ZxwRU2snlhCVRDRZQ5vF2DSxYW+DBXyFAXdhC6oym1hmgAWL3JbmCZAB1CpME0AV1CiME2AAf9+szsBNyl6bIwdqry5zjLhFen79i2A3ia0lTIlYB4wm1NFmYFf2oNJG5ydq/Pi4GlvsNt79mW3s/ehmrk1dB89RFtogF6gPfQeDdEIMeeH88v57fxxH7jv3E/ucIW2nCrmHqot99tfIrKL4Q==</latexit>

�Krz = ��z

<latexit sha1_base64="MMkJmoKr9qTSAA3bGOrpt3X1w5U=">AAADbXicfVJtixMxEM51fTnXt57iJ0WD5VCkV3blfPkiHKggiljhendwW0o2nW7DbTYhyZbWsB/9NX7VH+Ov8C+YbBe0HutAhiczz8zDDJPKnGkTRT+3OsGFi5cub18Jr167fuNmd+fWkRalojCiIhfqJCUaclbAyDCTw4lUQHiaw3F69trnjxegNBPFoVlJGHOSFWzGKDEuNOk+2PswUclUGJukHH+p8Cu8lwznbNL8J91eNIhqw+dB3IAeamw42ek8d+1oyaEwNCdan8aRNGNLlGE0hypMSg2S0DOSwamDBeGgx7aepMK7LjLFM6HcKwyuo39XWMI1J2bexw54iq6RXvG0j1Nef4QsXCPP2qhcriU29c3s5diyQpYGCrqWn5U5NgL7ZeEpU0BNvsJh8gbcNAo+urafJChihHpiE6IyTpaVmy7DSR97/D8qK/5QHW6jLoiqrHdtBCoWlfWujaAzp+RdGwGUk/CujWBc/rA9LV068Ts2xg5V1a6zlNAw09S+dcRw162VUMXcPWA6J4pQ4w40dJcW/3tX58HR00G8P3j2eb938L65uW10Fz1Ej1GMXqAD9A4N0QhR9BV9Q9/Rj86v4E5wL7i/pna2mprbaMOCR78BFs0emQ==</latexit>

Page 14: On the Global Convergence of Gradient-Based Learning in

Construction of a Lyapunov Function

✓0 Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y 0

<latexit sha1_base64="68xbnCbel7SSAr7uR4yMKfhNmDk=">AAAD23icfZNbb9MwFMfdhssItw6eEC/WqqGBupKicXmcBEgICVHEuk2aq8px3dRa4gT7pEpk5Yk3BI98OD4HXwA7jYAyBUs5+eec3zkndk7CLBYaguBHp+tdunzl6tY1//qNm7du97bvHOs0V4xPWBqn6jSkmsdC8gkIiPlppjhNwpifhOcvXfxkxZUWqTyCMuPThEZSLASjYF2z3jcS8wXs+STkkZAmoaBEUfkYkzDBAX6AP86Mk0XtKKvHRH9SYAjQfGaKqr6XFSE2Y79ByxotWlBbsi7tEy7nv/sRJaIlPJz1+sEwqBe+KEaN6KNmjWfb3WdknrI84RJYTLU+GwUZTA1VIFjMbeFc84yycxrxMyslTbiemvrcKrxrPXO8SJW9JODa+3eGoYm2b7gcYCscomulyyQc4DCpH9JM2kKO2sgs1i02+8PixdQImeXAJVu3X+QxhhS7T4PnQnEGcYl98orb3Sj+zpZ9n3FFIVWPDKEqSmhR2d1FmAyw0/9DhfyDWt2GrqiqjDNtAEtXlXGmDdCR7eRMG8CVbeFMGwA2ftQezmyYuDMGMGNVtfcpMt6QYWheW9DftcdKmRJ2HjBbUkUZ2N/Bt5M2+neuLorjJ8PRwfDph4P+4dtm5rbQfbSD9tAIPUeH6A0aowli6GfnXmen0/em3mfvi/d1jXY7Tc5dtLG8778A8thGWA==</latexit>

KA =

<latexit sha1_base64="sgIBdebyyRTbPVw5umFyFMmrwIc=">AAADVXicfZJtaxNBEMe3l1rr+dBWX/pmMRREQriUqn1TqKggihihaSu9EPY2k+vS2wd290LCcp/Ct/q5xA8juHs50FjOgRv+O/ObndthMlUwY5Pk50bU2by1dXv7Tnz33v0HO7t7D8+MLDWFEZWF1BcZMVAwASPLbAEXSgPhWQHn2fXrkD+fgzZMilO7VDDmJBdsxiixPvTlw8S9qjA+xpPdbtJPasM3xaARXdTYcLIXvUinkpYchKUFMeZykCg7dkRbRguo4rQ0oAi9JjlceikIBzN29R9XeN9Hpngmtf+ExXX07wpHuOHEXvWwFwExtTJLnvVwxuuDVMJfFKi1ysWqxXp/OzsaOyZUaUHQVftZWWArcRgKnjIN1BZLHKdvwL9Gw0d/7ScFmlipn7mU6JyTReVfl+O0h4P+H8rEH9TrNnROdOWCawOonFcuuDbA5L5TcG0AaN8iuDbA+vxpe1r5dBpmbK0b6qq9z0JBQ2aZe+vBeN+PlVDN/D5gekU0odYvYuw3bfDvXt0UZwf9wWH/+efD7sn7Zue20WP0BD1FA/QSnaB3aIhGiCKOvqJv6Hv0I/rV2exsrdBoo6l5hNass/MbL1EWHw==</latexit>

“magnetic”

�KA = K|A

<latexit sha1_base64="qCtZa9eceo/X6Op0lqn9hP1jOQE=">AAADZHicfZL/atRAEMe3F3/UWPVq0X8EWTwKIueRSFv9R6ioIIp4Qq8tNOex2ZtLl252w+7muGPNy/ivvpAv4HO4mwvoWeJAhm9mPrPDDJMWnGkTRT83OsGVq9eub94Ib27dun2nu333WMtSURhRyaU6TYkGzgSMDDMcTgsFJE85nKQXr33+ZA5KMymOzLKAcU4ywWaMEuNCk+79px8mr/BL7PwXmzBhQFHCq0m3Fw2i2vBlETeihxobTrY7B8lU0jIHYSgnWp/FUWHGlijDKIcqTEoNBaEXJIMzJwXJQY9tPUCFd11kimdSuU8YXEf/rrAk1zkx533shEd0rfQyT/s4zesfWQj3kKfWKherFuv9zezF2DJRlAYEXbWflRwbif2O8JQpoIYvcZi8ATeNgo/u2U8FKGKkemITorKcLCo3XYaTPvb6fygTf1Cn29A5UZX1rg2gcl5Z79oAnblO3rUBoFwL79oA4/JH7enCpRO/Y2PsUFXtfRYFNGSa2rcODHfdWglVzN0DpudEEepuTYfu0uJ/7+qyOH42iPcG+5/3eofvm5vbRA/QI/QYxeg5OkTv0BCNEEVf0Tf0Hf3o/Aq2gp3g3grtbDQ1O2jNgoe/AXM+Gtc=</latexit>

z|KAz = �z|K|Az = 0

<latexit sha1_base64="wxiX1jnIbS2YyCHnvp09xNZTuaQ=">AAADrXicfZJtaxNBEMe3OR/q+ZTqS98shoJIjHdSW98IFRVEESM0baGXhr3N5Lr09oHdvZC43MfzQ/gZfKvv3bscmFjOgR3+u/ObGXaYVOXM2Cj6sdUJrl2/cXP7Vnj7zt1797s7D46NLDSFEZW51KcpMZAzASPLbA6nSgPhaQ4n6eXbKn4yB22YFEd2qWDMSSbYjFFi/dOkO0mm0rok5fhbee4SJixoSvLy0+TNWgS/xs9wO7l+3cyKJt1eNIhqw1dF3Igeamw42ens+xK04CAszYkxZ3Gk7NgRbRnNoQyTwoAi9JJkcOalIBzM2NWTKPGuf5nimdT+CIvr1/UMR7jhxF70sRcVYmplljzt45TXF6mEL1RRG5mLVYvN/nb2auyYUIUFQVftZ0WOrcTVsPGUaaA2X+IweQf+Nxo++7JfFGhipX7qEqIzThal/12Gkz6u9P9QJv6iXrehc6JLV7k2gMp56SrXBpjMd6pcGwDat6hcG2B9/Kg9rHw4qWZsrRvqsr3PQkFDpql778Fw14+VUM38PmB6QTShfvVM6Dct/nevrorjF4N4b/Dy617v8GOzc9voEXqMnqAYHaBD9AEN0QhR9B39RL/Q7+B5MAqS4HyFdraanIdow4LsD0baOQc=</latexit>

KAz?z

<latexit sha1_base64="neBuDMkq07ZZAGthAHPYolPSkog=">AAADbXicfZJta9RAEMe3Fx9qfLpWfKXo4lGUchyJ1OrLigqiiCf02kJzHLt7c+nSJBt2J8edIS/9NL7VD+On8Cu4mwvYs8SBDP/M/GaHGYbniTQYBL82Ot6Vq9eub97wb966fedud2v7yKhCCxgJlSh9wpmBRGYwQokJnOQaWMoTOObnb1z+eA7aSJUd4jKHccriTM6kYGhDk+7jj5PX0VRhGfGUfq1oxBXSC4FJtxcMgtroZRE2okcaG062Ovu2XBQpZCgSZsxpGOQ4LplGKRKo/KgwkDNxzmI4tTJjKZhxWU9S0R0bmdKZ0vbLkNbRixUlS03K8KxPrXCIqZVZprxPeVr/qDyzDzlqrXKxarHeH2evxqXM8gIhE6v2syKhqKhbFp1KDQKTJfWjt2Cn0fDJPvs5B81Q6d0yYjpO2aKy08U06lOn/4fK7C9qdRs6Z7oqnWsDhJpXpXNtgIltJ+faANC2hXNtANr8YXs6t+nI7RixHOqqvc8ih4bkvHxnQX/HrpUJLe09UHHGNBNoD9S3lxb+e1eXxdHzQbg3ePFlr3fwobm5TfKAPCHPSEhekgPyngzJiAjyjXwnP8jPzm/vvvfQe7RCOxtNzT2yZt7TP/C6Hzk=</latexit>

anti-symmetric

always perpendicular to the velocity

✓ab

◆= �

✓Sxx/⌧x Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y �Syy/⌧y

◆✓ab

<latexit sha1_base64="pDcRubuQGqNl6b0ahxJ23xbLFME=">AAAEn3icfZPfb9MwEMedEmCEH1vhkRdDNTRQ17Vo/HhBmgQIhAQUsW5Dc1U5rptaS5xgO1UjK38XfwsPvMK/wSWNGNmWWcrlG9/n7pzLxU9CoU2//9NpXXGvXru+dsO7eev2nfWN9t0DHaeK8RGLw1gd+VTzUEg+MsKE/ChRnEZ+yA/9k9eF/3DBlRax3DdZwscRDaSYCUYNbE3azpCEfGa2POLzQEgbUaPEMvcwJtNpbCzxI0xzQmobfu4RLqf/YKJEMDeP8SvsbTel+zopQ5e4tDnewcTQdGJBPqo7s3yH6O8KSlVAec9Wh9iu0KzKczEKKetgdlowu/jwXnMfzrbh8i5MNjr9Xr9c+LwYVKKDqjWctFvPISdLIy4NC6nWx4N+YsaWKiNYyCFxqnlC2QkN+DFISSOux7b89jnehJ0pnsUKLmlwuft/hKWRhhPOuxhEgehS6Szyu9iPyoc4kZCooGqRy1WJen0zezm2Qiap4ZKtys/SEJsYF+OFp0JxZsIMe+QNh7dR/COk/ZxwRU2snlhCVRDRZQ5vF2DSxYW+DBXyFAXdhC6oym1hmgAWL3JbmCZAB1CpME0AV1CiME2AAf9+szsBNyl6bIwdqry5zjLhFen79i2A3ia0lTIlYB4wm1NFmYFf2oNJG5ydq/Pi4GlvsNt79mW3s/ehmrk1dB89RFtogF6gPfQeDdEIMeeH88v57fxxH7jv3E/ucIW2nCrmHqot99tfIrKL4Q==</latexit>

Page 15: On the Global Convergence of Gradient-Based Learning in

Construction of a Lyapunov Function

z = � �z|{z}electric

� KAz| {z }magnetic

� KS z|{z}frictional

<latexit sha1_base64="l4OG2I3/r/1Kd4hzAoLjUK1z48c=">AAAD7nicfVPfaxNBEN7Laa3xV6qPviyGgkgMibTVF6GiglTEiE1b6IWwt5kkS/f2jt29kLjcv+Gb6KP/kv9NZ5PDmrbnwA3fzXwz3+7cXJxJYWyn8yeohTdubtzavF2/c/fe/QeNrYdHJs01hz5PZapPYmZACgV9K6yEk0wDS2IJx/HZW58/noE2IlWHdpHBIGETJcaCM4uhYeNXNBql1kVxQr8V9DV9TqNcjUDHmnFwUW8qhmWyQGBhbh1I4FYLXhSX2B+Hb2h00e1vgZcEe23B12sLxtjdH49JjDWanXZnafQq6JagSUrrDbdqe9iT5wkoyyUz5rTbyezAMY1HkFDUo9xAxvgZm8ApQsUSMAO3HGVBtzEyouNU46MsXUb/rXAsMQmz0xZF4ClmicwiiVs0TpYvaaawkWetVc5XEuv6dvxq4ITKcguKr+THuaQ2pf5r0ZHQOGy5oPXoHeBtNHzCtp8z0Mym+pmLmJ4kbF7g7SY0alGP/0cV6oKKuIo6Y7pw3lUReDornHdVBDNBJe+qCKBRwrsqgsX8YXU6w3TkZ2yt6+miWmeeQcmMY/ceifVtHCvjWuA+UD5luIoW/5A6blr38l5dBUcv2t2d9u6Xneb+Qblzm+QxeUKeki55SfbJB9IjfcKDjaAV7AZ7YRZ+D3+EP1fUWlDWPCJrFv4+B8GwUgA=</latexit>

L(z) =1

2|z|2 + �(z)

<latexit sha1_base64="5cl1MoJ+3yX5oH4uqB/kMpjAC+4=">AAADgXicfZJti9NAEMf3Gh/O+NTTl75ZLAd3WkpSzwcQ4UAFEcUK17uDSy2b7SZdLptddjeldS8fxE/jW/0IfhsnaUDrEQcy/LPzmxlmmFhl3Ngg+LXV8a5cvXZ9+4Z/89btO3e7O/eOjSw0ZWMqM6lPY2JYxnM2ttxm7FRpRkScsZP4/HUVP1kwbbjMj+xKsYkgac4TTomFp2n3yYe9KBb46z5+haNEE+rC0g3Li2gmrasj5cWXIX6Mo9GcN+i02wsGQW34sggb0UONjaY7nWdQjxaC5ZZmxJizMFB24oi2nGas9KPCMEXoOUnZGcicCGYmrp6uxLvwMsOJ1PDlFtevf2c4Iowgdt7HICrE1MqsRNzHsah/pMqhUEVtZC7XLTb72+TFxPFcFZbldN0+KTJsJa4WiGdcM2qzFfajNwym0ewjlP2kmCZW6kcuIjoVZFnCdCmO+rjS/0N5/gcF3YYuiC5d5doAKhelq1wbYFLoVLk2gGloUbk2wEL8qD2sIBxVO7bWjXTZ3mepWEPGsXsLoL8LayVUc7gHTOcE7tDC0fpwaeG/d3VZHA8H4cHg6eeD3uH75ua20QP0EO2hED1Hh+gdGqExougb+o5+oJ+e5+17gTdco52tJuc+2jDv5W/e3SSG</latexit>

L = �z|KS z

<latexit sha1_base64="7CO6+Vzx6BXKIfXMVpPzk69qDXQ=">AAADf3icfZJti9NAEMe3jQ9nfLievvTNYjkUqSWR8+mFcKCC+IAVr3cHl1o222luuc1u2N2U1iWfw0/jW/0Mfht304BXjziQ4Z+Z3+www6QFZ9pE0e9ON7h0+crVrWvh9Rs3b233dm4falkqCmMquVTHKdHAmYCxYYbDcaGA5CmHo/Tslc8fLUBpJsWBWRUwyUkm2JxRYlxo2ouTmTT4A36JH3llkzTH36qvNmHCgKKEV++n9kt1Ljft9aNhVBu+KOJG9FFjo+lO96krp2UOwlBOtD6Jo8JMLFGGUQ5VmJQaCkLPSAYnTgqSg57YerYK77rIDM+lcp8wuI6er7Ak1zkxpwPshEd0rfQqTwc4zesfWQj3kKc2KpfrFpv9zfz5xDJRlAYEXbeflxwbif368IwpoIavcJi8BjeNgo/u2U8FKGKkemgTorKcLCs3XYaTAfb6fygTf1Gn29AFUZX1rg2gclFZ79oAnblO3rUBoFwL79oA4/IH7enCpRO/Y2PsSFXtfZYFNGSa2jcODHfdWglVzN0DpqdEEepuT4fu0uJ/7+qiOHw8jPeGTz7v9fffNTe3he6ie+gBitEztI/eohEaI4q+ox/oJ/oVdIL7wTCI1mi309TcQRsWvPgDkTsl1A==</latexit>

Energy conservation as a Lyapunov function:

✓ab

◆= �

✓Sxx/⌧x Sxy/

p⌧x⌧y

�Syx/p⌧x⌧y �Syy/⌧y

◆✓ab

<latexit sha1_base64="pDcRubuQGqNl6b0ahxJ23xbLFME=">AAAEn3icfZPfb9MwEMedEmCEH1vhkRdDNTRQ17Vo/HhBmgQIhAQUsW5Dc1U5rptaS5xgO1UjK38XfwsPvMK/wSWNGNmWWcrlG9/n7pzLxU9CoU2//9NpXXGvXru+dsO7eev2nfWN9t0DHaeK8RGLw1gd+VTzUEg+MsKE/ChRnEZ+yA/9k9eF/3DBlRax3DdZwscRDaSYCUYNbE3azpCEfGa2POLzQEgbUaPEMvcwJtNpbCzxI0xzQmobfu4RLqf/YKJEMDeP8SvsbTel+zopQ5e4tDnewcTQdGJBPqo7s3yH6O8KSlVAec9Wh9iu0KzKczEKKetgdlowu/jwXnMfzrbh8i5MNjr9Xr9c+LwYVKKDqjWctFvPISdLIy4NC6nWx4N+YsaWKiNYyCFxqnlC2QkN+DFISSOux7b89jnehJ0pnsUKLmlwuft/hKWRhhPOuxhEgehS6Szyu9iPyoc4kZCooGqRy1WJen0zezm2Qiap4ZKtys/SEJsYF+OFp0JxZsIMe+QNh7dR/COk/ZxwRU2snlhCVRDRZQ5vF2DSxYW+DBXyFAXdhC6oym1hmgAWL3JbmCZAB1CpME0AV1CiME2AAf9+szsBNyl6bIwdqry5zjLhFen79i2A3ia0lTIlYB4wm1NFmYFf2oNJG5ydq/Pi4GlvsNt79mW3s/ehmrk1dB89RFtogF6gPfQeDdEIMeeH88v57fxxH7jv3E/ucIW2nCrmHqot99tfIrKL4Q==</latexit>

Page 16: On the Global Convergence of Gradient-Based Learning in

L(z) =1

2|z|2 + �(z)

<latexit sha1_base64="5cl1MoJ+3yX5oH4uqB/kMpjAC+4=">AAADgXicfZJti9NAEMf3Gh/O+NTTl75ZLAd3WkpSzwcQ4UAFEcUK17uDSy2b7SZdLptddjeldS8fxE/jW/0IfhsnaUDrEQcy/LPzmxlmmFhl3Ngg+LXV8a5cvXZ9+4Z/89btO3e7O/eOjSw0ZWMqM6lPY2JYxnM2ttxm7FRpRkScsZP4/HUVP1kwbbjMj+xKsYkgac4TTomFp2n3yYe9KBb46z5+haNEE+rC0g3Li2gmrasj5cWXIX6Mo9GcN+i02wsGQW34sggb0UONjaY7nWdQjxaC5ZZmxJizMFB24oi2nGas9KPCMEXoOUnZGcicCGYmrp6uxLvwMsOJ1PDlFtevf2c4Iowgdt7HICrE1MqsRNzHsah/pMqhUEVtZC7XLTb72+TFxPFcFZbldN0+KTJsJa4WiGdcM2qzFfajNwym0ewjlP2kmCZW6kcuIjoVZFnCdCmO+rjS/0N5/gcF3YYuiC5d5doAKhelq1wbYFLoVLk2gGloUbk2wEL8qD2sIBxVO7bWjXTZ3mepWEPGsXsLoL8LayVUc7gHTOcE7tDC0fpwaeG/d3VZHA8H4cHg6eeD3uH75ua20QP0EO2hED1Hh+gdGqExougb+o5+oJ+e5+17gTdco52tJuc+2jDv5W/e3SSG</latexit>

L = �z|KS z

<latexit sha1_base64="7CO6+Vzx6BXKIfXMVpPzk69qDXQ=">AAADf3icfZJti9NAEMe3jQ9nfLievvTNYjkUqSWR8+mFcKCC+IAVr3cHl1o222luuc1u2N2U1iWfw0/jW/0Mfht304BXjziQ4Z+Z3+www6QFZ9pE0e9ON7h0+crVrWvh9Rs3b233dm4falkqCmMquVTHKdHAmYCxYYbDcaGA5CmHo/Tslc8fLUBpJsWBWRUwyUkm2JxRYlxo2ouTmTT4A36JH3llkzTH36qvNmHCgKKEV++n9kt1Ljft9aNhVBu+KOJG9FFjo+lO96krp2UOwlBOtD6Jo8JMLFGGUQ5VmJQaCkLPSAYnTgqSg57YerYK77rIDM+lcp8wuI6er7Ak1zkxpwPshEd0rfQqTwc4zesfWQj3kKc2KpfrFpv9zfz5xDJRlAYEXbeflxwbif368IwpoIavcJi8BjeNgo/u2U8FKGKkemgTorKcLCs3XYaTAfb6fygTf1Gn29AFUZX1rg2gclFZ79oAnblO3rUBoFwL79oA4/IH7enCpRO/Y2PsSFXtfZYFNGSa2jcODHfdWglVzN0DpqdEEepuT4fu0uJ/7+qiOHw8jPeGTz7v9fffNTe3he6ie+gBitEztI/eohEaI4q+ox/oJ/oVdIL7wTCI1mi309TcQRsWvPgDkTsl1A==</latexit>

Energy conservation as a Lyapunov function:

KS � 0

<latexit sha1_base64="D0Hebtuh8p/tUvnGJggIoL+YpKo=">AAADV3icfZJti9NAEMf3Uj1rfLrqS98slgORUtLjfHh5oIIoYsXr3UFTymY7zS2X7C67m9IS8jF8q5/rPo3OpgGtRxzI8N+Z3+xkh0l0JqyLouu9oHPr9v6d7t3w3v0HDx8d9B6fWVUYDhOuMmUuEmYhExImTrgMLrQBlicZnCdXb33+fAXGCiVP3UbDLGepFEvBmcPQ9NP8G41twTmN5gf9aBjVRm+KUSP6pLHxvBe8iheKFzlIxzNm7XQUaTcrmXGCZ1CFcWFBM37FUpiilCwHOyvrf67oIUYWdKkMftLROvp3RclymzN3OaAoPGJrZTd5MqBJXh+UlniRp3Yq19sWu/3d8s2sFFIXDiTftl8WGXWK+rHQhTDAXbahYfwO8DUGPuO1XzQY5pR5UcbMpDlbV/i6lMYD6vX/UCH/oKjb0BUzVeldG8DVqiq9awNsip28awPAYAvv2gCH+dP2tMZ07GfsXDk2VXuftYaGTJLyPYLhIY6VcSNwHyi/ZIZxh6sY4qaN/t2rm+LsaDg6Hr78etw/+djsXJc8Jc/IczIir8kJ+UDGZEI4UeQ7+UF+BtfBr85+p7tFg72m5gnZsU7vN1EfFyo=</latexit>

KS =

<latexit sha1_base64="hdReeMhKZkYHrRzeUh3elYIZOY0=">AAADVXicfZJtaxNBEMe3l1rr+dBWX/pmMRREQriUqn1TKKggihixaSu9EPY2k+vS2wd290LCcp/Ct/q5xA8juHs50FjOgRv+O/ObndthMlUwY5Pk50bU2by1dXv7Tnz33v0HO7t7D8+MLDWFEZWF1BcZMVAwASPLbAEXSgPhWQHn2fWrkD+fgzZMilO7VDDmJBdsxiixPvTl/cR9rjA+xpPdbtJPasM3xaARXdTYcLIXvUinkpYchKUFMeZykCg7dkRbRguo4rQ0oAi9JjlceikIBzN29R9XeN9Hpngmtf+ExXX07wpHuOHEXvWwFwExtTJLnvVwxuuDVMJfFKi1ysWqxXp/OzsaOyZUaUHQVftZWWArcRgKnjIN1BZLHKevwb9Gwwd/7UcFmlipn7mU6JyTReVfl+O0h4P+H8rEH9TrNnROdOWCawOonFcuuDbA5L5TcG0AaN8iuDbA+vxpe1r5dBpmbK0b6qq9z0JBQ2aZe+PBeN+PlVDN/D5gekU0odYvYuw3bfDvXt0UZwf9wWH/+afD7sm7Zue20WP0BD1FA/QSnaC3aIhGiCKOvqJv6Hv0I/rV2exsrdBoo6l5hNass/MbZA0WMQ==</latexit>

“frictional”

✓Sxx/⌧x � rIn 0

0 �(Syy/⌧y � rIm)

<latexit sha1_base64="HnJRN60PzVuJk4rDHr01L3rjZxo=">AAADznicfZNdb9MwFIbdho8RButAcMONRTXUoa6kaHxcTgIkQEIUsW6T5ipyXDe1ljiR7VSJrAgu+Yv8AX4HJ2n4KFOwlKM35zzHb+KcBGkktPG8752uc+XqtetbN9yb27du7/R275zoJFOMT1kSJeosoJpHQvKpESbiZ6niNA4ifhpcvKrqpyuutEjksSlSPotpKMVCMGog5fe+kogvzMAlAQ+FtDE1SuSli/Fn35IgxjmuY/mEGJr5Ni/xgXrnS/yoznuEAForyBwMmqaiThW/moqy6on3XcLl/LcFUSJcmn2/1/dGXr3wZTFuRB81a+Lvdp+TecKymEvDIqr1+dhLzcxSZQSLOGycaZ5SdkFDfg5S0pjrma2PqsR7kJnjRaLgkgbX2b87LI01POFyiEFUiK6VLuJgiIO4vklSCRtV1EZnvrbY9DeLlzMrZJoZLtnafpFF2CS4+hp4LhRnJiqwS15zeBvFP8C2H1OuqEnUY0uoCmOal/B2ISZDXOn/oUL+QUG3oSuqSluFNoAlq9JWoQ3QIThVoQ3gCiyq0AYYqB+3l1Mok+qMjbETVbb75ClvyCCwbwB09+BYKVMC5gGzJVWUGfgDXJi08b9zdVmcPB2ND0fPPh32j943M7eFHqCHaIDG6AU6Qm/RBE0RQz862517nfvOxFk5pfNljXY7Tc9dtLGcbz8BijE+eQ==</latexit>

If

then all bounded trajectories converge.

and steady states exists,⌧�1x �inf(Sxx) > ⌧�1

y �sup(Syy)

<latexit sha1_base64="r94Pw7Tshrjs/eIu+SEpJMr4MAA=">AAADqXicfZLdbtMwFMe9ho8Rvjq45MaimjSgVAkaG1doEiAhJEQn1q2iKZHjOpm1xLFsp2pk5eF4DJ6AW3gDTtII6KZwpBz94/M7/stHJ5Ip18bzvm/1nGvXb9zcvuXevnP33v3+zoNTnReKsgnN01xNI6JZygWbGG5SNpWKkSxK2Vl08aauny2Z0jwXJ6aUbJ6RRPCYU2LgKOzPAkOK0K6qr/a5XwUpdC5IaAMu4mrvM4gow6smVU8wfo3XeHkZ14X8g5dNAjzsD7yR1wS+KvxWDFAb43CndxAsclpkTBiaEq1nvifN3BJlOE1Z5QaFZpLQC5KwGUhBMqbntplChXfhZIHjXMEnDG5O/+2wJNMZMedDDKJGdKN0mUVDHGXNTy4FXFRTG52rtcWmv4lfzS0XsjBM0LV9XKTY5LgeNF5wxahJS+wGbxm8RrGPcO0nyRQxuXpqA6KSjKwqeF2CgyGu9f9QLv6ioLvQJVGVrVMXQPNlZevUBegEnOrUBTAFFnXqAgzUT7rLEspBPWNj7FhV3T4ryVoyiuw7AN1dGCuhisM+YHpOFKEGltuFTfMv79VVcfpi5O+PXh7vD44+tDu3jR6hx2gP+egQHaH3aIwmiKJv6Af6iX45z5xjZ+p8WaO9rbbnIdoIh/4GRSE2kA==</latexit>

Construction of a Lyapunov Function

Page 17: On the Global Convergence of Gradient-Based Learning in

Summary

minx

maxy

S(x,y)

<latexit sha1_base64="NsgQ8OVYc7bq9u0xnxC3odXMzq8=">AAADd3icfZJbixMxFMezHS/reOvqow8Gy0pXSpmR9fK4oIIIYsXt7sJOKZk0nQ07uZCkpSXMZ/DT+Kqfw4/imyfTAa3LeCCHf875nRxyOLkuuXVJ8nOnE127fuPm7q349p279+539x6cWLUwlI2pKpU5y4llJZds7Lgr2Zk2jIi8ZKf55ZuQP10yY7mSx26t2USQQvI5p8RBaNo9yASXU5/lAq+qTJDVRq8r/KVfBwe4vh/gabeXDJPa8FWRNqKHGhtN9zovs5miC8GkoyWx9jxNtJt4YhynJavibGGZJvSSFOwcpCSC2Ymv/1ThfYjM8FwZONLhOvp3hSfCCuIuBhhEQGyt7FrkA5yL+qK0hIcCtVW52rTY7u/mryeeS71wTNJN+/mixE7hMDY844ZRV65xnL1l8BvDPsKznzQzxCnzzGfEFDC9Cn5X4GyAg/4fyuUfFHQbuiSm8sG1AVQtKx9cG2AL6BRcG8AMtAiuDXCQP25Pa0hnYcbO+ZGp2vusNGvIPPfvAIz3YayEGg77gOkFMYQ6WNUYNi39d6+uipPnw/Rw+OLzYe/oQ7Nzu+gReoL6KEWv0BF6j0ZojCj6ir6h7+hH51f0OHoa9TdoZ6epeYi2LEp/A6dFIj8=</latexit>

(⌧xx = �rxS(x,y)

⌧yy = ryS(x,y)

<latexit sha1_base64="lHH0gyrUqk1i4hWAuwiLq5PAO8s=">AAAD0nicfVNda9swFFXifXTeV7o9bXsRC4VupCEZ3cfLoLANxmAsY0lbqEKQFMUVtWUjySGe8MPYHvcH9xf2K3blGNq0eAJfHd97zj1IvmZZLI0dDP602sG16zdubt0Kb9+5e+9+Z/vBoUlzzcWEp3Gqjxk1IpZKTKy0sTjOtKAJi8URO3vn60dLoY1M1dgWmZgmNFJyITm1kJp1fhEmIqkchx6mDIml+cytSjJPrSMswasSv8V7RFEW01md+bZb7T3st+IZIbWsOJcVXnZRVVxWhUSoeW0763QH/UG18FUwrEEX1Ws0226/AiueJ0JZHlNjToaDzE4d1VbyWMA5ciMyys9oJE4AKpoIM3XVdZV4BzJzvEg1PMriKntR4WhiEmpPexiAp5gKmSJhPcyS6iXNFDTyrA3lam2x6W8Xb6ZOqiy3QvG1/SKPsU2x/yJ4LrXgNi5wSN4LOI0Wn6Htl0xoalP93BGqo4SuSjhdhEkPe/w/qlTnVMBN1CXVpfOhicDTZel8aCKYCJx8aCIIDRY+NBEs1MfN5QzKxN+xtW6ky2afVSZqJmPuAxDDHbhWyrWEecD8lGrKLfwFIUza8PJcXQWHL/rD/f7Lr/vdg0/1zG2hJ+gp2kVD9BodoI9ohCaIo7+tTutR63EwDr4HP4Kfa2q7VWseoo0V/P4HCSZDIQ==</latexit>

Continuous-time system:

1. System converges when

2. Boundedness, Discrete time approximation

3. Gradient projection, Rectified gradient descent-ascent

⌧�1x �inf(Sxx) > ⌧�1

y �sup(Syy)

<latexit sha1_base64="r94Pw7Tshrjs/eIu+SEpJMr4MAA=">AAADqXicfZLdbtMwFMe9ho8Rvjq45MaimjSgVAkaG1doEiAhJEQn1q2iKZHjOpm1xLFsp2pk5eF4DJ6AW3gDTtII6KZwpBz94/M7/stHJ5Ip18bzvm/1nGvXb9zcvuXevnP33v3+zoNTnReKsgnN01xNI6JZygWbGG5SNpWKkSxK2Vl08aauny2Z0jwXJ6aUbJ6RRPCYU2LgKOzPAkOK0K6qr/a5XwUpdC5IaAMu4mrvM4gow6smVU8wfo3XeHkZ14X8g5dNAjzsD7yR1wS+KvxWDFAb43CndxAsclpkTBiaEq1nvifN3BJlOE1Z5QaFZpLQC5KwGUhBMqbntplChXfhZIHjXMEnDG5O/+2wJNMZMedDDKJGdKN0mUVDHGXNTy4FXFRTG52rtcWmv4lfzS0XsjBM0LV9XKTY5LgeNF5wxahJS+wGbxm8RrGPcO0nyRQxuXpqA6KSjKwqeF2CgyGu9f9QLv6ioLvQJVGVrVMXQPNlZevUBegEnOrUBTAFFnXqAgzUT7rLEspBPWNj7FhV3T4ryVoyiuw7AN1dGCuhisM+YHpOFKEGltuFTfMv79VVcfpi5O+PXh7vD44+tDu3jR6hx2gP+egQHaH3aIwmiKJv6Af6iX45z5xjZ+p8WaO9rbbnIdoIh/4GRSE2kA==</latexit>