97
Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett 1

Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning with Kernel Losses

Aldo Pacchiano

UC Berkeley

Joint work with Niladri Chatterji and Peter Bartlett 1

Page 2: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Talk Overview

• Intro to Online Learning

• Linear Bandits

• Kernel Bandits

2

Page 3: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

3

Page 4: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

3

Learner Adversaryt = 1, · · · , n

Page 5: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action at 2 A

3

Learner Adversaryt = 1, · · · , n

Page 6: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action at 2 A

Adversary reveals loss (or reward) `t 2 W

3

Learner Adversaryt = 1, · · · , n

Page 7: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action at 2 A

Adversary reveals loss (or reward) Can be i.i.d or

adversarial`t 2 W

3

Learner Adversaryt = 1, · · · , n

Page 8: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action at 2 A

Adversary reveals loss (or reward) Can be i.i.d or

adversarial`t 2 W

3

Learner Adversary

nX

t=1

`t(at)

t = 1, · · · , n

Page 9: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action at 2 A

Adversary reveals loss (or reward) Can be i.i.d or

adversarial`t 2 W

3

Learner Adversary

R(n) =nX

t=1

`t(at) � mina⇤2A

nX

t=1

`t(at)

t = 1, · · · , n

Page 10: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Online Learning

Learner chooses an action

The learner’s objective is to minimize Regret

at 2 A

Adversary reveals loss (or reward) Can be i.i.d or

adversarial`t 2 W

3

Learner Adversary

R(n) =nX

t=1

`t(at) � mina⇤2A

nX

t=1

`t(at)

t = 1, · · · , n

Page 11: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Full information vs Bandit feedback

4

Page 12: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Full information vs Bandit feedback

Full Information: Learner gets to sees all of

`t(·)

4

Page 13: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Full information vs Bandit feedback

Full Information:

Bandit Feedback:

Learner gets to sees all of

Learner only sees the value

`t(·)

`t(at)

4

Page 14: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Full information vs Bandit feedback

Full Information:

Bandit Feedback:

Learner gets to sees all of

Learner only sees the value

`t(·)

`t(at)

4

Page 15: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Multi Armed Bandits

5

P1

µ1

P2 P3

µ2 µ3

Page 16: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Multi Armed Bandits

at 2 {1, · · · ,K}

5

Learner choosesP1

µ1

P2 P3

µ2 µ3

Page 17: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Multi Armed Bandits

Xat ⇠ Pat

at 2 {1, · · · ,K}

5

Learner choosesP1

µ1

Gets reward

P2 P3

µ2 µ3

Page 18: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Multi Armed Bandits

Xat ⇠ Pat

at 2 {1, · · · ,K}

5

Learner choosesP1

µ1

Gets reward

P2 P3

µ2 µ3

R(n) = maxa⇤2{1,···K}

nµa⇤ � E"

nX

t=1

Xat

#

Page 19: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Multi Armed Bandits

Xat ⇠ Pat

at 2 {1, · · · ,K}

5

Learner choosesP1

µ1

Gets reward

P2 P3

µ2 µ3

R(n) = maxa⇤2{1,···K}

nµa⇤ � E"

nX

t=1

Xat

#

R(n) = O(pKn log(n))MAB regret

[Auer et al. 2002]

Page 20: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Structured losses

6

Arms = Paths

MAB regret ExponentialPacket routing

Network (V,E)

at 2 A ⇢ {0, 1}E

wt 2 W = [0, 1]ELoss = delay

hat, wtiDelay is linear

R(n) = O(

p|num paths| · n log(n))

Page 21: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Structured losses

6

Arms = Paths

MAB regret ExponentialPacket routing

Network (V,E)

at 2 A ⇢ {0, 1}E

wt 2 W = [0, 1]ELoss = delay

hat, wtiDelay is linear

R(n) = O(

p|num paths| · n log(n))

Page 22: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

7

Page 23: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

7

Page 24: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

Adversary’s loss `t(a) = hwt, ai for wt 2 W ⇢ Rd

7

Page 25: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

Adversary’s loss `t(a) = hwt, ai for Can be i.i.d or

adversarial

wt 2 W ⇢ Rd

7

Page 26: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

Adversary’s loss `t(a) = hwt, ai for

Learner only experienceshwt, ati Can be i.i.d or adversarial

wt 2 W ⇢ Rd

7

Page 27: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

Adversary’s loss `t(a) = hwt, ai for

Learner only experienceshwt, ati Can be i.i.d or adversarial

wt 2 W ⇢ Rd

7

Expected regret:

R(n) = E"

nX

t=1

hwt, ati � infa2A

nX

t=1

hwt, ai#

Page 28: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear Bandits

Learner chooses an action at 2 A ⇢ Rd

Adversary’s loss `t(a) = hwt, ai for

Learner only experienceshwt, ati Can be i.i.d or adversarial

wt 2 W ⇢ Rd

MAB reduces to Linear Bandits

7

A = {e1, · · · , ed} W = [0, 1]d

Expected regret:

,

R(n) = E"

nX

t=1

hwt, ati � infa2A

nX

t=1

hwt, ai#

Page 29: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

Page 30: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Page 31: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Page 32: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Page 33: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

See hwt, ati

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Page 34: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

See hwt, ati

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Build loss estimator wt

Page 35: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial linear bandits

8

For t = 1, · · · , n :

See hwt, ati

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Update

Build loss estimator wt

qt(a) / exp(�⌘hwt, ai)qt�1(a)| {z }Exponential weights

Page 36: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights

9

qt(a) / exp(�⌘htX

i=1

wi, ai)

Page 37: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights

9

A

tX

i=1

wi

qt(a) / exp(�⌘htX

i=1

wi, ai)

Page 38: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights

9

A A

tX

i=1

wi

qt

qt(a) / exp(�⌘htX

i=1

wi, ai)

Page 39: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Unbiased estimator of the loss

⌃t = Ea⇠pt

⇥aa>

⇤wt = (⌃t)

�1 athwt, atiLet and set

10

Page 40: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Unbiased estimator of the loss

⌃t = Ea⇠pt

⇥aa>

⇤wt = (⌃t)

�1 athwt, atiLet and set

10

Page 41: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Unbiased estimator of the loss

⌃t = Ea⇠pt

⇥aa>

⇤wt = (⌃t)

�1 athwt, atiLet and set

is an unbiased estimator of :wt wt

10

Page 42: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Unbiased estimator of the loss

⌃t = Ea⇠pt

⇥aa>

⇤wt = (⌃t)

�1 athwt, atiLet and set

is an unbiased estimator of :

Eat⇠pt [wt|Ft�1] =�Ea⇠pt

⇥aaT

⇤��1 Eat⇠pt [athwt, ati|Ft�1]

=�Ea⇠pt

⇥aaT

⇤��1 Eat⇠pt

⇥ata

>t |Ft�1

⇤wt

= wt

wt wt

10

Page 43: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear bandits regret

11

Theorem. (Linear Bandits Regret).

R(n) �n+log(|A|)

⌘+ ⌘

nX

t=1

EEa⇠pt(hwt, ai)2

[See for example Bubeck ‘11]

Uniform over , [Cesa-Bianchi, Lugosi, ’12]

John’s distribution [Bubeck, Cesa-Bianchi, Kakade ’12]

A

O(dpn)

O(p

dn log(|A|)) = O(dpn)

Exploration over Barycentric Spanner, [Dani, Hayes, Kakade ’08]

O(dpn log(|A|)) = O(d3/2

pn)

Page 44: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear bandits regret Dimension dependence

Variance bound:E⇥Eat⇠pt

⇥(hwt, ai)2

⇤⇤ d

12

Page 45: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear bandits regret Dimension dependence

Variance bound:E⇥Eat⇠pt

⇥(hwt, ai)2

⇤⇤ d

Dimension dependence

12

Page 46: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linear bandits regret Dimension dependence

Variance bound:E⇥Eat⇠pt

⇥(hwt, ai)2

⇤⇤ d

Dimension dependence

12

⌘dn

R(n) �n+log(|A|)

⌘+ ⌘

nX

t=1

EEa⇠pt(hwt, ai)2

| {z }

Page 47: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Recap

13

• Intro to Online Learning

• Linear Bandits

• Kernel Bandits

Page 48: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Covfefe

Online Quadratic losses

Bt

`t(a) = hbt, ai+ a>Bta

Offline problem haspolytime solution

14

min `t(a)

a 2 A

Symmetric and possibly non convex

Strong Duality

z = x2 � .5 ⇤ y2 + x ⇤ y � .5 ⇤ x+ .5y + 1Peter Bartlett Niladri Chatterji

at 2 A = {a s.t. kak2 1}

Page 49: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Linearization of Quadratic losses

Quadratic losses are linear in the space of

`(a) = hbt, ai+ a>Bta `(a) =

⌧✓Bt

bt

◆,

✓aa>

a

◆�

matricesvector( )

We can use the linear bandits machinery

Exponential weights for quadratic bandits

15

Page 50: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial quadratic bandits

16

For t = 1, · · · , n :

See

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Update

Build loss estimator

hbt, ati+ a>t Btat

✓Bt

bt

qt(a) / exp(�⌘(hbt, ai+ a>Bta))qt�1(a)| {z }Exponential weights

Sampling is poly time

Page 51: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Beyond “Finite Dimensional” Losses

17

Evasion games:

Gaussian kernel -`t(a) = exp(�ka� wtk2)

Infinite dimensional

Obstacle avoidance

Page 52: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Space of Quadratics as a Reproducing Kernel Hilbert Space

The Reproducing Kernel Hilbert Space of HK

Quadratics losses lie in an RKHS K(x, y) = hx, yi+ (hx, yi)2

K

Feature mapx ! �(x) 2 RD

Dot productK(x, y) = h�(x),�(y)i

18

Page 53: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits

19

Page 54: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits

If `t 2 HK can we leverage linearity?

19

Page 55: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits

If `t 2 HK can we leverage linearity?

Main challenge:

Dimension of HK might be infinite.

Naive Linear regret

19

O(p

dn log(|A|))

Page 56: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits

If `t 2 HK can we leverage linearity?

Main challenge:

Dimension of HK might be infinite.

Naive Linear regret

Encouraging facts:

Kernel spaces are “small”

19

O(p

dn log(|A|))

Page 57: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Towards an Algorithm

Algorithm Strategy:

20

Page 58: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Towards an Algorithm

Algorithm Strategy:

1) Construct an dimensional proxy kernel that uniformly approximates the original kernel over .

20

Km(x, y) ⇡ K(x, y)

m < 1A⇥A

Page 59: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Towards an Algorithm

Algorithm Strategy:

1) Construct an dimensional proxy kernel that uniformly approximates the original kernel over .

II) Exponential weights using the proxy kernel. Control the bias.

20

Km(x, y) ⇡ K(x, y)

m < 1A⇥A

Page 60: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

K(wt, at)

Towards an Algorithm

Algorithm Strategy:

1) Construct an dimensional proxy kernel that uniformly approximates the original kernel over .

II) Exponential weights using the proxy kernel. Control the bias.

20

Km(x, y) ⇡ K(x, y)

Receive

m < 1A⇥A

Page 61: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

K(wt, at)

Towards an Algorithm

Algorithm Strategy:

1) Construct an dimensional proxy kernel that uniformly approximates the original kernel over .

II) Exponential weights using the proxy kernel. Control the bias.

20

Km(x, y) ⇡ K(x, y)

Receive

m < 1A⇥A

Page 62: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

K(wt, at)

Towards an Algorithm

Algorithm Strategy:

1) Construct an dimensional proxy kernel that uniformly approximates the original kernel over .

II) Exponential weights using the proxy kernel. Control the bias.

20

Km(x, y) ⇡ K(x, y)

Km(wt, at)Receive Pretend it was

m < 1A⇥A

Page 63: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel functions are “small”

Kernel function evaluations can be uniformly approximated by a small number of basis functions.

21

Page 64: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel functions are “small”

Kernel function evaluations can be uniformly approximated by a small number of basis functions.

Mercer’s Theorem

There exist functions {�i}1i=1 and nonnegative values {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

such that:

8x, y 2 A⇥A

21

Page 65: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Eigendecay

22

K(x, y) =1X

i=1

µi�i(x)�i(y)

Page 66: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Eigendecay

22

Gaussian KernelSobolev Kernel

K(x, y) = exp(�kx� yk2) K(x, y) = min(x, y)

K(x, y) =1X

i=1

µi�i(x)�i(y)

Page 67: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Eigendecay

22

Gaussian KernelSobolev Kernel

µj Ce��jExponential decay

{�j(x)} = {sin(j⇡x), cos(j⇡x)}

µj ⇡ e�cj log(j)

K(x, y) = exp(�kx� yk2) K(x, y) = min(x, y)

K(x, y) =1X

i=1

µi�i(x)�i(y)

Page 68: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Eigendecay

22

Gaussian KernelSobolev Kernel

µj Ce��j µj Cj��Exponential decay Polynomial decay

{�j(x)} = {sin(j⇡x), cos(j⇡x)}

µj ⇡ e�cj log(j)

K(x, y) = exp(�kx� yk2) K(x, y) = min(x, y)

µj ⇡1

j2

�j(x) ⇡ sin

✓2j⇡x

2

K(x, y) =1X

i=1

µi�i(x)�i(y)

Page 69: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Eigendecay

22

Gaussian KernelSobolev Kernel

µj Ce��j µj Cj��Exponential decay Polynomial decay

{�j(x)} = {sin(j⇡x), cos(j⇡x)}

µj ⇡ e�cj log(j)

K(x, y) = exp(�kx� yk2) K(x, y) = min(x, y)

µj ⇡1

j2

�j(x) ⇡ sin

✓2j⇡x

2

K(x, y) =1X

i=1

µi�i(x)�i(y)

Page 70: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Construction of a finite dimensional Proxy Kernel

Eigenfunctions {�i}1i=1 with eigenvalues {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

23

Page 71: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Construction of a finite dimensional Proxy Kernel

Eigenfunctions {�i}1i=1 with eigenvalues {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

Truncate at i = m

23

Page 72: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Construction of a finite dimensional Proxy Kernel

Eigenfunctions {�i}1i=1 with eigenvalues {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

Truncate at i = m

Ko(x, y) =mX

i=1

µi�i(x)�i(y)

23

Page 73: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Construction of a finite dimensional Proxy Kernel

Eigenfunctions {�i}1i=1 with eigenvalues {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

Truncate at i = m

Deterministic Proxy KernelKo(x, y) =

mX

i=1

µi�i(x)�i(y)

23

Page 74: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Construction of a finite dimensional Proxy Kernel

Eigenfunctions {�i}1i=1 with eigenvalues {µi}1i=1

K(x, y) =1X

i=1

µi�i(x)�i(y)

Truncate at i = m

Deterministic Proxy KernelKo(x, y) =

mX

i=1

µi�i(x)�i(y)

23

Building proxy Kernel with samplesKernel PCA

Page 75: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

Page 76: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

Km(x, y) = h�m(x),�m(y)iBuild from P

Page 77: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

For t = 1, · · · , n :

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Km(x, y) = h�m(x),�m(y)iBuild from P

Page 78: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

For t = 1, · · · , n :

See

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Km(x, y) = h�m(x),�m(y)iBuild from P

K(wt, at)

Page 79: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

For t = 1, · · · , n :

See

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Build loss estimator

Km(x, y) = h�m(x),�m(y)iBuild from P

K(wt, at)

wt

Page 80: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

For t = 1, · · · , n :

See

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Update

Build loss estimator

Km(x, y) = h�m(x),�m(y)iBuild from P

K(wt, at)

wt

qt(a) / exp(�⌘hwt,�m(a)i)qt�1(a)| {z }Exponential weights

Page 81: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Exponential weights for adversarial kernel bandits

24

For t = 1, · · · , n :

See

Sample mixture at ⇠ pt = (1� �)qt| {z }Exploitation

+ �⌫|{z}Exploration

Update

Build loss estimator

Sampling might not be poly time

Km(x, y) = h�m(x),�m(y)iBuild from P

K(wt, at)

wt

qt(a) / exp(�⌘hwt,�m(a)i)qt�1(a)| {z }Exponential weights

Page 82: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Biased loss estimates

is biased:

Eat⇠pt [wt|Ft�1] = EhK(at, yt)

⇣(⌃(t)

m )�1�m(at)⌘ ���Ft�1

i

= �m(yt) + E

2

664⇣K(at, yt)� Km(at, yt)

⌘⇣(⌃(t)

m )�1�m(at)⌘

| {z }=:⇠t, the bias

���Ft�1

3

775

25

Let and set⌃(t)m = Ea⇠pt

⇥�m(a)�m(a)>

wt

wt := K(at, yt)⇣(⌃(t)

m )�1�m(at)⌘

Page 83: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits Regret

26

Expected regret

game vs Km K game

⇠t bias

R(n) = E"

nX

t=1

K(wt, at)� infa2A

nX

t=1

K(wt, a⇤)

#

Theorem.

R(n) 2�n+ ⌘mn+2✏n

⌘| {z }Bias variance

+2✏n+1

⌘log(|A|).

Page 84: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits Regret

26

Expected regret

game vs Km K game

⇠t bias

R(n) = E"

nX

t=1

K(wt, at)� infa2A

nX

t=1

K(wt, a⇤)

#

Theorem.

R(n) 2�n+ ⌘mn+2✏n

⌘| {z }Bias variance

+2✏n+1

⌘log(|A|).

Page 85: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits Regret

26

Expected regret

game vs Km K game

⇠t bias

R(n) = E"

nX

t=1

K(wt, at)� infa2A

nX

t=1

K(wt, a⇤)

#

Theorem.

R(n) 2�n+ ⌘mn+2✏n

⌘| {z }Bias variance

+2✏n+1

⌘log(|A|).

Page 86: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits Regret

26

Expected regret

game vs Km K game

⇠t bias

R(n) = E"

nX

t=1

K(wt, at)� infa2A

nX

t=1

K(wt, a⇤)

#

Theorem.

R(n) 2�n+ ⌘mn+2✏n

⌘| {z }Bias variance

+2✏n+1

⌘log(|A|).

Page 87: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Kernel Bandits Regret

27

Corollary.

1 Polynomial Decay. µj Cj�� and � > 2:

R(n) O

⇣log(|A|)

��22(��1)n

�2(��1)

2 Exponential Decay. µj Ce��j:

R(n) O

⇣plog(|A|) log(n)n

Page 88: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Experiments

28

Page 89: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Page 90: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

:

Page 91: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

:

Page 92: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

Exponential decay µj C exp(��j)<latexit sha1_base64="6KrhvBMH2GSczXQefXPt+SwlH88=">AAACBnicbVC7TkJBEJ2LL8QXamlMNqIJFpJ7bbQk0lhiIo+ES8jeZYCFvQ939xoJobLxV2wsNGrLN9j5IfYuj0LRk0xycs5MZuZ4keBK2/anlVhYXFpeSa6m1tY3NrfS2ztlFcaSYYmFIpRVjyoUPMCS5lpgNZJIfU9gxesVxn7lFqXiYXCt+xHWfdoOeIszqo3USO+7ftzoElfgDSkQF++iLDlxPdSUdMlxI52xc/YE5C9xZiSTP/x6GwFAsZH+cJshi30MNBNUqZpjR7o+oFJzJnCYcmOFEWU92saaoQH1UdUHkzeG5MgoTdIKpalAk4n6c2JAfaX6vmc6fao7at4bi/95tVi3zusDHkSxxoBNF7ViQXRIxpmQJpfItOgbQpnk5lbCOlRSpk1yKROCM//yX1I+zTl2zrkyaVzAFEnYgwPIggNnkIdLKEIJGNzDIzzDi/VgPVmv1vu0NWHNZnbhF6zRN/dDmck=</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="rxOBCj0G4MeW/w28ys/mhsY83/8=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBHqwpK40WWxG5cV7AOaECbTm3baycOZiVhCV278FTcuFHHrN7jzb5y2WWjrgQuHc+7l3nv8hDOpLOvbWFpeWV1bL2wUN7e2d3bNvf2mjFNBoUFjHou2TyRwFkFDMcWhnQggoc+h5Q9rE791D0KyOLpVowTckPQiFjBKlJY888gJU2+AHQ53uIYdeEjK+MzxQRE8wKeeWbIq1hR4kdg5KaEcdc/8croxTUOIFOVEyo5tJcrNiFCMchgXnVRCQuiQ9KCjaURCkG42fWOMT7TSxUEsdEUKT9XfExkJpRyFvu4MierLeW8i/ud1UhVcuhmLklRBRGeLgpRjFeNJJrjLBFDFR5oQKpi+FdM+EYQqnVxRh2DPv7xImucV26rYN1apepXHUUCH6BiVkY0uUBVdozpqIIoe0TN6RW/Gk/FivBsfs9YlI585QH9gfP4ALOKW+g==</latexit>

:

:

Page 93: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

Exponential decay µj C exp(��j)<latexit sha1_base64="6KrhvBMH2GSczXQefXPt+SwlH88=">AAACBnicbVC7TkJBEJ2LL8QXamlMNqIJFpJ7bbQk0lhiIo+ES8jeZYCFvQ939xoJobLxV2wsNGrLN9j5IfYuj0LRk0xycs5MZuZ4keBK2/anlVhYXFpeSa6m1tY3NrfS2ztlFcaSYYmFIpRVjyoUPMCS5lpgNZJIfU9gxesVxn7lFqXiYXCt+xHWfdoOeIszqo3USO+7ftzoElfgDSkQF++iLDlxPdSUdMlxI52xc/YE5C9xZiSTP/x6GwFAsZH+cJshi30MNBNUqZpjR7o+oFJzJnCYcmOFEWU92saaoQH1UdUHkzeG5MgoTdIKpalAk4n6c2JAfaX6vmc6fao7at4bi/95tVi3zusDHkSxxoBNF7ViQXRIxpmQJpfItOgbQpnk5lbCOlRSpk1yKROCM//yX1I+zTl2zrkyaVzAFEnYgwPIggNnkIdLKEIJGNzDIzzDi/VgPVmv1vu0NWHNZnbhF6zRN/dDmck=</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="rxOBCj0G4MeW/w28ys/mhsY83/8=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBHqwpK40WWxG5cV7AOaECbTm3baycOZiVhCV278FTcuFHHrN7jzb5y2WWjrgQuHc+7l3nv8hDOpLOvbWFpeWV1bL2wUN7e2d3bNvf2mjFNBoUFjHou2TyRwFkFDMcWhnQggoc+h5Q9rE791D0KyOLpVowTckPQiFjBKlJY888gJU2+AHQ53uIYdeEjK+MzxQRE8wKeeWbIq1hR4kdg5KaEcdc/8croxTUOIFOVEyo5tJcrNiFCMchgXnVRCQuiQ9KCjaURCkG42fWOMT7TSxUEsdEUKT9XfExkJpRyFvu4MierLeW8i/ud1UhVcuhmLklRBRGeLgpRjFeNJJrjLBFDFR5oQKpi+FdM+EYQqnVxRh2DPv7xImucV26rYN1apepXHUUCH6BiVkY0uUBVdozpqIIoe0TN6RW/Gk/FivBsfs9YlI585QH9gfP4ALOKW+g==</latexit>

Rn � ⌦⇣n1/2

<latexit sha1_base64="nsGSg0R6M5uCBYoBTJSubG1VG7g=">AAACHHicbVDBbtNAFHwuhZaUggtHLqtGlcIltdNDeqzaCzdaRNpKcYjWm2dn1fXa7D5XiixLfAPiwoVf4cKhCHHhgMTfdJ30UBJGWmk0857ezsSFkpaC4K+39mD94aONzcetrSfbT5/5O8/PbV4agQORq9xcxtyikhoHJEnhZWGQZ7HCi/jqpPEvrtFYmet3NCtwlPFUy0QKTk4a+wdRxmkquKre1mPNohQ/sOhNhimPFCbUYQ76fcXC/R6rIyPTKb0a++2gG8zBVkl4R9pH/c+fPgLA6dj/HU1yUWaoSShu7TAMChpV3JAUCutWVFosuLjiKQ4d1TxDO6rm4Wq255QJS3LjniY2V+9vVDyzdpbFbrKJYpe9RvyfNywpORxVUhcloRaLQ0mpGOWsaYpNpEFBauYIF0a6vzIx5YYLcn22XAnhcuRVct7rhkE3PHNtHMMCm/ASdqEDIfThCF7DKQxAwBf4Bjfww/vqffd+er8Wo2ve3c4L+Afen1vQ6aIV</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="P7MkYu2AcvooIzrrzmOwU1Nkj5c=">AAACHHicbVA9T8MwFHT4LOWrwMhiUSHBUpIywFjBwkZBFJCaUjnuS2rVcYL9glRF/SEs/BUWBhBiYUDi3+CWDlA4ydLp7j093wWpFAZd99OZmp6ZnZsvLBQXl5ZXVktr65cmyTSHBk9koq8DZkAKBQ0UKOE61cDiQMJV0Dse+ld3oI1I1AX2U2jFLFIiFJyhldqlfT9m2OVM5ueDtqJ+BLfUP40hYr6EEHeohbrJqbdXpQNfi6iLu+1S2a24I9C/xBuTMhmj3i69+52EZzEo5JIZ0/TcFFs50yi4hEHRzwykjPdYBE1LFYvBtPJRuAHdtkqHhom2TyEdqT83chYb048DOzmMYia9ofif18wwPGzlQqUZguLfh8JMUkzosCnaERo4yr4ljGth/0p5l2nG0fZZtCV4k5H/kstqxXMr3plbrh2N6yiQTbJFdohHDkiNnJA6aRBO7skjeSYvzoPz5Lw6b9+jU854Z4P8gvPxBfZln/g=</latexit>

:

:

Page 94: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

Exponential decay µj C exp(��j)<latexit sha1_base64="6KrhvBMH2GSczXQefXPt+SwlH88=">AAACBnicbVC7TkJBEJ2LL8QXamlMNqIJFpJ7bbQk0lhiIo+ES8jeZYCFvQ939xoJobLxV2wsNGrLN9j5IfYuj0LRk0xycs5MZuZ4keBK2/anlVhYXFpeSa6m1tY3NrfS2ztlFcaSYYmFIpRVjyoUPMCS5lpgNZJIfU9gxesVxn7lFqXiYXCt+xHWfdoOeIszqo3USO+7ftzoElfgDSkQF++iLDlxPdSUdMlxI52xc/YE5C9xZiSTP/x6GwFAsZH+cJshi30MNBNUqZpjR7o+oFJzJnCYcmOFEWU92saaoQH1UdUHkzeG5MgoTdIKpalAk4n6c2JAfaX6vmc6fao7at4bi/95tVi3zusDHkSxxoBNF7ViQXRIxpmQJpfItOgbQpnk5lbCOlRSpk1yKROCM//yX1I+zTl2zrkyaVzAFEnYgwPIggNnkIdLKEIJGNzDIzzDi/VgPVmv1vu0NWHNZnbhF6zRN/dDmck=</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="rxOBCj0G4MeW/w28ys/mhsY83/8=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBHqwpK40WWxG5cV7AOaECbTm3baycOZiVhCV278FTcuFHHrN7jzb5y2WWjrgQuHc+7l3nv8hDOpLOvbWFpeWV1bL2wUN7e2d3bNvf2mjFNBoUFjHou2TyRwFkFDMcWhnQggoc+h5Q9rE791D0KyOLpVowTckPQiFjBKlJY888gJU2+AHQ53uIYdeEjK+MzxQRE8wKeeWbIq1hR4kdg5KaEcdc/8croxTUOIFOVEyo5tJcrNiFCMchgXnVRCQuiQ9KCjaURCkG42fWOMT7TSxUEsdEUKT9XfExkJpRyFvu4MierLeW8i/ud1UhVcuhmLklRBRGeLgpRjFeNJJrjLBFDFR5oQKpi+FdM+EYQqnVxRh2DPv7xImucV26rYN1apepXHUUCH6BiVkY0uUBVdozpqIIoe0TN6RW/Gk/FivBsfs9YlI585QH9gfP4ALOKW+g==</latexit>

Rn � ⌦⇣n1/2

<latexit sha1_base64="nsGSg0R6M5uCBYoBTJSubG1VG7g=">AAACHHicbVDBbtNAFHwuhZaUggtHLqtGlcIltdNDeqzaCzdaRNpKcYjWm2dn1fXa7D5XiixLfAPiwoVf4cKhCHHhgMTfdJ30UBJGWmk0857ezsSFkpaC4K+39mD94aONzcetrSfbT5/5O8/PbV4agQORq9xcxtyikhoHJEnhZWGQZ7HCi/jqpPEvrtFYmet3NCtwlPFUy0QKTk4a+wdRxmkquKre1mPNohQ/sOhNhimPFCbUYQ76fcXC/R6rIyPTKb0a++2gG8zBVkl4R9pH/c+fPgLA6dj/HU1yUWaoSShu7TAMChpV3JAUCutWVFosuLjiKQ4d1TxDO6rm4Wq255QJS3LjniY2V+9vVDyzdpbFbrKJYpe9RvyfNywpORxVUhcloRaLQ0mpGOWsaYpNpEFBauYIF0a6vzIx5YYLcn22XAnhcuRVct7rhkE3PHNtHMMCm/ASdqEDIfThCF7DKQxAwBf4Bjfww/vqffd+er8Wo2ve3c4L+Afen1vQ6aIV</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="P7MkYu2AcvooIzrrzmOwU1Nkj5c=">AAACHHicbVA9T8MwFHT4LOWrwMhiUSHBUpIywFjBwkZBFJCaUjnuS2rVcYL9glRF/SEs/BUWBhBiYUDi3+CWDlA4ydLp7j093wWpFAZd99OZmp6ZnZsvLBQXl5ZXVktr65cmyTSHBk9koq8DZkAKBQ0UKOE61cDiQMJV0Dse+ld3oI1I1AX2U2jFLFIiFJyhldqlfT9m2OVM5ueDtqJ+BLfUP40hYr6EEHeohbrJqbdXpQNfi6iLu+1S2a24I9C/xBuTMhmj3i69+52EZzEo5JIZ0/TcFFs50yi4hEHRzwykjPdYBE1LFYvBtPJRuAHdtkqHhom2TyEdqT83chYb048DOzmMYia9ofif18wwPGzlQqUZguLfh8JMUkzosCnaERo4yr4ljGth/0p5l2nG0fZZtCV4k5H/kstqxXMr3plbrh2N6yiQTbJFdohHDkiNnJA6aRBO7skjeSYvzoPz5Lw6b9+jU854Z4P8gvPxBfZln/g=</latexit>

:

:

Page 95: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

Exponential decay µj C exp(��j)<latexit sha1_base64="6KrhvBMH2GSczXQefXPt+SwlH88=">AAACBnicbVC7TkJBEJ2LL8QXamlMNqIJFpJ7bbQk0lhiIo+ES8jeZYCFvQ939xoJobLxV2wsNGrLN9j5IfYuj0LRk0xycs5MZuZ4keBK2/anlVhYXFpeSa6m1tY3NrfS2ztlFcaSYYmFIpRVjyoUPMCS5lpgNZJIfU9gxesVxn7lFqXiYXCt+xHWfdoOeIszqo3USO+7ftzoElfgDSkQF++iLDlxPdSUdMlxI52xc/YE5C9xZiSTP/x6GwFAsZH+cJshi30MNBNUqZpjR7o+oFJzJnCYcmOFEWU92saaoQH1UdUHkzeG5MgoTdIKpalAk4n6c2JAfaX6vmc6fao7at4bi/95tVi3zusDHkSxxoBNF7ViQXRIxpmQJpfItOgbQpnk5lbCOlRSpk1yKROCM//yX1I+zTl2zrkyaVzAFEnYgwPIggNnkIdLKEIJGNzDIzzDi/VgPVmv1vu0NWHNZnbhF6zRN/dDmck=</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="rxOBCj0G4MeW/w28ys/mhsY83/8=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBHqwpK40WWxG5cV7AOaECbTm3baycOZiVhCV278FTcuFHHrN7jzb5y2WWjrgQuHc+7l3nv8hDOpLOvbWFpeWV1bL2wUN7e2d3bNvf2mjFNBoUFjHou2TyRwFkFDMcWhnQggoc+h5Q9rE791D0KyOLpVowTckPQiFjBKlJY888gJU2+AHQ53uIYdeEjK+MzxQRE8wKeeWbIq1hR4kdg5KaEcdc/8croxTUOIFOVEyo5tJcrNiFCMchgXnVRCQuiQ9KCjaURCkG42fWOMT7TSxUEsdEUKT9XfExkJpRyFvu4MierLeW8i/ud1UhVcuhmLklRBRGeLgpRjFeNJJrjLBFDFR5oQKpi+FdM+EYQqnVxRh2DPv7xImucV26rYN1apepXHUUCH6BiVkY0uUBVdozpqIIoe0TN6RW/Gk/FivBsfs9YlI585QH9gfP4ALOKW+g==</latexit>

Rn � ⌦⇣n1/2

<latexit sha1_base64="nsGSg0R6M5uCBYoBTJSubG1VG7g=">AAACHHicbVDBbtNAFHwuhZaUggtHLqtGlcIltdNDeqzaCzdaRNpKcYjWm2dn1fXa7D5XiixLfAPiwoVf4cKhCHHhgMTfdJ30UBJGWmk0857ezsSFkpaC4K+39mD94aONzcetrSfbT5/5O8/PbV4agQORq9xcxtyikhoHJEnhZWGQZ7HCi/jqpPEvrtFYmet3NCtwlPFUy0QKTk4a+wdRxmkquKre1mPNohQ/sOhNhimPFCbUYQ76fcXC/R6rIyPTKb0a++2gG8zBVkl4R9pH/c+fPgLA6dj/HU1yUWaoSShu7TAMChpV3JAUCutWVFosuLjiKQ4d1TxDO6rm4Wq255QJS3LjniY2V+9vVDyzdpbFbrKJYpe9RvyfNywpORxVUhcloRaLQ0mpGOWsaYpNpEFBauYIF0a6vzIx5YYLcn22XAnhcuRVct7rhkE3PHNtHMMCm/ASdqEDIfThCF7DKQxAwBf4Bjfww/vqffd+er8Wo2ve3c4L+Afen1vQ6aIV</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="P7MkYu2AcvooIzrrzmOwU1Nkj5c=">AAACHHicbVA9T8MwFHT4LOWrwMhiUSHBUpIywFjBwkZBFJCaUjnuS2rVcYL9glRF/SEs/BUWBhBiYUDi3+CWDlA4ydLp7j093wWpFAZd99OZmp6ZnZsvLBQXl5ZXVktr65cmyTSHBk9koq8DZkAKBQ0UKOE61cDiQMJV0Dse+ld3oI1I1AX2U2jFLFIiFJyhldqlfT9m2OVM5ueDtqJ+BLfUP40hYr6EEHeohbrJqbdXpQNfi6iLu+1S2a24I9C/xBuTMhmj3i69+52EZzEo5JIZ0/TcFFs50yi4hEHRzwykjPdYBE1LFYvBtPJRuAHdtkqHhom2TyEdqT83chYb048DOzmMYia9ofif18wwPGzlQqUZguLfh8JMUkzosCnaERo4yr4ljGth/0p5l2nG0fZZtCV4k5H/kstqxXMr3plbrh2N6yiQTbJFdohHDkiNnJA6aRBO7skjeSYvzoPz5Lw6b9+jU854Z4P8gvPxBfZln/g=</latexit>

:

:

A = {(Aj)1j=1 s.t. |Aj | = 1 8j}

<latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit>

Page 96: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Lower Bound

29

Polynomial decay µj Cj��<latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit><latexit sha1_base64="ychUrnvNonFPPlbYXzEDvS1xm1g=">AAAB/3icbVDLSsNAFJ3UV62vqODGzWAR3FgSEXRZ7MZlBfuAJobJ9KaddvJwZiKU2IW/4saFIm79DXf+jdM2C209cOFwzr3ce4+fcCaVZX0bhaXlldW14nppY3Nre8fc3WvKOBUUGjTmsWj7RAJnETQUUxzaiQAS+hxa/rA28VsPICSLo1s1SsANSS9iAaNEackzD5ww9QbY4XCPa4O77NTxQZGxZ5atijUFXiR2TsooR90zv5xuTNMQIkU5kbJjW4lyMyIUoxzGJSeVkBA6JD3oaBqREKSbTe8f42OtdHEQC12RwlP190RGQilHoa87Q6L6ct6biP95nVQFl27GoiRVENHZoiDlWMV4EgbuMgFU8ZEmhAqmb8W0TwShSkdW0iHY8y8vkuZZxbYq9s15uXqVx1FEh+gInSAbXaAqukZ11EAUPaJn9IrejCfjxXg3PmatBSOf2Ud/YHz+ADjylZQ=</latexit>

Rn � ⌦⇣n

�+12�

<latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit><latexit sha1_base64="BYJ7uAoCHWKs/Xbmcb2avX6dlxA=">AAACNHicbVBNSxxBFOxRE80ak4055tK4CAZhmZGAHsVcBA8xwVVhe7O86X0z22xPz9j9RliG+VFe8kNykUAOCcGrvyG94x78Kmgoqt7jdVVcaOUoDH8FC4tLL14ur7xqrb5ee/O2/W791OWlldiTuc7teQwOtTLYI0UazwuLkMUaz+LJ55l/donWqdyc0LTAQQapUYmSQF4ato9EBjSWoKtv9dBwkeIFF18yTEFoTGiLe5jvFReJBVmJGAm2o7raaVjNG9TCqnRMH4ftTtgNG/CnJJqTDpvjeNj+KUa5LDM0JDU414/CggYVWFJSY90SpcMC5ARS7HtqIEM3qJrQNd/0yognufXPEG/U+xsVZM5Ns9hPziK6x95MfM7rl5TsDSplipLQyLtDSak55XzWIB8pi5L01BOQVvm/cjkGXw/5nlu+hOhx5KfkdKcbhd3o66fO/sG8jhX2gW2wLRaxXbbPDtkx6zHJrtg1+8P+Bj+C38G/4OZudCGY77xnDxDc/gdfaqlg</latexit>

Exponential decay µj C exp(��j)<latexit sha1_base64="6KrhvBMH2GSczXQefXPt+SwlH88=">AAACBnicbVC7TkJBEJ2LL8QXamlMNqIJFpJ7bbQk0lhiIo+ES8jeZYCFvQ939xoJobLxV2wsNGrLN9j5IfYuj0LRk0xycs5MZuZ4keBK2/anlVhYXFpeSa6m1tY3NrfS2ztlFcaSYYmFIpRVjyoUPMCS5lpgNZJIfU9gxesVxn7lFqXiYXCt+xHWfdoOeIszqo3USO+7ftzoElfgDSkQF++iLDlxPdSUdMlxI52xc/YE5C9xZiSTP/x6GwFAsZH+cJshi30MNBNUqZpjR7o+oFJzJnCYcmOFEWU92saaoQH1UdUHkzeG5MgoTdIKpalAk4n6c2JAfaX6vmc6fao7at4bi/95tVi3zusDHkSxxoBNF7ViQXRIxpmQJpfItOgbQpnk5lbCOlRSpk1yKROCM//yX1I+zTl2zrkyaVzAFEnYgwPIggNnkIdLKEIJGNzDIzzDi/VgPVmv1vu0NWHNZnbhF6zRN/dDmck=</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="sfWIAO7uvw7TRl0fvNwYBT6YgNU=">AAACBnicbVC7SgNBFJ2Nr5j4WLUUYTAKsTDs2mgZTGMZwTwgG8Ls5CaZZPbhzGwwLKls/BUbC0VtBf/Azg/R2smj0MQDFw7n3Mu997ghZ1JZ1qeRWFhcWl5JrqbSa+sbm+bWdlkGkaBQogEPRNUlEjjzoaSY4lANBRDP5VBxe4WRX+mDkCzwr9QghLpH2j5rMUqUlhrmnuNFjS52OFzjAnbgJsziY8cFRXAXHzXMjJWzxsDzxJ6STP7g6+W9n/4uNswPpxnQyANfUU6krNlWqOoxEYpRDsOUE0kICe2RNtQ09YkHsh6P3xjiQ600cSsQunyFx+rviZh4Ug48V3d6RHXkrDcS//NqkWqd1WPmh5ECn04WtSKOVYBHmeAmE0AVH2hCqGD6Vkw7RBCqdHIpHYI9+/I8KZ/kbCtnX+o0ztEESbSL9lEW2egU5dEFKqISougW3aNH9GTcGQ/Gs/E6aU0Y05kd9AfG2w/v45tD</latexit><latexit sha1_base64="rxOBCj0G4MeW/w28ys/mhsY83/8=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBHqwpK40WWxG5cV7AOaECbTm3baycOZiVhCV278FTcuFHHrN7jzb5y2WWjrgQuHc+7l3nv8hDOpLOvbWFpeWV1bL2wUN7e2d3bNvf2mjFNBoUFjHou2TyRwFkFDMcWhnQggoc+h5Q9rE791D0KyOLpVowTckPQiFjBKlJY888gJU2+AHQ53uIYdeEjK+MzxQRE8wKeeWbIq1hR4kdg5KaEcdc/8croxTUOIFOVEyo5tJcrNiFCMchgXnVRCQuiQ9KCjaURCkG42fWOMT7TSxUEsdEUKT9XfExkJpRyFvu4MierLeW8i/ud1UhVcuhmLklRBRGeLgpRjFeNJJrjLBFDFR5oQKpi+FdM+EYQqnVxRh2DPv7xImucV26rYN1apepXHUUCH6BiVkY0uUBVdozpqIIoe0TN6RW/Gk/FivBsfs9YlI585QH9gfP4ALOKW+g==</latexit>

Rn � ⌦⇣n1/2

<latexit sha1_base64="nsGSg0R6M5uCBYoBTJSubG1VG7g=">AAACHHicbVDBbtNAFHwuhZaUggtHLqtGlcIltdNDeqzaCzdaRNpKcYjWm2dn1fXa7D5XiixLfAPiwoVf4cKhCHHhgMTfdJ30UBJGWmk0857ezsSFkpaC4K+39mD94aONzcetrSfbT5/5O8/PbV4agQORq9xcxtyikhoHJEnhZWGQZ7HCi/jqpPEvrtFYmet3NCtwlPFUy0QKTk4a+wdRxmkquKre1mPNohQ/sOhNhimPFCbUYQ76fcXC/R6rIyPTKb0a++2gG8zBVkl4R9pH/c+fPgLA6dj/HU1yUWaoSShu7TAMChpV3JAUCutWVFosuLjiKQ4d1TxDO6rm4Wq255QJS3LjniY2V+9vVDyzdpbFbrKJYpe9RvyfNywpORxVUhcloRaLQ0mpGOWsaYpNpEFBauYIF0a6vzIx5YYLcn22XAnhcuRVct7rhkE3PHNtHMMCm/ASdqEDIfThCF7DKQxAwBf4Bjfww/vqffd+er8Wo2ve3c4L+Afen1vQ6aIV</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="bgg4x9+jU1xGbJqPS40MZx+45cU=">AAACHHicbVC7SgNBFJ31bXxFY2czKII2cVcLLUUbO6MYFbIxzE7ubobMzq4zd4Ww5CfExsZfsbFQxMZC8A/8DCeJha8DA4dz7uXOOUEqhUHXfXeGhkdGx8YnJgtT0zOzc8X5hVOTZJpDlScy0ecBMyCFgioKlHCeamBxIOEsaO/3/LMr0EYk6gQ7KdRjFikRCs7QSo3ilh8zbHEm8+NuQ1E/gkvqH8YQMV9CiGvUQl3k1NvYpF1fi6iF643iilt2+6B/ifdFVna3b647pY/FSqP46jcTnsWgkEtmTM1zU6znTKPgEroFPzOQMt5mEdQsVSwGU8/74bp01SpNGibaPoW0r37fyFlsTCcO7GQvivnt9cT/vFqG4U49FyrNEBQfHAozSTGhvaZoU2jgKDuWMK6F/SvlLaYZR9tnwZbg/Y78l5xulj237B3ZNvbIABNkiSyTNeKRbbJLDkiFVAknt+SePJIn5855cJ6dl8HokPO1UyI/4Lx9AjGoox0=</latexit><latexit sha1_base64="P7MkYu2AcvooIzrrzmOwU1Nkj5c=">AAACHHicbVA9T8MwFHT4LOWrwMhiUSHBUpIywFjBwkZBFJCaUjnuS2rVcYL9glRF/SEs/BUWBhBiYUDi3+CWDlA4ydLp7j093wWpFAZd99OZmp6ZnZsvLBQXl5ZXVktr65cmyTSHBk9koq8DZkAKBQ0UKOE61cDiQMJV0Dse+ld3oI1I1AX2U2jFLFIiFJyhldqlfT9m2OVM5ueDtqJ+BLfUP40hYr6EEHeohbrJqbdXpQNfi6iLu+1S2a24I9C/xBuTMhmj3i69+52EZzEo5JIZ0/TcFFs50yi4hEHRzwykjPdYBE1LFYvBtPJRuAHdtkqHhom2TyEdqT83chYb048DOzmMYia9ofif18wwPGzlQqUZguLfh8JMUkzosCnaERo4yr4ljGth/0p5l2nG0fZZtCV4k5H/kstqxXMr3plbrh2N6yiQTbJFdohHDkiNnJA6aRBO7skjeSYvzoPz5Lw6b9+jU854Z4P8gvPxBfZln/g=</latexit>

:

:

A = {(Aj)1j=1 s.t. |Aj | = 1 8j}

<latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit><latexit sha1_base64="ZFCD40L2l5Z5TQRQEOGunFfdM7w=">AAAfI3icpVnbbtzGGd6kp1Q9Oe1lUWNiQYASrNbatVPHARaIZTmHRlZUy6dmuRGG5OySFsmhyaG0ypiP0wfoG/S+d0WBohcF2jfp98+QKx7kpGmVWJr5T/PPf55dN43CXO3u/uONN7/3/R/88Edv/XjjJz/92c9/ce3tXz7NZZF54oknI5k9d3kuojART1SoIvE8zQSP3Ug8c0/vE/7ZmcjyUCaP1UUq5jFfJuEi9LgC6OTa507MVeDxSN8rp8zRbPveyYt3T/SL6bj8ygmThbpgjhIrpVk+UiNWslcgYK/YlI2Zw5izkBmPIvaCOeXJtc3d0a75Yf3FuFpsDqqfo5O3f/Nnx5deEYtEeRHP89l4N1VzzTMVepEoN5wiFyn3TvlSzLBMeCzyuTa3LtkWID7D+fiXKGagTQ7N4zy/iF1Q0h3zLo6AV+FmhVp8MNdhkhZKJJ49aFFETElGJmR+mAlPRRdYcC8LoSvzAp5xT8HQG/Sz1RSX8pQc8LWY6vEdLx5ObntxOYx5tgyT6WQSx0Ml0+n4DhaB4D4Yp+/H8VwvhYyFyi5K9h3kjW99Z3md67aQehWplco4jMAELXJj8OfigD8Wz9ukRRJ60hc7xq5XGntja6sDVkHcIzU+a4v2eErh2oHmhbsIl33g1dQiQZxlXIkOHNnhIhliumK9ZsYabTpKrjBZEplX5ErG8AAzGdfRNX9ZSCVyQ4jAhNkYQjdHnJwJZnEdw6UGSgxIJXkOtowirCKGMmvF2p5zpYIec72QUsVhTqHKoDejfWJYfXmeMK6YCgSz1MwGSltQfrascmtVJ5fNJ5aGnioySDoPVcCOn37CDGH7AsuMp0HorYgNNcOLCl9UsDblLJEi8ZHg0TLNRYHcR8C00zwrImEpJFIriCdtvAZISRnlPegOtFTWve0oK1TgRqcdrwdIZBh50QbPzKXh5tN8SL+tDabMjQoxZEUWWQAgy0yIZMi8UImaKBP+kPHEC2TW5Ju/5iwtEaV+D0phCqFxO86oE5Apc6EICZBOJK3ASOlOSY7SOUx51lheIso2b8U5nYCuIrr9LdzNCJdnSASFDGNQkuoBdHYF7s8oQwroipIoVjxOI5F3y6FIlrhNMNcud0VE14yLSIWGk0ehuuj4L5XRxTKSeR6aKpSIc5EhT1BrE25SCmXZiCI+3BEm5RAYcZJnDGrOI95cCBYolX5486aneDIqQneUyJsxqrjMb64EcvVmxOl349DmepT6i+qYmKM5Ir1nD3maQvcp2HYo2YcH4ZKbnJmiRA6P0VvFFM1vrg/CpFixgxCmUmjX7AuqxFaYTOTVwiz7Q6684EDi5h56/fBAKFz/GBYS050Jid4XL/jTgh3zJGcPIaxca6kCEqzvYyTIQs4e1nW4gZzRYidXFzipSLNwGajhGaoEQvx0Jw/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+dupBJVeZwIVKDbPcNo6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJnESY8Qi1MUL3MpAN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYfDBjEMTC6DnISppHxwsiq+/xh55RKlwKNk5zxKolzPuSlRiW2yZRP0OOIGREDmqMU/wKwRVLItEjSpb1lohmOV51cXy0zAt9c54NDH6PQ6EzETMHolUoK7nMioMVSfJXPxFpZ265I4hs76lSiizC8EzW7i60B04Kh1C5xVhTZ2fTsweYuz2/d0hqzOLOOWQhbmbTBc8ym2FtEtkd+iaVKpytpexoQtk2xmAkZHKpls618Joe2rLc6NS13V5ihO90+G6Jlf7dSG3+0Y9pnYJsWQsbNFnyQBQEcOz7Xf1JTa6XQJknhqiBBstTfDNzYQgcuG1SzcA5CFGsxhFBkRZQhvI2qkISk2VVk9GGNwoMzCzkaBILBSzxwyrWHcFaq4YNiO/L7RwO3KNVFvL+6RdaoQb0e+O3icOsFymsfMUMRcLfUja7YmDg2NmIVUX6BELP1Qyy/XvEPQPzBpd0seSAULbssdiFNSPzdAiT+2U1af67HjvEGr04PtffGbBzQmHYh2ZyqM0QLNAgxo2UuAS2Az2u3eH7TxC+GIxVRmmgSpozKYR7TiV+z62qP/mjaZBNMKeMPsCdUnsgTaUZj66OFYZxcS6SemtDVi1kk2xCZX1fSjqM4QQAcrhFsNktKQChyfIDa+JvNHhznvs+Tfy5zc2tsoN1prYjFMxGdrLl+1U0HgXRWQtup7buJgxrDaGfQ3KdbMzyLwa62F05EtpMskuoTQCHv0Ao4VZxmnuhXZJbBtXSEkjNOmmjG8VcnU92kEe2O7QxsIMyJHYYCKhCEStcOrguRNFG0S9DjJtDUn2W8/npshQm6gzhzUj2Y1qutl4roHEa9sVcNwyk0Vq983+6mDIrzl06ayXjumZQmNSuE1z5ea43LLciFSFERwdSGS6pqdmU1o8Ir8+y+jdf/e0+3vMT0Wdq4zBRIb7Nf3WzpXrS+lFVOQBVTsLtZ86uAvtHNx79MkD7TAH/5/Zq0zI6A6mhkLRrc2ZdluuSUBRSbK/IQjNWBC9bXkGS3dsHO0Iq3IlZPyBcS5dvnl77uIsNIDaACI5CzOZ0McYusZVJqhbhA0JtuXQm66aSJp47URWO3Npp5ZCqTVi5RYZDrSm/1lRNBeYC9AcYXK3EfKE2GoEkP4s+RBXjdNAV7XamGxYMirOyH/3gr22PI8uq/yIHbLtye74zrtDhirLDkfsC0xF7J7n0VxT4CqZeWsioOKcyQW7f39n7w87h/d3bo92R0Yv6xW99g29NE380Dv3HIka0HwgkBA+U9wtYBeMIBAYYqIDyfqtUT0wSAIK0od1HFVguHIF9gpYCxqNRq12oS1VOZvMZ8jy3969nFFh37m2sUPWd+VqpuZ6dNe4CPMg5miblMA3IhzSMGyYyjAe3UKMM6Mes/6vG7idbj3yBWa3SkoVdJtV+jG2ObGLKgTXhoJ96JlvZxYUbgQhjJ5jSm5djZpASbVjm5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+y7c0xRLQIn/mz3YosVPpZeTLpEPg+jKsxvXPP5L8iP0KXxmZSdnhWa5koAKsuNlciTIxyl59dntD1OnTnTSnnXSxXRi9K+9Hm2DHPnlcQ01MmzUlO+tV73YuJyBxQXaODfWRwpKDr6kddmV82sV8StllJITs8MxQJx1zBHM+XqmdWZSiMXf1SIyq6h/h+3iHJ+ySpMaT1Dm6kuPWNXaZdepWtbapV1sV+fmBMWnvl84PSmBdR47zanFgTd+MnqlxJrwpMaOQJs+qQPVj7GwZ7YOXOarfNu8TpWg9DbYKjYplczYKxwZ/daii/b7m2NyfDzVtdpfNIpqYrO6+6DM6rDm3sV4RrN6iSDNINR6uzCiPflICrIvHjddTQgR930fst9H4/qj5tEXza5T8y9WdNk3uZPurSPGnTwLxP+ucoWznqq/Sj+zKI/O4B0jBvjr9CYomsF4BpJl9cch9h16ZwbHugpqjb8414aT8JQF0uUaXx8EHNXAO7UZm5Niq/KcyiLF0TbddE3VARxZrGecVqKuP+/yblv9WB/4vlTXJ3S64tJ5a1V3LPmtiznlOa2J7LXjaxL3tBF4RNPO1pQjgU58cXsYtOugF1F44X2nhw3jH/mdCwmIBHi6qAoXCZnCFwdAajP0UkVPsM+6yxPxWNQlUznaIIBly1MCf0DcW+8DA+CPr0DANPxjEWaedxVlIR3Njo1NRwsbASZIo5/B2zzGIT6httYu3s0UcSe3KFkkuffNO7H6/I6o/5fBUDKg1SCHy5MLzNafOIoLaVNkZJd8EMwsGwhQstwihaH+Q4M4zGVSg3RaX/nyhlP5jS1d9SV59UtbH1cKYfVIsWNhKwSjmrtnN9YPYtEmiZSvouBTl7SXjUgLbI8Y9npw3KRxbQIvJkJiO496JBd38Na5Gaoal7+v4l0A6Xl4KTF8J8f9KSvAa2qfkqlHGD8J7Zvyb28FiIOQKl+tsOKgLiFaAdWdHTM4LemlEYhyrXFb7b1EBEn0ye5igiM+eMR4W4fBXSkFCtyYIoinMK55Nrm+PuN7z9xdPJaLw7Gv/+9uZHe9W3v28Nfj24MdgejAd3Bh8NPh0cDZ4MvMGfBn8f/Gvw7+t/vP6X63+9/jdL+uYbFc+vBq2f6//8D/V/dzU=</latexit>

W = {(wj)1j=1 s.t. |wj | = µj 8j}

<latexit sha1_base64="CBpVtRddhdK/ayNAh00fK6khM2E=">AAAfMHicpVnrbtzGFd6kt1S9Oe3PosbEggAnWK21a6eOAywQS3Iujayolm/NciMMydklLZJDk0NplTFfqg9Q9DHaX0WB/irQPkO/M0OueJGTplViaebc5sy5z66bRmGudnb++sab3/v+D374o7d+vPGTn/7s57+49vYvn+ayyDzxxJORzJ67PBdRmIgnKlSReJ5mgsduJJ65p3uEf3YmsjyUyWN1kYp5zJdJuAg9rgA6ufbcibkKPB7pZyWbMkczdvP85MW7J/rFdFx+5YTJQl0w/DhKrJRm+UiNWMlegeYV0cfFyYsKVTJnITMeRewFc8qTa5s7ox3zw/qLcbXYHFQ/Rydv/+bPji+9IhaJ8iKe57PxTqrmmmcq9CJRbjhFLlLunfKlmGGZ8Fjkc21MULItQHyG8/EvUcxAmxyax3l+EbugpAvnXRwBr8LNCrX4YK7DJC2USDx70KKImJKM7Mn8MBOeii6w4F4WQlfmBTzjnoLVN+hnqyku5Sl542sx1eO7Xjyc3PHichjzbBkm08kkjodKptPxXSwCwX0wTt+P47leChkLlV2U7DvIG9/+zvI6120h9SpSK5VxGIEJWuTG4M/FAX8snrdJiyT0pC+2jV2vNPbG1lYHrIK4R2p81hbt8ZRitwPNC3cRLvvAq6lFgjjLuBIdOFLFRWbEdMV6zYw12nSUaWGyJDKvyJWM4QFm0q+ja/6ykErkhhCBCbMxhG6OODkTzOI6hksNlBiQSvIcbBlFWEUMZdaKtT3nSgU95nohpYrDnEKVQW9G+8Sw+vI8YVwxFQhmqZkNlLag/GxZ5daqTi6bTywNPVVkkHQeqoAdP/2EGcL2BZYZT4PQWxEbyocXFb6oYG3KWSJF4iPBo2WaiwK5j4Bpp3lWRMJSSKRWEE/aeA2QkjLKe9BtaKmse9tRVqjAjU47Xg+QyDDyog2emUvDzaf5kH5bG0yZGxViyIossgBAlpkQyZB5oRI1USb8IeOJF8isyTd/zVlaIkr9HpTCFELjdpxRWyBT5kIREiCdSFqBkdKdkhylc5jyrLG8RJRt3opzOgFdRXTnW7ibES7PkAgKGcagJNUD6OwK3J9RhhTQFSVRrHicRiLvlkORLHGbYK5d7oqIrhkXkQoNJ49CddHxXyqji2Uk8zw0VSgR5yJDnqDWJtykFMqyEUV8uCNMyiEw4iTPGNScR7y5ECxQKv3w1i1P8WRUhO4okbdiVHGZ31oJ5OqtiNPvxqHN9Sj1F9UxMUefRHrPHvI0he5TsG1Tsg8PwiU3OTNFiRweo9GKKZrfXB+ESbFiByFMpdC72RdUia0wmcirhVn2h1x5wYHEzT00/uGBULj+MSwkptsTEr0vXvCnBTvmSc4eQli51lIFJFjvYT7IQs4e1nW4gZzRYjtXFzipSLNwGajhGaoEQvx0Ow/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+duqpJVeZwIVKDbPcMY6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJ7ESY8Qi1MUL3M2AN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYQaasjFNLICeh6ikfXCwKL7+GnvkEaXCoWTnPEugXs64K1GJbbFlEvU74ARGQuSoxjzBrxBUsSwSNapsWWuFYJbnVRfLT8O01Nvj0cTo9zgQMhMxeyRSgbqey6gwVJ0kc/EXlXbqkjuGzPqWKqHMLgTPbOHqQrfhqHQInVeENXV+OjF7iLHb93eGrM4s4pRDFuZuMl3wKLcV0i6R3aFrUqnK2V7Ghi6QbWcARkYqm27pXAtz7qktz41KXdflKU70Tofrmlzt14Xc7hv1mNolxJKxsEWfJQNARUzStt/Vl9jodgmQeWqIEmy0NME3NxOCyIXXLt0AkIcYzWIUGRBlCW0ga6ciKDVVWj0ZYXCjzMDMRoIisVDMHjOsYt0VqLli2Iz8vtDC7cg1Um0t75N2qRFuRL8zep84wHKZxs5TxFws9CFptysODo6ZhVRdoEcs/FDJLNe/Q9A/MGt0SR9LBghtyx6LUVA/NkOLPLVTVp/qs+PdQ6jRg+9/8ZkFNyccinVkKo/SAM0CDWrYSIFLYDPY790btvMI4YvFVGWYBqqgMZtGtONU7vvYov6bB5sG0Qh7wuwL1CWxC9pQmvno4lhlFBPrJqW3NmDVSjbFJlTWe1DUZwghApTDLYbJaEkFDk+QG14TeaPDnffY82/kz29sbJUbrDWxGadiMrSXL9upoPEuishadD23cTFjWG0M+xqU62ZnkHk11sPoyJfSZJJdQmkEPPoBRguzjNPcC+2S2DaukJJGaNJNGd8q5Op6tI08sN2hjYUZkCOxwURCEYha4dTBcyeKNoh6HWTaGpLst57PTZGhNlFnDmtGshvVdLPxXAOJF7kr4LhlJovU7pv91cGQX3Po0lkvHdMzhcakcIfmys1xuWW5EakKIzg6kMh0TU/NprR4RH59ltG7/+5p9/eYn4o6VxmDiQz3a/qtnSvXl9KLqMgDqnYWaj99cBfaObj/6JMH2mEO/j+zV5mQ0R1MDYWiW5sz7bZck4CikmR/QxCasSB62/IMlu7YONoRVuVKyPgD41y6fPP23MVZaAC1AURyFmYyoY8xdI2rTFC3CBsSbMuhN101kTTx2omsdubSTi2FUmvEyi0yHGhN/7OiaC4wF6A5wuRuI+QJsdUIIP1Z8iGuGqeBrmq1MdmwZFSckf/uBXtteR5dVvkRO2Q3Jzvju+8OGaosOxyxLzAVsfueR3NNgatk5q2JgIpzJhdsb2979w/bh3vbd0Y7I6OX9Ype+4ZemiZ+6J17jkQNaD4QSAifKe4WsAtGEAgMMdGBZP3WqB4YJAEF6cM6jiowXLkCewWsBY1Go1a70JaqnE3mM2T5b+9dzqiw71zb2CHru3I1U3M9umdchHkQc7RNSuAbEQ5pGDZMZRiPbiPGmVGPWf/XDdxOtx75ArNbJaUKus0q/RjbnNhFFYJrQ8E+9My3MwsKN4IQRs8xJbeuRk2gpNpxk5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+ym5tjiGgRPvNnOxVZqPSz8mTSIfB9GFdjeueeyX9FfoQujc2k7PCs1jJRAFZdbK5EmBjl6g8y75cndL0O3XlTynkXy5XRi9J+tDl2zLPnFcT0lElzkpN+9V73YiIyB1TX6GAfGRwp6Lr6UVfml03sl4RtVlLIDs8MRcIxVzDH86XqmVUZCmNXv9SIiu4hvp93SPI+SWoMab2DGylufWOXaZdeZWubapV1sZ8fGJPWXvn8oDTmRdQ4rzYn1sTd+IkqV9KrAhMaecKsOmQP1v6GwR5YubPabfMucbrWw1Cb4KhYJlezYGzwZ7cbyu9brpubk+Hm7a7SeSRT05WdV10G51WHNvYrwrUbVEkG6Yaj1VmFkW9KwFWR+PE6aujAj7vo/RZ6vx9Vn7YIPu3yH5n6s6bJvUwfdWmetGlg3if9c5StHPVV+tF9GUR+9wBpmDfHXyGxRNYLwDSTLy65j7BrUzi2PVBT1O35Rry0nwSgLpeo0nj4oGaugd2ozFwbld8UZlGWrolu1kTdUBHFmsZ5xWoq4/7/JuW/1YH/i+VNcndLri0nlrVXcs+a2LOeU5rYnsteNrEve0EXhE087WlCOBTnxxexi066AXUXjhfaeHDeMf+Z0LCYgEeLqoChcJmcIXB0BqM/RSRU+wz7rLE/FY1CVTOdoggGXLUwJ/QNxb7wMD4I+vQMA0/GMRZp53FWUhHc2OjU1HCxsBJkijn8HbPMYhPqG21i7ezSRxK7coWSS59807sfr8jqj/l8FQMqDVIIfLkwvM1p84igtpU2Rkl3wQzCwbCFCy3CKFof5DgzjMZVKDdFpf+fKGU/mNLV31JXn1S1sfVwph9UixY2ErBKOau2c31g9i0SaJlK+i4FOXtJeNSAtsjxj2enDcpHFtAi8mQmI7j3okG3t4a1SM3Q1D19/xJoh8tLwckLYb4/aUleA9vUfBXKuEF43+xfE3t4LMQcgVL9bQcVAfEK0I6s6OkZQW/NKIxDlesK321qIKJPJk9zFJGZc8ajQly+CmlIqNZkQRTFOYXzybXNcfcb3v7i6WQ03hmNf39n86Pd6tvftwa/HtwY3ByMB3cHHw0+HRwNngy8wZ8G/xj8a/Dv63+8/pfrf7v+d0v65hsVz68GrZ/r//wP7S59LA==</latexit><latexit sha1_base64="CBpVtRddhdK/ayNAh00fK6khM2E=">AAAfMHicpVnrbtzGFd6kt1S9Oe3PosbEggAnWK21a6eOAywQS3Iujayolm/NciMMydklLZJDk0NplTFfqg9Q9DHaX0WB/irQPkO/M0OueJGTplViaebc5sy5z66bRmGudnb++sab3/v+D374o7d+vPGTn/7s57+49vYvn+ayyDzxxJORzJ67PBdRmIgnKlSReJ5mgsduJJ65p3uEf3YmsjyUyWN1kYp5zJdJuAg9rgA6ufbcibkKPB7pZyWbMkczdvP85MW7J/rFdFx+5YTJQl0w/DhKrJRm+UiNWMlegeYV0cfFyYsKVTJnITMeRewFc8qTa5s7ox3zw/qLcbXYHFQ/Rydv/+bPji+9IhaJ8iKe57PxTqrmmmcq9CJRbjhFLlLunfKlmGGZ8Fjkc21MULItQHyG8/EvUcxAmxyax3l+EbugpAvnXRwBr8LNCrX4YK7DJC2USDx70KKImJKM7Mn8MBOeii6w4F4WQlfmBTzjnoLVN+hnqyku5Sl542sx1eO7Xjyc3PHichjzbBkm08kkjodKptPxXSwCwX0wTt+P47leChkLlV2U7DvIG9/+zvI6120h9SpSK5VxGIEJWuTG4M/FAX8snrdJiyT0pC+2jV2vNPbG1lYHrIK4R2p81hbt8ZRitwPNC3cRLvvAq6lFgjjLuBIdOFLFRWbEdMV6zYw12nSUaWGyJDKvyJWM4QFm0q+ja/6ykErkhhCBCbMxhG6OODkTzOI6hksNlBiQSvIcbBlFWEUMZdaKtT3nSgU95nohpYrDnEKVQW9G+8Sw+vI8YVwxFQhmqZkNlLag/GxZ5daqTi6bTywNPVVkkHQeqoAdP/2EGcL2BZYZT4PQWxEbyocXFb6oYG3KWSJF4iPBo2WaiwK5j4Bpp3lWRMJSSKRWEE/aeA2QkjLKe9BtaKmse9tRVqjAjU47Xg+QyDDyog2emUvDzaf5kH5bG0yZGxViyIossgBAlpkQyZB5oRI1USb8IeOJF8isyTd/zVlaIkr9HpTCFELjdpxRWyBT5kIREiCdSFqBkdKdkhylc5jyrLG8RJRt3opzOgFdRXTnW7ibES7PkAgKGcagJNUD6OwK3J9RhhTQFSVRrHicRiLvlkORLHGbYK5d7oqIrhkXkQoNJ49CddHxXyqji2Uk8zw0VSgR5yJDnqDWJtykFMqyEUV8uCNMyiEw4iTPGNScR7y5ECxQKv3w1i1P8WRUhO4okbdiVHGZ31oJ5OqtiNPvxqHN9Sj1F9UxMUefRHrPHvI0he5TsG1Tsg8PwiU3OTNFiRweo9GKKZrfXB+ESbFiByFMpdC72RdUia0wmcirhVn2h1x5wYHEzT00/uGBULj+MSwkptsTEr0vXvCnBTvmSc4eQli51lIFJFjvYT7IQs4e1nW4gZzRYjtXFzipSLNwGajhGaoEQvx0Ow/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+duqpJVeZwIVKDbPcMY6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJ7ESY8Qi1MUL3M2AN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYQaasjFNLICeh6ikfXCwKL7+GnvkEaXCoWTnPEugXs64K1GJbbFlEvU74ARGQuSoxjzBrxBUsSwSNapsWWuFYJbnVRfLT8O01Nvj0cTo9zgQMhMxeyRSgbqey6gwVJ0kc/EXlXbqkjuGzPqWKqHMLgTPbOHqQrfhqHQInVeENXV+OjF7iLHb93eGrM4s4pRDFuZuMl3wKLcV0i6R3aFrUqnK2V7Ghi6QbWcARkYqm27pXAtz7qktz41KXdflKU70Tofrmlzt14Xc7hv1mNolxJKxsEWfJQNARUzStt/Vl9jodgmQeWqIEmy0NME3NxOCyIXXLt0AkIcYzWIUGRBlCW0ga6ciKDVVWj0ZYXCjzMDMRoIisVDMHjOsYt0VqLli2Iz8vtDC7cg1Um0t75N2qRFuRL8zep84wHKZxs5TxFws9CFptysODo6ZhVRdoEcs/FDJLNe/Q9A/MGt0SR9LBghtyx6LUVA/NkOLPLVTVp/qs+PdQ6jRg+9/8ZkFNyccinVkKo/SAM0CDWrYSIFLYDPY790btvMI4YvFVGWYBqqgMZtGtONU7vvYov6bB5sG0Qh7wuwL1CWxC9pQmvno4lhlFBPrJqW3NmDVSjbFJlTWe1DUZwghApTDLYbJaEkFDk+QG14TeaPDnffY82/kz29sbJUbrDWxGadiMrSXL9upoPEuishadD23cTFjWG0M+xqU62ZnkHk11sPoyJfSZJJdQmkEPPoBRguzjNPcC+2S2DaukJJGaNJNGd8q5Op6tI08sN2hjYUZkCOxwURCEYha4dTBcyeKNoh6HWTaGpLst57PTZGhNlFnDmtGshvVdLPxXAOJF7kr4LhlJovU7pv91cGQX3Po0lkvHdMzhcakcIfmys1xuWW5EakKIzg6kMh0TU/NprR4RH59ltG7/+5p9/eYn4o6VxmDiQz3a/qtnSvXl9KLqMgDqnYWaj99cBfaObj/6JMH2mEO/j+zV5mQ0R1MDYWiW5sz7bZck4CikmR/QxCasSB62/IMlu7YONoRVuVKyPgD41y6fPP23MVZaAC1AURyFmYyoY8xdI2rTFC3CBsSbMuhN101kTTx2omsdubSTi2FUmvEyi0yHGhN/7OiaC4wF6A5wuRuI+QJsdUIIP1Z8iGuGqeBrmq1MdmwZFSckf/uBXtteR5dVvkRO2Q3Jzvju+8OGaosOxyxLzAVsfueR3NNgatk5q2JgIpzJhdsb2979w/bh3vbd0Y7I6OX9Ype+4ZemiZ+6J17jkQNaD4QSAifKe4WsAtGEAgMMdGBZP3WqB4YJAEF6cM6jiowXLkCewWsBY1Go1a70JaqnE3mM2T5b+9dzqiw71zb2CHru3I1U3M9umdchHkQc7RNSuAbEQ5pGDZMZRiPbiPGmVGPWf/XDdxOtx75ArNbJaUKus0q/RjbnNhFFYJrQ8E+9My3MwsKN4IQRs8xJbeuRk2gpNpxk5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+ym5tjiGgRPvNnOxVZqPSz8mTSIfB9GFdjeueeyX9FfoQujc2k7PCs1jJRAFZdbK5EmBjl6g8y75cndL0O3XlTynkXy5XRi9J+tDl2zLPnFcT0lElzkpN+9V73YiIyB1TX6GAfGRwp6Lr6UVfml03sl4RtVlLIDs8MRcIxVzDH86XqmVUZCmNXv9SIiu4hvp93SPI+SWoMab2DGylufWOXaZdeZWubapV1sZ8fGJPWXvn8oDTmRdQ4rzYn1sTd+IkqV9KrAhMaecKsOmQP1v6GwR5YubPabfMucbrWw1Cb4KhYJlezYGzwZ7cbyu9brpubk+Hm7a7SeSRT05WdV10G51WHNvYrwrUbVEkG6Yaj1VmFkW9KwFWR+PE6aujAj7vo/RZ6vx9Vn7YIPu3yH5n6s6bJvUwfdWmetGlg3if9c5StHPVV+tF9GUR+9wBpmDfHXyGxRNYLwDSTLy65j7BrUzi2PVBT1O35Rry0nwSgLpeo0nj4oGaugd2ozFwbld8UZlGWrolu1kTdUBHFmsZ5xWoq4/7/JuW/1YH/i+VNcndLri0nlrVXcs+a2LOeU5rYnsteNrEve0EXhE087WlCOBTnxxexi066AXUXjhfaeHDeMf+Z0LCYgEeLqoChcJmcIXB0BqM/RSRU+wz7rLE/FY1CVTOdoggGXLUwJ/QNxb7wMD4I+vQMA0/GMRZp53FWUhHc2OjU1HCxsBJkijn8HbPMYhPqG21i7ezSRxK7coWSS59807sfr8jqj/l8FQMqDVIIfLkwvM1p84igtpU2Rkl3wQzCwbCFCy3CKFof5DgzjMZVKDdFpf+fKGU/mNLV31JXn1S1sfVwph9UixY2ErBKOau2c31g9i0SaJlK+i4FOXtJeNSAtsjxj2enDcpHFtAi8mQmI7j3okG3t4a1SM3Q1D19/xJoh8tLwckLYb4/aUleA9vUfBXKuEF43+xfE3t4LMQcgVL9bQcVAfEK0I6s6OkZQW/NKIxDlesK321qIKJPJk9zFJGZc8ajQly+CmlIqNZkQRTFOYXzybXNcfcb3v7i6WQ03hmNf39n86Pd6tvftwa/HtwY3ByMB3cHHw0+HRwNngy8wZ8G/xj8a/Dv63+8/pfrf7v+d0v65hsVz68GrZ/r//wP7S59LA==</latexit><latexit sha1_base64="CBpVtRddhdK/ayNAh00fK6khM2E=">AAAfMHicpVnrbtzGFd6kt1S9Oe3PosbEggAnWK21a6eOAywQS3Iujayolm/NciMMydklLZJDk0NplTFfqg9Q9DHaX0WB/irQPkO/M0OueJGTplViaebc5sy5z66bRmGudnb++sab3/v+D374o7d+vPGTn/7s57+49vYvn+ayyDzxxJORzJ67PBdRmIgnKlSReJ5mgsduJJ65p3uEf3YmsjyUyWN1kYp5zJdJuAg9rgA6ufbcibkKPB7pZyWbMkczdvP85MW7J/rFdFx+5YTJQl0w/DhKrJRm+UiNWMlegeYV0cfFyYsKVTJnITMeRewFc8qTa5s7ox3zw/qLcbXYHFQ/Rydv/+bPji+9IhaJ8iKe57PxTqrmmmcq9CJRbjhFLlLunfKlmGGZ8Fjkc21MULItQHyG8/EvUcxAmxyax3l+EbugpAvnXRwBr8LNCrX4YK7DJC2USDx70KKImJKM7Mn8MBOeii6w4F4WQlfmBTzjnoLVN+hnqyku5Sl542sx1eO7Xjyc3PHichjzbBkm08kkjodKptPxXSwCwX0wTt+P47leChkLlV2U7DvIG9/+zvI6120h9SpSK5VxGIEJWuTG4M/FAX8snrdJiyT0pC+2jV2vNPbG1lYHrIK4R2p81hbt8ZRitwPNC3cRLvvAq6lFgjjLuBIdOFLFRWbEdMV6zYw12nSUaWGyJDKvyJWM4QFm0q+ja/6ykErkhhCBCbMxhG6OODkTzOI6hksNlBiQSvIcbBlFWEUMZdaKtT3nSgU95nohpYrDnEKVQW9G+8Sw+vI8YVwxFQhmqZkNlLag/GxZ5daqTi6bTywNPVVkkHQeqoAdP/2EGcL2BZYZT4PQWxEbyocXFb6oYG3KWSJF4iPBo2WaiwK5j4Bpp3lWRMJSSKRWEE/aeA2QkjLKe9BtaKmse9tRVqjAjU47Xg+QyDDyog2emUvDzaf5kH5bG0yZGxViyIossgBAlpkQyZB5oRI1USb8IeOJF8isyTd/zVlaIkr9HpTCFELjdpxRWyBT5kIREiCdSFqBkdKdkhylc5jyrLG8RJRt3opzOgFdRXTnW7ibES7PkAgKGcagJNUD6OwK3J9RhhTQFSVRrHicRiLvlkORLHGbYK5d7oqIrhkXkQoNJ49CddHxXyqji2Uk8zw0VSgR5yJDnqDWJtykFMqyEUV8uCNMyiEw4iTPGNScR7y5ECxQKv3w1i1P8WRUhO4okbdiVHGZ31oJ5OqtiNPvxqHN9Sj1F9UxMUefRHrPHvI0he5TsG1Tsg8PwiU3OTNFiRweo9GKKZrfXB+ESbFiByFMpdC72RdUia0wmcirhVn2h1x5wYHEzT00/uGBULj+MSwkptsTEr0vXvCnBTvmSc4eQli51lIFJFjvYT7IQs4e1nW4gZzRYjtXFzipSLNwGajhGaoEQvx0Ow/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+duqpJVeZwIVKDbPcMY6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJ7ESY8Qi1MUL3M2AN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYQaasjFNLICeh6ikfXCwKL7+GnvkEaXCoWTnPEugXs64K1GJbbFlEvU74ARGQuSoxjzBrxBUsSwSNapsWWuFYJbnVRfLT8O01Nvj0cTo9zgQMhMxeyRSgbqey6gwVJ0kc/EXlXbqkjuGzPqWKqHMLgTPbOHqQrfhqHQInVeENXV+OjF7iLHb93eGrM4s4pRDFuZuMl3wKLcV0i6R3aFrUqnK2V7Ghi6QbWcARkYqm27pXAtz7qktz41KXdflKU70Tofrmlzt14Xc7hv1mNolxJKxsEWfJQNARUzStt/Vl9jodgmQeWqIEmy0NME3NxOCyIXXLt0AkIcYzWIUGRBlCW0ga6ciKDVVWj0ZYXCjzMDMRoIisVDMHjOsYt0VqLli2Iz8vtDC7cg1Um0t75N2qRFuRL8zep84wHKZxs5TxFws9CFptysODo6ZhVRdoEcs/FDJLNe/Q9A/MGt0SR9LBghtyx6LUVA/NkOLPLVTVp/qs+PdQ6jRg+9/8ZkFNyccinVkKo/SAM0CDWrYSIFLYDPY790btvMI4YvFVGWYBqqgMZtGtONU7vvYov6bB5sG0Qh7wuwL1CWxC9pQmvno4lhlFBPrJqW3NmDVSjbFJlTWe1DUZwghApTDLYbJaEkFDk+QG14TeaPDnffY82/kz29sbJUbrDWxGadiMrSXL9upoPEuishadD23cTFjWG0M+xqU62ZnkHk11sPoyJfSZJJdQmkEPPoBRguzjNPcC+2S2DaukJJGaNJNGd8q5Op6tI08sN2hjYUZkCOxwURCEYha4dTBcyeKNoh6HWTaGpLst57PTZGhNlFnDmtGshvVdLPxXAOJF7kr4LhlJovU7pv91cGQX3Po0lkvHdMzhcakcIfmys1xuWW5EakKIzg6kMh0TU/NprR4RH59ltG7/+5p9/eYn4o6VxmDiQz3a/qtnSvXl9KLqMgDqnYWaj99cBfaObj/6JMH2mEO/j+zV5mQ0R1MDYWiW5sz7bZck4CikmR/QxCasSB62/IMlu7YONoRVuVKyPgD41y6fPP23MVZaAC1AURyFmYyoY8xdI2rTFC3CBsSbMuhN101kTTx2omsdubSTi2FUmvEyi0yHGhN/7OiaC4wF6A5wuRuI+QJsdUIIP1Z8iGuGqeBrmq1MdmwZFSckf/uBXtteR5dVvkRO2Q3Jzvju+8OGaosOxyxLzAVsfueR3NNgatk5q2JgIpzJhdsb2979w/bh3vbd0Y7I6OX9Ype+4ZemiZ+6J17jkQNaD4QSAifKe4WsAtGEAgMMdGBZP3WqB4YJAEF6cM6jiowXLkCewWsBY1Go1a70JaqnE3mM2T5b+9dzqiw71zb2CHru3I1U3M9umdchHkQc7RNSuAbEQ5pGDZMZRiPbiPGmVGPWf/XDdxOtx75ArNbJaUKus0q/RjbnNhFFYJrQ8E+9My3MwsKN4IQRs8xJbeuRk2gpNpxk5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+ym5tjiGgRPvNnOxVZqPSz8mTSIfB9GFdjeueeyX9FfoQujc2k7PCs1jJRAFZdbK5EmBjl6g8y75cndL0O3XlTynkXy5XRi9J+tDl2zLPnFcT0lElzkpN+9V73YiIyB1TX6GAfGRwp6Lr6UVfml03sl4RtVlLIDs8MRcIxVzDH86XqmVUZCmNXv9SIiu4hvp93SPI+SWoMab2DGylufWOXaZdeZWubapV1sZ8fGJPWXvn8oDTmRdQ4rzYn1sTd+IkqV9KrAhMaecKsOmQP1v6GwR5YubPabfMucbrWw1Cb4KhYJlezYGzwZ7cbyu9brpubk+Hm7a7SeSRT05WdV10G51WHNvYrwrUbVEkG6Yaj1VmFkW9KwFWR+PE6aujAj7vo/RZ6vx9Vn7YIPu3yH5n6s6bJvUwfdWmetGlg3if9c5StHPVV+tF9GUR+9wBpmDfHXyGxRNYLwDSTLy65j7BrUzi2PVBT1O35Rry0nwSgLpeo0nj4oGaugd2ozFwbld8UZlGWrolu1kTdUBHFmsZ5xWoq4/7/JuW/1YH/i+VNcndLri0nlrVXcs+a2LOeU5rYnsteNrEve0EXhE087WlCOBTnxxexi066AXUXjhfaeHDeMf+Z0LCYgEeLqoChcJmcIXB0BqM/RSRU+wz7rLE/FY1CVTOdoggGXLUwJ/QNxb7wMD4I+vQMA0/GMRZp53FWUhHc2OjU1HCxsBJkijn8HbPMYhPqG21i7ezSRxK7coWSS59807sfr8jqj/l8FQMqDVIIfLkwvM1p84igtpU2Rkl3wQzCwbCFCy3CKFof5DgzjMZVKDdFpf+fKGU/mNLV31JXn1S1sfVwph9UixY2ErBKOau2c31g9i0SaJlK+i4FOXtJeNSAtsjxj2enDcpHFtAi8mQmI7j3okG3t4a1SM3Q1D19/xJoh8tLwckLYb4/aUleA9vUfBXKuEF43+xfE3t4LMQcgVL9bQcVAfEK0I6s6OkZQW/NKIxDlesK321qIKJPJk9zFJGZc8ajQly+CmlIqNZkQRTFOYXzybXNcfcb3v7i6WQ03hmNf39n86Pd6tvftwa/HtwY3ByMB3cHHw0+HRwNngy8wZ8G/xj8a/Dv63+8/pfrf7v+d0v65hsVz68GrZ/r//wP7S59LA==</latexit><latexit sha1_base64="CBpVtRddhdK/ayNAh00fK6khM2E=">AAAfMHicpVnrbtzGFd6kt1S9Oe3PosbEggAnWK21a6eOAywQS3Iujayolm/NciMMydklLZJDk0NplTFfqg9Q9DHaX0WB/irQPkO/M0OueJGTplViaebc5sy5z66bRmGudnb++sab3/v+D374o7d+vPGTn/7s57+49vYvn+ayyDzxxJORzJ67PBdRmIgnKlSReJ5mgsduJJ65p3uEf3YmsjyUyWN1kYp5zJdJuAg9rgA6ufbcibkKPB7pZyWbMkczdvP85MW7J/rFdFx+5YTJQl0w/DhKrJRm+UiNWMlegeYV0cfFyYsKVTJnITMeRewFc8qTa5s7ox3zw/qLcbXYHFQ/Rydv/+bPji+9IhaJ8iKe57PxTqrmmmcq9CJRbjhFLlLunfKlmGGZ8Fjkc21MULItQHyG8/EvUcxAmxyax3l+EbugpAvnXRwBr8LNCrX4YK7DJC2USDx70KKImJKM7Mn8MBOeii6w4F4WQlfmBTzjnoLVN+hnqyku5Sl542sx1eO7Xjyc3PHichjzbBkm08kkjodKptPxXSwCwX0wTt+P47leChkLlV2U7DvIG9/+zvI6120h9SpSK5VxGIEJWuTG4M/FAX8snrdJiyT0pC+2jV2vNPbG1lYHrIK4R2p81hbt8ZRitwPNC3cRLvvAq6lFgjjLuBIdOFLFRWbEdMV6zYw12nSUaWGyJDKvyJWM4QFm0q+ja/6ykErkhhCBCbMxhG6OODkTzOI6hksNlBiQSvIcbBlFWEUMZdaKtT3nSgU95nohpYrDnEKVQW9G+8Sw+vI8YVwxFQhmqZkNlLag/GxZ5daqTi6bTywNPVVkkHQeqoAdP/2EGcL2BZYZT4PQWxEbyocXFb6oYG3KWSJF4iPBo2WaiwK5j4Bpp3lWRMJSSKRWEE/aeA2QkjLKe9BtaKmse9tRVqjAjU47Xg+QyDDyog2emUvDzaf5kH5bG0yZGxViyIossgBAlpkQyZB5oRI1USb8IeOJF8isyTd/zVlaIkr9HpTCFELjdpxRWyBT5kIREiCdSFqBkdKdkhylc5jyrLG8RJRt3opzOgFdRXTnW7ibES7PkAgKGcagJNUD6OwK3J9RhhTQFSVRrHicRiLvlkORLHGbYK5d7oqIrhkXkQoNJ49CddHxXyqji2Uk8zw0VSgR5yJDnqDWJtykFMqyEUV8uCNMyiEw4iTPGNScR7y5ECxQKv3w1i1P8WRUhO4okbdiVHGZ31oJ5OqtiNPvxqHN9Sj1F9UxMUefRHrPHvI0he5TsG1Tsg8PwiU3OTNFiRweo9GKKZrfXB+ESbFiByFMpdC72RdUia0wmcirhVn2h1x5wYHEzT00/uGBULj+MSwkptsTEr0vXvCnBTvmSc4eQli51lIFJFjvYT7IQs4e1nW4gZzRYjtXFzipSLNwGajhGaoEQvx0Ow/QbqZVbZ/rQyHZA6Ro1juhfzv4PRPwlyfjmCe+duqpJVeZwIVKDbPcMY6xADRUpHDioVYJdY4EY2sOyFpAmBfkuDUsBCbEWwFSAgjGF7AI1VnqYWJ7ESY8Qi1MUL3M2AN+H57VVSJEiAoVaAdhHiY+uKALcg5IFC83xaQRYQaasjFNLICeh6ikfXCwKL7+GnvkEaXCoWTnPEugXs64K1GJbbFlEvU74ARGQuSoxjzBrxBUsSwSNapsWWuFYJbnVRfLT8O01Nvj0cTo9zgQMhMxeyRSgbqey6gwVJ0kc/EXlXbqkjuGzPqWKqHMLgTPbOHqQrfhqHQInVeENXV+OjF7iLHb93eGrM4s4pRDFuZuMl3wKLcV0i6R3aFrUqnK2V7Ghi6QbWcARkYqm27pXAtz7qktz41KXdflKU70Tofrmlzt14Xc7hv1mNolxJKxsEWfJQNARUzStt/Vl9jodgmQeWqIEmy0NME3NxOCyIXXLt0AkIcYzWIUGRBlCW0ga6ciKDVVWj0ZYXCjzMDMRoIisVDMHjOsYt0VqLli2Iz8vtDC7cg1Um0t75N2qRFuRL8zep84wHKZxs5TxFws9CFptysODo6ZhVRdoEcs/FDJLNe/Q9A/MGt0SR9LBghtyx6LUVA/NkOLPLVTVp/qs+PdQ6jRg+9/8ZkFNyccinVkKo/SAM0CDWrYSIFLYDPY790btvMI4YvFVGWYBqqgMZtGtONU7vvYov6bB5sG0Qh7wuwL1CWxC9pQmvno4lhlFBPrJqW3NmDVSjbFJlTWe1DUZwghApTDLYbJaEkFDk+QG14TeaPDnffY82/kz29sbJUbrDWxGadiMrSXL9upoPEuishadD23cTFjWG0M+xqU62ZnkHk11sPoyJfSZJJdQmkEPPoBRguzjNPcC+2S2DaukJJGaNJNGd8q5Op6tI08sN2hjYUZkCOxwURCEYha4dTBcyeKNoh6HWTaGpLst57PTZGhNlFnDmtGshvVdLPxXAOJF7kr4LhlJovU7pv91cGQX3Po0lkvHdMzhcakcIfmys1xuWW5EakKIzg6kMh0TU/NprR4RH59ltG7/+5p9/eYn4o6VxmDiQz3a/qtnSvXl9KLqMgDqnYWaj99cBfaObj/6JMH2mEO/j+zV5mQ0R1MDYWiW5sz7bZck4CikmR/QxCasSB62/IMlu7YONoRVuVKyPgD41y6fPP23MVZaAC1AURyFmYyoY8xdI2rTFC3CBsSbMuhN101kTTx2omsdubSTi2FUmvEyi0yHGhN/7OiaC4wF6A5wuRuI+QJsdUIIP1Z8iGuGqeBrmq1MdmwZFSckf/uBXtteR5dVvkRO2Q3Jzvju+8OGaosOxyxLzAVsfueR3NNgatk5q2JgIpzJhdsb2979w/bh3vbd0Y7I6OX9Ype+4ZemiZ+6J17jkQNaD4QSAifKe4WsAtGEAgMMdGBZP3WqB4YJAEF6cM6jiowXLkCewWsBY1Go1a70JaqnE3mM2T5b+9dzqiw71zb2CHru3I1U3M9umdchHkQc7RNSuAbEQ5pGDZMZRiPbiPGmVGPWf/XDdxOtx75ArNbJaUKus0q/RjbnNhFFYJrQ8E+9My3MwsKN4IQRs8xJbeuRk2gpNpxk5Yk910Ud1wJ+mTMN83Ir4W2eN9zHoZZRqzmtrmnaV+ym5tjiGgRPvNnOxVZqPSz8mTSIfB9GFdjeueeyX9FfoQujc2k7PCs1jJRAFZdbK5EmBjl6g8y75cndL0O3XlTynkXy5XRi9J+tDl2zLPnFcT0lElzkpN+9V73YiIyB1TX6GAfGRwp6Lr6UVfml03sl4RtVlLIDs8MRcIxVzDH86XqmVUZCmNXv9SIiu4hvp93SPI+SWoMab2DGylufWOXaZdeZWubapV1sZ8fGJPWXvn8oDTmRdQ4rzYn1sTd+IkqV9KrAhMaecKsOmQP1v6GwR5YubPabfMucbrWw1Cb4KhYJlezYGzwZ7cbyu9brpubk+Hm7a7SeSRT05WdV10G51WHNvYrwrUbVEkG6Yaj1VmFkW9KwFWR+PE6aujAj7vo/RZ6vx9Vn7YIPu3yH5n6s6bJvUwfdWmetGlg3if9c5StHPVV+tF9GUR+9wBpmDfHXyGxRNYLwDSTLy65j7BrUzi2PVBT1O35Rry0nwSgLpeo0nj4oGaugd2ozFwbld8UZlGWrolu1kTdUBHFmsZ5xWoq4/7/JuW/1YH/i+VNcndLri0nlrVXcs+a2LOeU5rYnsteNrEve0EXhE087WlCOBTnxxexi066AXUXjhfaeHDeMf+Z0LCYgEeLqoChcJmcIXB0BqM/RSRU+wz7rLE/FY1CVTOdoggGXLUwJ/QNxb7wMD4I+vQMA0/GMRZp53FWUhHc2OjU1HCxsBJkijn8HbPMYhPqG21i7ezSRxK7coWSS59807sfr8jqj/l8FQMqDVIIfLkwvM1p84igtpU2Rkl3wQzCwbCFCy3CKFof5DgzjMZVKDdFpf+fKGU/mNLV31JXn1S1sfVwph9UixY2ErBKOau2c31g9i0SaJlK+i4FOXtJeNSAtsjxj2enDcpHFtAi8mQmI7j3okG3t4a1SM3Q1D19/xJoh8tLwckLYb4/aUleA9vUfBXKuEF43+xfE3t4LMQcgVL9bQcVAfEK0I6s6OkZQW/NKIxDlesK321qIKJPJk9zFJGZc8ajQly+CmlIqNZkQRTFOYXzybXNcfcb3v7i6WQ03hmNf39n86Pd6tvftwa/HtwY3ByMB3cHHw0+HRwNngy8wZ8G/xj8a/Dv63+8/pfrf7v+d0v65hsVz68GrZ/r//wP7S59LA==</latexit>

Page 97: Online Learning with Kernel Losses11-11-00)-11... · 2019-06-11 · Online Learning with Kernel Losses Aldo Pacchiano UC Berkeley Joint work with Niladri Chatterji and Peter Bartlett

Final thoughts

30

Thanks for your attention