數值方法 2008, Applied Mathematics NDHU 1 Nonlinear Recursive relations Nonlinear recursive...

數值方法 2008, Applied Mathematics NDHU 1

Nonlinear Recursive relations

Nonlinear recursive relationsTime series predictionHill-Valley classificationLorenz series

Neural Recursions

數值方法 2008, Applied Mathematics NDHU 2

Nonlinear recursion

Post-nonlinear combination of predecessors

z[t]=tanh(a1z[t-1]+ a2z[t-2]+…+ az[t-])+ e[t],

t= ,…,N

數值方法 2008, Applied Mathematics NDHU 3

Post-nonlinear Projection

tanhy

數值方法 2008, Applied Mathematics NDHU 4

Data generation

z[t]=tanh(a1z[t-1]+ a2z[t-2]+…+ az[t-])+ e[t],

t= ,…,N

Taa ],...,[ 1 a

tanh

][tz

數值方法 2008, Applied Mathematics NDHU 5

Nonlinear Recursion

LM learning

數值方法 2008, Applied Mathematics NDHU 6

Prediction

Use initial -1 instances to generate the full time series (red) based on estimated post-nonlinear filter

Post-nonlinear filter

數值方法 2008, Applied Mathematics NDHU 7

UCI Machine Learning Repository

Nonlinear recursions: hill-valley

Classify hill and valley

數值方法 2008, Applied Mathematics NDHU 8

Noise-free Hill Valley

數值方法 2008, Applied Mathematics NDHU 9

Noise-free Hill Valley

數值方法 2008, Applied Mathematics NDHU 10

Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 11

Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 12

Hill-Valley with noise

數值方法 2008, Applied Mathematics NDHU 13

Nonlinear recursion

LM_RBF: z[t]=F(z[t-1], z[t-2], …,z[t-])+ e[t],

t= ,…,N

][tzLM_RBF 1

數值方法 2008, Applied Mathematics NDHU 14

][tzLM_RBF 1

][tzLM_RBF 2

數值方法 2008, Applied Mathematics NDHU 15

Nonlinear Recursion

LM learning

MLPotts 1 (H)

數值方法 2008, Applied Mathematics NDHU 16

Nonlinear Recursion

LM learning

MLPotts 2 (V)

數值方法 2008, Applied Mathematics NDHU 17

Approximating error

LM_RBF 1 (H)

Green: Original HillBlue: Approximated HillRed: Approximating error

數值方法 2008, Applied Mathematics NDHU 18

Approximating error

LM_RBF 2 (V)

Green: Original HillBlue: Approximated HillRed: Approximating error

數值方法 2008, Applied Mathematics NDHU 19

Separable 2D diagram of Hill-Valley

Noise-free data set 1

LM_RBF 2 (V)

LM_RBF 1 (H)

Mean Approximating Errors ofTwo Models

數值方法 2008, Applied Mathematics NDHU 20

Separable 2D diagram of Hill-Valley

Noise-free data set 2

Mean Approximating Errors ofTwo Models

LM_RBF 2 (V)

LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 21

Separable 2D diagram of Hill-Valley

Dataset 1 (Noise)

Mean Approximating Errors ofTwo Models

LM_RBF 2 (V)

LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 22

Separable 2D diagram of Hill-Valley

Dataset 2 (Noise)

Mean Approximating Errors ofTwo Models

LM_RBF 2 (V)

LM_RBF 1 (H)

數值方法 2008, Applied Mathematics NDHU 23

Chaos Time Series

Eric's Home Page

數值方法 2008, Applied Mathematics NDHU 24

Lorenz Attractor

數值方法 2008, Applied Mathematics NDHU 25

Lorenz

>> NN=20000;>> [x,y,z] = lorenz(NN);>> plot3(x,y,z,'.');

3

8:

28:

10:

lz

ly

lx

數值方法 2008, Applied Mathematics NDHU 26

3

8:

13:

10:

lz

ly

lx

13

數值方法 2008, Applied Mathematics NDHU 27

Nonlinear Recursion

Neural nonlinear recursive relations

數值方法 2008, Applied Mathematics NDHU 28

Prediction

Use initial -1 instances to generate the full time series (red) based on derived neural recursions

數值方法 2008, Applied Mathematics NDHU 29

ρ=14, σ=10, β=8/3 ρ=13, σ=10, β=8/3

ρ=15, σ=10, β=8/3 ρ=28, σ=10, β=8/3

Rayleigh number

數值方法 2008, Applied Mathematics NDHU 30

Sunspot time series

1700 1750 1800 1850 1900 19500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

sunpot data

load sunspot_train.mat;n=length(Y);plot(1712:1712+n-1,Y);

數值方法 2008, Applied Mathematics NDHU 31

Sunspot time series

1700 1750 1800 1850 1900 19500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

sunpot data (testing)

load sunspot_test_1.mat;n=length(Y);plot(1952:1952+n-1,Y);

數值方法 2008, Applied Mathematics NDHU 32

1700 1750 1800 1850 1900 19500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

S[1]S[2]S[3]..S[-1]S[]..S[t-]S[t- +1]..S[t-1]S[t]S[t+1]...

x[t]

y[t]

x[-1]

y[-1]

Time-series 2 paired data

數值方法 2008, Applied Mathematics NDHU 33

aM

a2

a1

x

Network

tanh

tanh

tanh

…….

1 b2

b1

bM

1

r1

r2

rM

rM+1

y

數值方法 2008, Applied Mathematics NDHU 34

Paired data : tau=2

x=[];len=3;for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len);endfigure;plot3(x(2,:),x(1,:),y,'.');hold on;

00.2

0.40.6

0.81

0

0.5

10

0.2

0.4

0.6

0.8

1y

x

數值方法 2008, Applied Mathematics NDHU 35

Learning and prediction

M=input('keyin the number of hidden units:');[a,b,r]=learn_MLP(x',y',M);y=eval_MLP2(x,r,a,b,M);hold on;plot(1712+2:1712+2+length(y)-1,y,'r')

MSE for training data 0.005029ME for training data 0.052191K

數值方法 2008, Applied Mathematics NDHU 36

Prediction

1700 1750 1800 1850 1900 1950-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

數值方法 2008, Applied Mathematics NDHU 37

Sunspot

Source codes

數值方法 2008, Applied Mathematics NDHU 38

Paired data: tau=5

x=[];len=5;for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len);end

數值方法 2008, Applied Mathematics NDHU 39

Learning and prediction

M=input('keyin the number of hidden units:');[a,b,r]=learn_MLP(x',y',M);y=eval_MLP2(x,r,a,b,M);hold on;plot(1712+2:1712+2+length(y)-1,y,'r')

???

數值方法 2008, Applied Mathematics NDHU 40

Example

Repeat numerical experiments in slide #34 with

Summarize your numerical results in a table

20,15,10,5, 2 M

20,15,10,5, 5 M

20,15,10,5, 10 M

數值方法 2008, Applied Mathematics NDHU 41

RBF

y

exp exp exp exp

x

2

1

2

1

2μx

2

2

2 i

i

μx

2

1

2

1

2

M

M

μx

2

2

2 M

M

μx .. ..

0w1w iw 1Mw

Mw

數值方法 2008, Applied Mathematics NDHU 42

addpath('..\..\..\Methods\FunAppr\LM_RBF');% x : Nxd% y : 1xNN=200;d=2;x=rand(N,d)*2-1;y=tanh(-x(:,1).^2-x(:,2).^2);eval(['load data\sunspot_train.mat']); %([1 2 4 6 7 10],:)tx = X'; ty = Y;Net=LM_RBF(tx,ty);[yhat,D]=evaRBF(tx,Net);mean((ty-yhat).^2)/2

數值方法 2008, Applied Mathematics NDHU 43

eval(['load data\sunspot_test_1.mat']); tex1 = X'; tey1 = Y; [yhat,D]=evaRBF(tex1,Net); mean((tey1-yhat).^2)/2

One_step_look_ahead prediction