Trend and System Identification With Orthogonal Basis...

Trend and System Identification With Orthogonal Basis Function

OSE SEMINAR 2013

ÄMIR SHIRDEL

CENTER OF EXCELLENCE IN

OPTIMIZATION AND SYSTEMS ENGINEERING

AT ÅBO AKADEMI UNIVERSITY

ÅBO NOVEMBER 15 2013

Background

– System identification is difficult when process measurements are

corrupted by structured disturbances, such as trends, outliers, level

shifts

– Standard approach: removal by data preprocessing but difficult to

separate between the effects of known system inputs and unknown

disturbances (trends, etc.)

– Orthonormal basis function models are categorized as output-error

(ballistic simulation) models

2|N

Background

Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

2|22

Present contribution

– Identification of system model parameters and disturbances

simultaneously

– Sparse optimization used in system identification problem

– Applying the method on simulated and real example

3|N

Present contribution

Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

3|22

4

4 4

Present contribution

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

4|22

Present contribution

SYSTEN

(model) U Y

0 100 200 300 400 500 600 700 800 900 1000-10

-8

-6

-4

-2

0

2

4

6

8

d

0 100 200 300 400 500 600 700 800 900 1000-25

-20

-15

-10

-5

0

5

10

15

20

0 100 200 300 400 500 600 700 800 900 1000-4

-3

-2

-1

0

1

2

3

4

5

Orthogonal basis functions has some advantages:

• The corresponding approximation (representation) has simple and direct

solution.

• It corresponds to allpass filters which is robust to implement and use in

numerical computation.

• It is popular because a few parameters can describe the system.

• It is a kind of output-error model, and can be insensitive to noise.

5 5

Orthogonal basis function (Fixed-pole model)

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

5|22

Orthogonal basis function

6 6 6

Orthogonal basis function (Fixed-pole model)

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

6|22

Orthogonal basis function

FIR network:

Laguerre function:

Kautz function:

7

Model:

where

The parameters can be estimated using standard least squares method:

7 7

System Identification

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

7|22

System identification

2)ˆ)(( θ(k)ky T

d(k)θ(k)y(k) T

System u

d

y

T

n

T

n aaakukukuk ],...,,[,)](),...,(),([)( 2121

)()()(or)()()( kuqkukuqLku nnnn

8 8 8

Problem formulation

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

8|22

Problem formulation

We consider the linear system model:

It is assumed the measured output is given by

where d(k) is a structured disturbance:

• outlier signal,

• level shifts,

• piecewise constant trends

)()()( kdkyky L

),(...)()()( 2211 kuakuakuaky nnL

9 9 9

Disturbance models

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

9|22

Disturbance models

Sequence of outliers:

Level shifts:

Sequence of trends:

10 10 10

Sparse representation of disturbance

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

10|22

Sparse representation of disturbance

For the structured disturbances, the vectors are sparse,

where and , depends on disturbances

Outliers:

Level shifts:

Trends:

dDi

ID 0

iD

11

Identification by sparse optimization:

subject to

where

This is an intractable combinatorial optimization problem.

Instead we use and solve the convex problem

11 11 11 11

Sparse optimization approach

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

11|22

Sparse optimization approach

1

2ˆ,ˆ

ˆ))(ˆ)((min dDλkyky i

kd

2ˆ,ˆ

))(ˆ)((min k

dkyky

elements nonzero ofnumber 0

MdDi 0

relaxation1 l

12

Algorithm

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

12|22

Algorithm

1. Make the basis function expansion based on given prior knowledge of system (pole and system order).

2. Solution of sparse optimization problem by iterative reweighting:

Minimize the weighted cost to give the estimates

where i = 0, 1 or 2 (user selected).

3. Calculate new weights W and go to step 2.

4. Continue until convergence.

5. Use model order reduction to get lower order model.

1

2ˆ,ˆ

ˆ))(ˆˆ)((min dDWλkdky ii

k

T

d

We apply the proposed identification detrending method to the ARX model

with parameter vector

given by

u(k) and e(k) are normally distributed signals with variances 1 and 0.1, and

d(k) is unknown structured disturbance ,

For making the Kautz basis function we used N=4 as order of system and pole

is

13 13

Example1

13|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

4.07.0

14

Example1

14|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0.9340ˆ 9125.0 ˆ RMSE :without

5.2937RMSE 0.7697- 1.3465 2.8084 1.5214 ˆ

0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ

LS

LS

RMSEd(k)

0 100 200 300 400 500 600 700 800 900 1000 -20

-10

0

10

20

K data points

Output

Output (model)

Output (LS)

0 100 200 300 400 500 600 700 800 900 1000

-5

0

5

10

K data points

Disturbance (Identified)

Disturbance

15

Example1

15|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0 1 2 3 4 50

1

2

3

4

5

6

7

8Hankel Singular Values (State Contributions)

State

Sta

te E

nerg

y

Stable modes

16

Example1

16|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0.9118RMSE 0.4933- 0.6946- ˆ

0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ

R

0 100 200 300 400 500 600 700 800 900 1000-20

-10

0

10

K data points

Output (model)

Output (Reduced model)

0 100 200 300 400 500 600 700 800 900 1000

-5

0

5

10

K data points

Disturbance (Identified)

Disturbance

17 17

Example2: Pilot-scale distillation column data

17|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0 200 400 600 800 1000 1200 1400 1600 1800 2000 91

91.5

K data points

Top column output

0 200 400 600 800 1000 1200 1400 1600 1800 2000 0

2

K data points

Bottom column output

0 200 400 600 800 1000 1200 1400 1600 1800 2000 156

158

160

162

K data points

Reflux (L)

0 200 400 600 800 1000 1200 1400 1600 1800 2000 87

88

89

90

K data points

Reboiling flow (V)

1

18 18

Example2: Identification results

18|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1

1.5

K data points

Output (real)

Output+dd (model)

0 200 400 600 800 1000 1200 1400 1600 1800 2000-0.2

-0.1

0

0.1

0.2

0.3

K data points

Disturbance (identified)

0.0933RMSE 0.0456- 0.0353- ˆ

0.0826 RMSE 12) ...(total 0.0147- 0.0338- 0.0285- 0.0022- ˆ

0.0643RMSE 12) (total ... 0.0151- 0.0346- 0.0280- 0.0031- ˆ

RED

LS

19 19

Example2: Hankel singular value for reduction

19|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0 2 4 6 8 10 120

0.1

0.2

0.3

0.4

0.5

0.6

0.7Hankel Singular Values (State Contributions)

State

Sta

te E

nerg

y

Stable modes

20 20

Example2: Validation data with estimation of d(k)

20|22

Example

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

0.0891RMSE 0.0456- 0.0353- ˆ RED

0 200 400 600 800 1000 1200-1

0

1

Output (real)

Output(Test Model+Disturbance)

0 200 400 600 800 1000 1200

-0.2

0

0.2

Disturbance

0 200 400 600 800 1000 1200

-0.5

0

0.5

0 200 400 600 800 1000 1200-1

0

1

Summary:

• Presented a method for identification of linear systems in the presence of

structured disturbances (outliers, level shifts and trends) by using sparse

optimization and orthogonal basis function as the system model

• Gives acceptable results for simulated example

• Gives acceptable results for distillation column example

• The orthogonal basis model improves the robustness and more insensitive

to noise

Future work:

• Nonlinear system identification and trends and more general disturbances

21 21

Discussion and future work

21|22

Discussion and future work

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University

22 22

Thank you for your attention!

Questions?

22 22

22|22

Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University