Special Topics: Data Science - subjects.ee.unsw.edu.au · Special Topics: Data Science L3 - System Identi cation Introduction c Prof. Victor Solo School of Electrical Engineering

Special Topics: Data ScienceL3 - System Identification Introduction

c© Prof. Victor Solo

School of Electrical EngineeringUniversity of New South Wales

Sydney, AUSTRALIA

Topics

1 What is System Identification?

2 Motivating Examples.

3 Exploratory Analysis.

4 System Identification: Questions & Answers

Terminology/LanguageSystem Identification

Statistical Signal ProcessingStatistics/

Time Series Other

random stochastic probabilistic

random variable stochastic variable probabilistic variablecontinuous

time signal process functiondiscrete

time signal sequence chain,series1

random process stochastic process probabilistic process

random sequence stochastic sequence probabilistic sequence

analog2 continuousvalued -

digital2 discretevalued categorical

parameter changes much slower than a signal or not at all1 but actually a series is the partial sum of a sequence; 2 extended usage.

V. Solo (UNSW) ELEC9782 2 / 21

System Models

Physical Models for Control

The standard starting point incontrol system design is to writedown equations describing thedynamics of the system under studyusing physics. e.g.• Elec Eng: Kirchoff’s laws;Shroedinger’s equation (for quantumcontrol).• Mech Eng: Newton’s laws ofmotion;• Biomed Eng: e.g. automaticmetering of diabetes injections usesalso physiology;• Civil Eng: fluid mechanics;• Economics: micro-economic theory;etc.,etc.,etc.

Known Parameters

The resulting equations haveassociated parameters such as timeconstants, resonant frequencies,gains, inductances etc. It is usuallyassumed these are known.

In robust control it is allowed thatonly nominal values are known aswell as a tolerance region (e.g.resistor tolerances).


Enter System Identification

Problem

But in practice:• parameters may not be known(this is typical); or• the dynamics may be onlyapproximately known e.g. this istypical in Biomedical applications; or• the dynamics may be too complexto write down exactly e.g. fluidmechanics/aerodynamics of aircraft;chemical process plants.

Solution

In such cases a powerful alternativeapproach is system identificationwhich allows these issues to beovercome a

In system identification we recordsystem input and output signals anduse statistical methods to fit dynamicmodels to these operating records.

aOne may do experiments to measureparticular parameter values but this isexpensive and so of limited use


System Structure & Noise

Black Box

But system identification doesrequire structural assumptions.e.g. if we use a linear structure suchas a linear time-invariant (LTI)model then we have only to estimateunknown parameters.In this case we have black boxmodeling.

White Box

If we can use physics/physiology tocompletely specify the systemequations except for some unknownparameters then we have white boxmodeling.

Grey Box

If we use some theory e.g. physics,physiology to help constrain thestructure of the model (this isparticularly the case with non-linearmodels) then we have grey boxmodeling.

We mostly treat black box modelingbut the principles apply to the othercases.

Noise is unavoidable in signalrecording/sensing so we need tostudy noise models such asautoregressive models.


Open or Closed Loop

The system identification problem comes in two forms.

Open Loop Stable

If the system is open loop stable thenit may be possible to record inputand output signals in open loop.

Closed Loop Stable

If the system is closed loop unstablethen it can only be operated inclosed loop.So input and output signals can onlybe recorded in closed loop.The system identification problem inthis case is more complicated.


Solar Heating 1

o ......o 1J

1

N o N 1J1

UJ o UJ

1J1

"'::l a. en o 0;-

o

::::<:

rl'

to

'< 1,"'::l a.

00o

......N

=

o 1J1

N:,-0 gN

~Ul

UJ o UJ

1J1

-

1Source: L. Ljung (1999). System IdentificationV. Solo (UNSW) ELEC9782 7 / 21

Solar Heating

Source: L. Ljung (1999).System Identification.

'SdlnlI!UI01:JllAldllI!jJU!ldUlllS'poIlddlnoq-O~llldAO/Al!SUdlU!llljOSpUll'nA11:JOjdAdUlnd't\:dlnllllddUldldjJll101S"'1;unll!.!1

AllSUdlUI11l[OS(ll)

O~~v017~£O£~ZOZ~101~

zV

9

8

01~--~~~~--~~~~~Zl

Al!SUdlUI11l[OS

slnoq

O~~v017~£O£~ZOZ~I01~0O~~v017~£O£~ZOZ~101~0~'O-OZ

r--o

017

O£

NOpUBddOUBd~~~~~----~~--------'~v (:)S;:J;l1~;:Jp);:JlnlBl;:Jdw;:J~;:J~B101S

'Sd:JUllqlnlSlpPdlnSlldUlUn:n~lndlno:t\:~d:JUllqlnlS!PPdlnSlldUl:/~lndu!:n:UldlSASdsnoqPdllldlj-llllOSd~£'1amll!.!1

A1POPA

dwnd:n

;:Jlnllll;:JdUl;:Jl

;:J~IlJ01S:t\:UO!llllpBl

lBloS:/


Human BalanceSource: K. Hidenori,Y. Jiang (2006)Control SystemsMagazine, pp 18-23.

PID PARAMETER ESTIMATION THROUGHBODY SWAY MEASUREMENT

with the control laws

e(t) = r(t) − θ(t), (2)

τ(t) = KPe(t − td) + KDde(t − td)

dt+ KI

∫ t

0e(t − td)dt, (3)

where τ represents the moment of rotation, td is the timelag, r(t) = 0, and KP, KD, and KI are the gains of a PID con-trol law. Combining (1) and (3), and approximatingsin θ(t) ≈ θ(t) leads to

[2m1 l2g + (m2 + m3)l21

]θ (t) − [2m1 lg + (m2 + m3)l1]gθ(t)

= KPe(t − td) + KDde(t − td)

dt+ KI

∫ t

0e(t − td)dt. (4)

The overall system is depicted in Figure 4.Estimating the PID gains based on measurements of

body sway is a closed-loop identification probleminvolving the estimation of control parameters instead ofplant parameters. The extent of body sway around theankle joint is small, often not more than 2◦ [6]. To mea-

FIGURE 4 Human static upright stance control. Since the aim of standing is to keep the body upright, the reference value of posturalsway angle is set to zero. The central nervous system detects the error signal and sends an output signal to the muscles so as to keepthe body upright in a state of equilibrium. The controller is modeled as a PID control, and the plant is modeled as an inverted pendulum.Here, com denotes the center of mass.

Sensors + Neuromuscular System

Inverted Pendulum Body

com

Sensory Systems

KP

KDs

Kl /s

+

_

θref = 0 ++

++

+

+

+e−std

Vcop

sin(ωt)

[2m1lg + (m2 + m3)l1]gsin θ

[2m1lg2 + (m2 + m3)l1

2]. s2 com

FIGURE 5 The body-sway measuring system composed of a markerfor image reorganization, a CMOS camera for image recording, anda personal computer for data processing. A whiteboard with markeris set 1.0 m before the subject to act as a visual index at eye level.

CMOS Camera


Speech (pressure) Signals

Note the cyclical/pulsating nature.

Waveform of syllable ’aah’ 2

• How to use speech signal characteristics to design:- speech synthesizer- speech recogniser

2SOURCE :RShumway(1988).AppliedStatisticalTimeSeriesAnalysis


Blood Pressures 4

Diastolic and Systolic Blood pressures 3

• For real-time monitoring (in intensive care) use diastolic pressure topredict systolic pressure and so predict abnormal cardiac event and take(very fast!) preventive action.

3SOURCE: R Shumway (1988). Applied Statistical Time Series Analysis

4Is blood pressure a parameter or a signal?!V. Solo (UNSW) ELEC9782 11 / 21

Earthquake and Nuclear Explosion• Use earthquake and nuclear testrecordings to:- discriminate between earthquakeand nuclear explosion- determine location and direction ofsource.

SOURCE: R Shumway (1988)Applied Statistical Time Series Analysis


Handwriting Recognition - Postcodes

• Use digit/handwriting records to:- Predict correct digit quickly andaccurately from arbitary handwrittenimage.- Discriminate handwriting fromdifferent individuals.

SOURCE: T Hastie etal (2001) Statistical Learning


Mortality and SO2 concentration

Note the sequence of peaks andtroughs.The main public health question ofinterest is whether there’s a relationbetween the two series and inparticular whether SO2 has a causaleffect on mortality.


20 40 60 80 100

−2

0

2

4

6

8

10adjusted mortality and SO_2 concentration

(−)standardisedadjusted mortality

(−.)standardisedSO_2 concentration

time(days)


Simple preliminary analysis

Basic issues that arise in initialanalysis of a time series are:skewness,amplitude heterogeneity, autocorrelation.

Transformed Histogram


Amplitude heterogeneity

Annual tobacco production of theUSA in the period 1871-1984 and log(tobacco production).The log transform has evened outheterogeneity very well. Note thatboth plots would still show a skewedhistogram because of the growthtrend.

Transformed Tobacco Data

1880 1900 1920 1940 1960 1980

0

1000

2000

annual USA tobacco production

1880 1900 1920 1940 1960 19805

6

7

8

log(annual tobacco production usa)


(Normalised) Autocorrelation Sequence - acs or acf

The acs measures the correlation of the time series with itself.The acs of lag r is the sample correlation of the time series with a replicaof itself shifted back r time units.The acs at lag r , ρr is calculated from the autocovariance of lag r ,Cr

ρr =Cr

C0, r = 0, 1, ...

(note that ρ0 = 1) and Cr is the sample covariance of the time series yt

(y at time t) with itself, r time units later.

Cr =1

nΣn−r

t=1(yt − y)(yt+r − y), r = 0, 1...

and y is the sample mean and y1...yn is the data.The theoretical acvs γr and theoretical acs ρr are introduced later.


acf plotsACF of Speech Signal



Questions/Problems RaisedForward/Design Problems

Solve a stochastic variable estimation problem by designing a filter.1 Signal Simulationwith prescribed characteristics e.g. speech.2 Signal Predictionof signal from its own past or past of other related signals e.g. bloodpressure.3 Signal Detection/Extraction( e.g. earthquake. )detect presence of signal in noise and estimate it.4 Pattern Recognition/DiscriminationDiscriminate between signals; recognise (classify) signals of givencharacteristics (features) e.g. handwriting recognition.5 Input/Output AnalysisDetermine source characterisitcs relating to recorded signals e.g. sonarsources.


Questions/Problems Raised IIInverse/Analysis/Statistical Inference Problems

Use training/sample data to estimate empirically, the statistical propertiesof the signals of interest e.g. acf, spectrum

6 Analysis of Signal StructureDetermine a statistical description/model of the data e.g. needed foralmost

7 Determine Dynamic Relationsbetween signals e.g. diastolic , systolic blood pressure.

• NB. Plug-in Method.To solve the Forward Problem we need the statistical characteristics (suchas spectra) obtained from the Inverse Problem. Then just substitute theseestimated spectra into the forward filter formulae. This may not beoptimal but generally works well.


Signal Characteristics

1 GaussianIf variables are jointly Gaussian then statistical characteristics arecompletely captured in first and second moments i.e. means andacf/spectrum.

2 non-GaussianGenerally full joint distributions are needed but data requirements foremprical estimation of these are prohibitive. So specially constrainednonlinear models are used. Still if only a few variables are involvedthen e.g. joint pdfs can be estimated.


Documents

Special Topics: Data Science - subjects.ee.unsw.edu.au · Special Topics: Data Science L3 - System Identi cation Introduction c Prof. Victor Solo School of Electrical Engineering