Model-based Programming as Estimating, Planning and

Executing based on Hidden State

Brian C. WilliamsArtificial Intelligence and Space Systems Labs

Massachusetts Institute of Technology

Objective

Create a hybrid estimation, monitoring, diagnosis and model learning capability for physical devices that exhibit complex discrete and continuous behaviors.

DEMONSTRATION:

JSC BIO-Plex

Mars Entry, descent & Landing

A Hybrid Discrete/Continuous System for Health Management

600 700 800 900 1000 1100 1200 1300 1400

400

500

600

700

800

900

1000

1100

1200

time (minutes)

CO2 concentration (ppm)

crew requests entry toplant growth chamber

crew enters chamber

lighting fault

crew leaveschamber

Support:• NASA IS

NASA BIO-Plex

• Failures can manifest themselves through a coupling of a system’s continuous dynamics and its evolution through different behavior modes must track over continuous state changes and discrete mode changes

• Symptoms are initially subtle; on the same scale as sensor/actuator noise need to extract mode estimates from subtle symptoms

Hybrid Plant Model for HME

mm11

21211212

2323

1313

mm33

mm22

2222

1111

3333

Hidden Markov Models Continuous Dynamics

11

1

( 1) ( ( ), ( ), ( )):

( ) ( ( ), ( ))

( 1) ( ( ), ( ), ( )):

( ) ( ( ), ( ))

c c c c c

c c c c

c ci c c ci

c ci c c

x k f x k u k v km

y k g x k v k

x k f x k u k v km

y k g x k v k

mr1 mr2 mr3

mr5mr4

0x

0x

0x

0.6x

0.6x

0.6x0 0.6x0.9

0.1 0.10.9

PHA

uc1

ud1

ud2 CPHA

yc2

yc1

continuousinput uci

output / observedvariable yci (cont.)

PHA component

A1 A2

wc1

A3

internalvariable

discreteinput udj

Concurrent Probabilistic Hybrid Automaton (CPHA):

Hybrid Mode / State Estimation

CPHA Model

estimated mode/state x = {xd ,xc}and its belief state h[x]sensor signals yc and

control inputs uc , ud

KalmanFilter Bank

yc(k)

uc(k-1)

HybridMode

Estimatorxci(k)

Pi(k)

^

k

Xk^

Hybrid State Estimator Maintains the set of most likely hybrid state estimates as a set of trajectories.

Hybrid Mode Estimator: Determines for each trajectory the possible transitions, and specifies (dynamically) the candidate trajectories to be tracked by the continuous state estimators.

Hybrid Mode / State Estimation

old estimate:Xk-1={mi,xk-1}

X+k-1={mj,xk-1}

new estimate:Xk={mj,xk}

1. HMM-style belief state update determines the likelihood for each discrete mode transition.

2. Kalman-filter-style updatedetermines likelihood of continuous state evolution.

# transitions at each time step is very large:e.g. model with 10 components, each with 3 successor modes has 310 = 59049 possible successor modes for each trajectory!

How to handle the exponential blowup?

• Generalize beam search to track the mostpromising hybrid states.

• Factor state space into lower dimensional subspaces through automated decomposition and filter synthesis.

x(k-1) x(k)PO...

PT1

PT2

PT3

PTl

component2

component1

component3

componentl

transition expansion estimation

h(k-1) h(k)

Simulation Result

components: 6 ( FR1, FR2, PIV1, PIV2, LS, PGC) total # of modes: 9600

fringe size: 20 (400 estimation steps): average candidates: 90.2 (< 1% !)max. candidates: 428 (< 5 %!)filter calculations: 242filter executions: 36050

average runtime: ~1 s/step (PII-400, 128mb)

850 900 950 1000 1050 1100 1150 12000

2

4

6PGC

time [minutes]

mod

e nu

mbe

r

850 900 950 1000 1050 1100 1150 1200460

480

500

520

540

560

time [minutes]

CO

2 co

ncen

trat

ion

[ppm

]

850 900 950 1000 1050 1100 1150 12000

2

4

6Lighting System

time [minutes]

mod

e nu

mbe

r

Future Directions

• Model-Learning as Hybrid EM• Automated Decomposition of HPCA using

Dissents• Model-based Hybrid Execution

Automated Decomposition

1 2

uc1ud1ud2

wc1

3

yc2

yc1vs1 vs3

vo1

vo2

A A

CA

A

vs2

1

vs1

2

3

vs3

yc2

yc1

vo1

vo2vo1

yc1

uc1 A A

A

vs2

uc1

uc1yc1yc2

PO

xc1xc2xc3

MIMO Filter

uc1yc1

PO2

yc1yc2

PO1

xc2xc3

xc1Filter 1

Filter 2

Filter Cluster

uc1

Idea: Support programmers with embedded languages that avoid commonsense mistakes, by reasoning from hardware models.

Polar Lander Leading Diagnosis:

• Legs deployed during descent.

• Noise spike on leg sensors latched by software monitors.

• Laser altimeter registers 50ft.

• Begins polling leg monitors to determine touch down.

• Latched noise spike read as touchdown.

• Engine shutdown at ~50ft.

Reactive Model-based Programming

To Address the Scope of Mars 98

Responding to the failures of Mars Polar Lander and Mars Climate Orbiter is a Hybrid control problem.

Hybrid Model-based Programming

SPlant

Obs Cntrl

Model-basedControl Programs

Model-basedExecutive

S’

PlantModel

Plant

RMPL Hybrid Model-based Executive

SequencerControlProgram

PlantModel

Control actionsObservations

Estimation & Control Engines

Hybrid Executives:• Can hook into existing estimation and

control approaches.• Should target “comfort zone” of systems

engineers.

Discrete Mode Est.

Continuous State Est.

State Estimation State Reconfiguration

Discrete Controller

Continuous Controller

att/pos goals

cont & discrstate estimates

h/w config goals

Mars Entry, Descent &

Landing

Demonstration:

Recent Publications

Hybrid Mode Estimation:

• Hofbaur, M. W. and B.C. Williams, “Mode Estimation of Probabilistic Hybrid Systems,” International Conference on Hybrid Systems: Computation and Control, March, 2002.

Hybrid Expectation Maximization (preliminary):

• Melvin Henry, Simulators that Learn: Automated Estimation of Hybrid Automata, June 2002

Hybrid Decomposition:

• Hofbaur, M. W. and B. C. Williams, “Hybrid Diagnosis with Unknown Behavioral Modes,” International Workshop on Principles of Diagnosis, Austria, May 3-5 2002.