System Architecture

Intelligently controlling image processing systems

Image Processing and Computer Vision: 7 2

Introduction So far

Presented methods of achieving goals Integration of methods?

Controlling execution Incorporating knowledge

Image Processing and Computer Vision: 7 3

What knowledge? What do algorithms achieve? What is known about the problem

being solved? Relationship between problem and

algorithm?

Image Processing and Computer Vision: 7 4

Knowledge representation Implied Feature vectors Relational structures Hierarchical structures Rules Frames

Image Processing and Computer Vision: 7 5

Implied knowledge Knowledge encoded in software Usually inflexible in

Execution Reuse

Simple to design and implement Systems often unreliable

Image Processing and Computer Vision: 7 6

Feature vectors As seen in statistical

representations Vector elements can be

Numerical Symbolic coded numerically

Image Processing and Computer Vision: 7 7

Example:

strokes 3

loops 1

w-h ratio 1

A Nstrokes 3

loops 0

w-h ratio 1

Image Processing and Computer Vision: 7 8

Relational structures Encodes relationships between

Objects Parts of objects

Can become unwieldy for Large scenes Complex objects

Image Processing and Computer Vision: 7 9

Follow natural division ofHierarchical structures

scene

objects

parts of object

Image Processing and Computer Vision: 7 10

Example:scene

roadway building grassland

grass treeroad junction

edges

Image Processing and Computer Vision: 7 11

Uses Structure defines possible

appearance of objects Structure guides processing

Image Processing and Computer Vision: 7 12

Rules Rules code quanta of knowledge

Interpretation Forwards Backwards

<antecedent> <action>

<two antiparallel lines> <road>

Image Processing and Computer Vision: 7 13

Forward chaining If <antecedent> is TRUE Execute <action>

Antecedent will be a test on some data

Action might modify the data

Suitable for low level processing

Image Processing and Computer Vision: 7 14

Backward chaining Action is some goal to achieve Antecedent defines how it should

be achieved

Suitable for high level processing Guides focus of system

Image Processing and Computer Vision: 7 15

System architecture

Database RulebaseInferenceengine

Image Processing and Computer Vision: 7 16

FramesA “data-structure for representing a

stereotyped situation”

Slot(attribute) Filler

(value: atomic, link to another frame, default or empty, call to a function to fill the slot)

Image Processing and Computer Vision: 7 17

Methods of control How to control how the system’s

knowledge is used. Hierarchical Heterarchical

Image Processing and Computer Vision: 7 18

Hierarchical control “Algorithm” defines control Compare other software:

Main programme calls subroutines Achieve a predefined sequence of

tasks Two extreme variants

Bottom-up Top-down

Image Processing and Computer Vision: 7 19

Bottom-up controlObject

recognition

Extracted features,Attributes,

Relationships

Image

Decision making

Feature extraction

Image Processing and Computer Vision: 7 20

Top-down controlHypothesised

object

Predicted features,Attributes,

Relationships

Features in image thatSupport or refute the

hypothesis

Prediction

Directed feature extraction

Image Processing and Computer Vision: 7 21

Critique Inflexible methods Errors propagate

Hybrid control Can make predictions Verify Modify predictions

Image Processing and Computer Vision: 7 22

Hybrid controlObject

recognition

Image

Decision making

Feature extraction

Extracted features,Attributes,

Relationships

Predicted features,Attributes,

Relationships

Prediction

Direciction

Image Processing and Computer Vision: 7 23

Heterarchical control “Data” defines control via

knowledge sources KSs contribute to process image KS fires in response to presence of

data Creates new data Modifies existing data

Can be chaotic Blackboard

Image Processing and Computer Vision: 7 24

Blackboard architecture

KS KS KS

Blackboard Blackboardscheduler

Image Processing and Computer Vision: 7 25

Information integration Hypotheses boolean

True or false Facts are real valued

True certainty = 1.0False certainty = 0.0Unsure 0.0 < certainty < 1.0

How is this represented?

Image Processing and Computer Vision: 7 26

Example

Recognising cars

Shape analyser - certainty = 0.56Position analyser - certainty = 0.78Texture analyser - certainty = 0.40

How to combine evidence?

Image Processing and Computer Vision: 7 27

Bayesian methods Define a belief network A tree structure

Reflects evidential support of a fact

F1

F2 F3

Image Processing and Computer Vision: 7 28

Propagation of certainty Leaf nodes

Certainty given by basic operations Non-leaf nodes

Combine child nodes’ certainties Results propagate to root node

Image Processing and Computer Vision: 7 29

Dempster-Shafer Bayesian theory has confidence in

belief only No measure of disbelief D-S attempts to define this

Image Processing and Computer Vision: 7 30

Certainty interval

0 .. A = measures of beliefA .. B = measures of uncertaintyB .. 1 = measures of disbelief

[A,B] starts large.As evidence accumulates to support or

refute a hypothesis, A and B change

Image Processing and Computer Vision: 7 31

Other formalisms Belief calculi exist Not yet widely used

A result is important Confidence in result is not quantified

Image Processing and Computer Vision: 7 32

Summary Intelligent (vision) systems

Knowledge representation Control strategies Integration of belief

Image Processing and Computer Vision: 7 33

Everything that can be invented has been inventedCharles Duell, Commissioner U.S. Office

of Patents, 1899