View
8
Download
0
Category
Preview:
Citation preview
Probabilistic RoboticsBayesian filtering
Martin MagnussonApril 3, 2012
Intro and recap Robot environment interaction Bayes filters Outro
Agenda
1 Intro and recap
2 Robot environment interaction
3 Bayes filters
4 Outro
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Important relations
Conditional probability(the prob that X = x if we know, or assume, that Y = y)
p(x | y) =p(x, y) joint probp(y) scaled to fit y
note that p(x, y) ≤ p(y)
Joint probability of x and y(the prob that both X = x and Y = y)
p(x, y) = p(x | y)︸ ︷︷ ︸cond prob
p(y)︸︷︷︸scaled to fit Ω
= p(y |x)p(x)
Conditional independence: x and y are independent, given z,iff
p(x, y | z) = p(x | z)p(y | z)
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Causal vs. diagnostic reasoning
Environment state X: open or closed.
Robot sensor reading Y: open or closed.
Assume we know p(Y = y |X = open)(i.e., quality of the sensor — causal knowledge)
and need to know p(X = open |Y = y)(i.e., prob that the door is open — diagnostic knowledge)
Bayes’ rule lets us use causal knowledge to infer diagnosticknowledge:
p(open | y) =p(y | open)p(open)
p(y)
(How to compute p(y)? We’ll see that later.)
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Bayes’ formula
p(x | y) =p(y |x)p(x)
p(y)
Compare def. of conditional probability:
p(x, y) = p(x | y)p(y) = p(y |x)p(x)
Theorem (for discrete RV)
p(x | y) =p(y |x)p(x)
p(y)
law of total prob.=
p(y |x)p(x)∑x′ p(y |x′)p(x′)
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Bayes’ formula, explained
Prior: p(x) (probability before sensor input)
Posterior: p(x | y) (probability after input = “diagnosis”)
Bayes’ rule: probability that x is true given y (the posterior)increases with
the prior of x (i.e., prob of x before the test),and the prob of finding y in a world where x is true
decreases withthe prior prob of finding y (i.e., prob of getting test result ywithout knowing the state of x)
The denominator doesn’t depend on x, so it’s the same forboth p(cancer |pos) and p(¬cancer |pos) and is used tomake the posterior p(x | y) integrate to 1.
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Bayes’ formula, robotics exampleX: world state, Z: robot measurements.
Noisy sensors:
p(Z = open |X = open) = 0.6p(Z = closed |X = open) = 0.4
hard to sense open door
p(Z = open |X = closed) = 0.2p(Z = closed |X = closed) = 0.8
easy to sense closed door
Prior probabilities
p(X = open) = 0.5
p(X = closed) = 0.5
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
State estimation example
Suppose the robot senses Z = open.
What is the probability that the door is actually open; thatis, p(X = open |Z = open)?
Apply Bayes’ formula:
p(X = open |Z = open) =
=p(Z = open |X = open)p(X = open)
p(Z = open |X = open)p(X = open) + p(Z = open |X = closed)p(X = closed)
=0.6 · 0.5
0.6 · 0.5 + 0.2 · 0.5 = 0.75
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Law of total probability
Where does the denominator come from? If all y are pairwisedisjoint and fill up all of Ω, then
Theorem (Discrete case)
p(x) =∑
y
p(x | y)p(y) =∑
y
p(x, y)
Follows from the definition of conditional probability andKolmogorov’s axioms.
Robot state variables fulfil the requirements: Can only be inone state at a time, and all outcomes = Ω.
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Law of total probability, illustration
p(x) =
n∑i=1
p(x, yi)=
n∑i=1
p(x | yi)p(yi)
x
y2y1
y3
y4
y5
y6
y7
Intro and recap Robot environment interaction Bayes filters Outro
Recap of last lecture
Law of total probability, proof
If x occurs, then one of yi must also occur (since yi aredisjoint and fill Ω).
So “x occurs” and “both x and one yi occurs” areequivalent.
Equivalent to “∪ni=1(x ∩ yi) occurs”.
∑y
p(y)axiom 1
= 1
p(x) = ∪ni=1(x ∩ yi)
axiom 3=
n∑i=1
p(x, yi)
p(x)def. of joint prob.
=n∑
i=1
p(x | yi)p(yi)
Intro and recap Robot environment interaction Bayes filters Outro
State
State
Description of what the robot needsto know.
State at time t is denoted xt.
State transitions over time:x0 → x1 → . . .
The set of all states from time t1 totime t2:xt1:t2 = xt1 ,xt1+1,xt1+1, . . . ,xt2
Internal state Typically the pose [x, y, θ].
External state Map, other agents, etc.
x, y
θ
Intro and recap Robot environment interaction Bayes filters Outro
State
Markov state
The Markov property
The conditional probability distribution offuture states depends only upon the presentstate, not on the sequence of events thatpreceded it.
In other words, past (x0:t−1) and future(xt+1:∞) states are conditionally independent,given the present state xt.
Intro and recap Robot environment interaction Bayes filters Outro
State
Markov state, example
Positions of chess pieces is Markovstate (complete state), in idealised
chess. . .
. . . but not in real-world chess!
In reality, complete state descriptions are infeasible.
Intro and recap Robot environment interaction Bayes filters Outro
Interaction
Measurements
Sensor input from environment.
Measurement at time t is denoted zt.
Measurements decrease uncertainty.
Intro and recap Robot environment interaction Bayes filters Outro
Interaction
Actions
Action at time t is denoted ut.Typical actions:
the robot turns its wheels to move,the robot uses its manipulator to grasp an object,do nothing (and let time pass by).
Note thatactions are never carried out with absolute certainty,actions generally increase uncertainty.
Intro and recap Robot environment interaction Bayes filters Outro
Interaction
Modelling actions
The outcome of an action u is modelled by the conditionalprobability distribution
p(x |u,x′)
That is, the probability that, when in state x′, executing action u,changes the state to x.
1 state x′ = [10 m,5 m,0]
2 action u = move 1 m forward
3 what is, for example, p(x = [11 m,5 m,0])?(p < 1 because of wheel slip, etc.)
Intro and recap Robot environment interaction Bayes filters Outro
Belief
Belief
We never know the true state of the robot.
All we have is the belief.
Represent belief through conditional probabilitydistribution:
bel(xt) = p(xt | z1:t,u1:t)
A belief distribution assigns a probability density (or mass)to each possible outcome, (given a sequence of actions andmeasurements).
Belief distributions are posterior probabilities over statevariables, conditioned on the available data.
Intro and recap Robot environment interaction Bayes filters Outro
Belief
Prediction vs. belief
Represent belief through conditional probability distribution:
bel(xt) = p(xt | z1:t,u1:t)
Prediction: the belief distribution before incorporating ameasurement
bel(xt) = p(xt | z1:t−1,u1:t)
Belief: the belief distribution after a measurement
bel(xt) = p(xt | z1:t,u1:t)
Intro and recap Robot environment interaction Bayes filters Outro
The algorithm
Bayes filters: framework
Given:1 stream of observations z and action data u
z1:t,u1:t = u1, z1, . . . ,ut, zt
2 sensor model p(z |x) (how accurate the sensors are)3 action model p(x |u,x′) (how reliable the actuators are)4 prior probability of the system state p(x).
Wanted:estimate of the state x (the belief)
bel(xt) = p(xt | z1:t,u1:t)
Update the belief recursively: bel(xt) is computed frombel(xt−1).
Intro and recap Robot environment interaction Bayes filters Outro
The algorithm
Bayes filters: assumptions
xt−2 xt−1 xt
ut−2
zt−2
ut−1 ut
zt−1 zt
x0 ... ...
Markov assumption impliesstatic worldindependent controlsperfect model — no approximation errors
p(xt |x0:t−1, z1:t,u1:t) = p(xt |xt−1,ut) state transition probability
p(zt |x0:t, z1:t,u1:t) = p(zt |xt) measurement probability
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation, example
Robot observing a door
Given a sensor reading open fromthe camera, what is theprobability that the door isactually open?
p(X = open |Z = open)
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation example, sensor model
Xt = open, closed : world state
Zt = open, closed: robot measurements.
Noisy sensors:
p(Zt = sense_open |Xt = open) = 0.6p(Zt = sense_closed |Xt = open) = 0.4
hard to sense open door
p(Zt = sense_open |Xt = closed) = 0.2p(Zt = sense_closed |Xt = closed) = 0.8
easy to sense closed door
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation example, actions
Actions Ut = push,null
p(Xt = open |Ut = push,Xt−1 = open) = 1p(Xt = closed |Ut = push,Xt−1 = open) = 0
door stays open
p(Xt = open |Ut = push,Xt−1 = closed) = 0.8p(Xt = closed |Ut = push,Xt−1 = closed) = 0.2
can’t always open door
p(Xt = open |Ut = null,Xt−1 = open) = 1p(Xt = closed |Ut = null,Xt−1 = open) = 0p(Xt = open |Ut = null,Xt−1 = closed) = 0p(Xt = closed |Ut = null,Xt−1 = closed) = 1
no other agents
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation example, t = 1Suppose at time t = 1, the robot takes action U1 = null andsenses Z1 = open.We want to compute an updated belief distribution bel(X1).With Bayes’ filter, we can do that using the prior beliefbel(X0).
bel(X1 = open)
= p(X = open |Z = open) =
=p(Z = open |X = open)p(X = open)
p(Z = open |X = open)p(X = open) + p(Z = open |X = closed)p(X = closed)
=0.6 · 0.5
0.6 · 0.5 + 0.2 · 0.5= 0.75
bel(X1 = closed) =0.2 · 0.5
0.6 · 0.5 + 0.2 · 0.5 = 0.25 = 1− bel(X1 = open)
Intro and recap Robot environment interaction Bayes filters Outro
Example
State transisions
p(x |u,x′) for u = push
X = open X = closed
0
0.8
1 0.2
This is a simple two-state Markov chain.
If the door is closed, the action push succeeds in 80% of thecases.
Intro and recap Robot environment interaction Bayes filters Outro
Example
Integrating the outcome of actions
We know p(x |u,x′) (that’s our action model).
How to compute the posterior p(x |u)? I.e., the resultingbelief after the action.
Integrate over all prior states x′.
The law of total probability gives us
p(x |u) =∑x′
p(x |u,x′)p(x′) discrete case
p(x |u) =
∫p(x |u,x′)p(x′) dx′ continuous case
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation example, executing an actionSuppose at time t = 2, the robot takes action u2 = push.
p(X = open | u2)
=∑x′
p(X = open | u2, x′)p(x′)
= p(X = open | u2,X = open)p(X = open)
+ p(X = open | u2,X = closed)p(X = closed)
= 1 · 0.75 + 0.8 · 0.25 = 0.95
p(X = closed | u2)
=∑x′
p(X = closed | u2, x′)p(x′)
= p(X = closed | u2,X = open)p(X = open)
+ p(X = closed | u2,X = closed)p(X = closed)
= 0 · 0.75 + 0.2 · 0.25 = 0.05
Intro and recap Robot environment interaction Bayes filters Outro
Example
Combining evidence
How can we integrate the next observation Z2?
More generally, how can we estimate p(X |Z1, . . . ,Zn)?
Intro and recap Robot environment interaction Bayes filters Outro
Example
Bayes’ rule, with background knowledge
p(x | y) =p(y |x)p(x)
p(y)
We can also condition Bayes’ rule on additional RVs(background knowledge):
p(x | y, z) =p(y |x, z)p(x | z)
p(y | z)
Intro and recap Robot environment interaction Bayes filters Outro
Example
Recursive Bayesian updating
p(x | z1, . . . , zt) =p(zt |x, z1, . . . , zt−1)p(x | z1, . . . , zt−1)
p(zt | z1, . . . , zt−1)
Markov assumption: zt is independent of z1:t−1 if we know x.Then we can simplify:
p(x | z1, . . . , zt) =
sensor model︷ ︸︸ ︷p(zt |x)
prior︷ ︸︸ ︷p(x | z1, . . . , zt−1)
p(zt | z1, . . . , zt−1)︸ ︷︷ ︸normaliser
Intro and recap Robot environment interaction Bayes filters Outro
Example
State estimation example, t = 2
After taking action u2 = push, it senses z2 = open.
bel(X2 = open)
= p(X2 = open | z1, z2) =
=p(z2 |X1 = open)p(X1 = open | z1)
p(z2 |X1 = open)p(X1 = open | z1) + p(z2 |X1 = closed)p(X1 = closed | z1)
=0.6 · 0.75
0.6 · 0.75 + 0.2 · 0.25= 0.90
bel(X2 = closed) =0.2 · 0.25
0.6 · 0.75 + 0.2 · 0.25= 0.10 = 1− bel(X2 = open)
Intro and recap Robot environment interaction Bayes filters Outro
Example
The Bayes filter algorithm
Giventhe previous belief distribution,the latest action,and the latest sensor measurement,
compute an updated belief distribution for time t.
1: function BayesFilter(bel(Xt−1),ut, zt)2: for all xt do3: bel(xt) =
∫p(xt |ut,xt−1) bel(xt−1) dxt−1 . control update
4: bel(xt) = p(zt |xt) bel(xt) p(zt)−1 . measurement update
5: end for6: return bel(Xt)7: end function
Intro and recap Robot environment interaction Bayes filters Outro
Example
The Bayes filter algorithm explained
The control update comes from the law of total probability:For all prior states xt−1, sum up (integrate)
the product of the prior for xt−1
and the prob that u makes the transition from xt−1 to xt.
The measurement update comes from Bayes ruleThe prob of getting zt in xt
times the prior for xt (after the control update),divided by the prior of zt, in order to make the total mass ofbel(xt) = 1.
Intro and recap Robot environment interaction Bayes filters Outro
Example
Why can’t we use the Bayes filter in reality?
Because we can’t compute the update rule for continuous statespaces!
Because of the integral in the denominator (normaliser) ofBayes’ rule
Because of the integral in the control update
Intro and recap Robot environment interaction Bayes filters Outro
SummaryMarkov assumptions: we don’t need history of all previousstates.Sensor measurements Z decrease uncertainty, robot actionsU increase uncertainty.Belief is represented as posterior PDF over possible stateoutcomes, conditioned on sensor data and actions.
Bayes rule allows us to compute probabilities that are hardto assess otherwise.Under the Markov assumption, recursive Bayesian updatingcan be used to efficiently combine evidence.Bayes filters are a probabilistic tool for estimating the stateof dynamic systems.The Bayes filter cannot be implemented for realistic,continuous, state spaces. (The remainder of the course willdiscuss approximations.)
Intro and recap Robot environment interaction Bayes filters Outro
Next lecture
Time and space
10.15–12.00, Wednesday April 11T-111
Reading material
Thrun et al., Chapters 5 and 6
Recommended