Data Association Aware Semantic Mapping andLocalization via a Viewpoint-Dependent Classifier

Model

V. Tchuiev, Y. Feldman and V. Indelman

Technion - Israel Institute of Technology

vovatch@campus.technion.ac.ilyurif@cs.technion.ac.il

vadim.indelman@technion.ac.il

November 7, 2019

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 1 / 14

Introduction

We propose an object based SLAMapproach.

Classification is a key component for object based SLAM.

Key challenge: operation in perceptually aliased environments.Classification aliasingData Association (DA) aliasing

0 10 20 30 40 50

X coordinate [m]

0

10

20

30

40

50

Y c

oord

inate

[m

]

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1

1Right image from: Pathak, Shashank, Antony Thomas, and Vadim Indelman. ”A unified framework for data associationaware robust belief space planning and perception.” The International Journal of Robotics Research 37, no. 2-3 (2018): 287-315.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 2 / 14

Related Work

Existing works:

Consider most likely class semantic measurements.(Mu et al. 2016, Bowman et al. 2017)

Utilize a viewpoint dependent classifier with DA solved:(Velez et al. 2012, Teacy et al. 2015, Feldman and Indelman 2018)

Consider hypotheses for DA only:(Pathak et al. 2018)

Our work:

Considers semantic measurements of class probability vectors.

Uses a viewpoint dependent classifier model for DA-aware semantic SLAM.

Considers joint DA and classification hypotheses.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 3 / 14

Contribution

We utilize the coupling between classifier outputs and relativeviewpoint between object and camera to:

1 Assist in data association (DA) disambiguation.

2 Improve accuracy and reduce uncertainty of pose inference for therobot and the objects in the scene.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 4 / 14

Definitions and Formulation

We aim to maintain the hybrid belief:

P(Xk ,C , β1:k |Hk)

Xk : all robot and object poses.

xk : robot pose at time k .

C : class hypothesis of all objects.

βk : data association realization.

zgeok : geometric measurement of an object.

z semk : class probability vector of an object.

Z geok

.= {zgeok }, Z sem

k.

= {z semk }: geometric and semantic measurements.

Hk.

= {Z geo1:k ,Z

sem1:k , a0:k−1}: measurement history.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 5 / 14

Assumptions

A single robot within a static environment.

A known number of objects.

Models: motion P(xk |xk−1, ak−1), geometric P(Z geok |Xk , βk), and

classifier P(Z semk |Xk ,C , βk) are Gaussian.

The object observation modelP(βk |x rel) determines if DArealization is feasible givenrelative pose.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 6 / 14

The Classifier Model

zsemk ∈ RM is viewpoint dependent.

0 10 20 30 40 50

X coordinate [m]

0

10

20

30

40

50

Y c

oord

inate

[m

]

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

1 2 30.0

0.5

1.0

Cla

ss P

robabilit

y

The model is assumed Gaussian P(zsemk |c , x rel) = N (hc ,Σc), wherehc and Σc depend on object class c and relative pose x rel .

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 7 / 14

General Approach

Split the hybrid belief to continuous and discrete parts:

P(Xk ,C , β1:k |Hk) = P(Xk |C , β1:k ,Hk)︸ ︷︷ ︸bCβ1:k

[Xk ]

P(C , β1:k |Hk)︸ ︷︷ ︸wCβ1:k

bCβ1:k[Xk ] is the continuous belief given a class and DA realization.

wCβ1:k

is the weight of bCβ1:k[Xk ], computed separately for each C and

β1:k .

We keep all continuous beliefs with large enough weights.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 8 / 14

Continuous Belief and Weights

Continuous belief update:

bCβ1:k[Xk ] ∝ bCβ1:k−1

[Xk−1]P(xk |xk−1, ak−1)P(Zk |Xk ,C , βk)

Weight update:

wCβ1:k

∝ wCβ1:k−1

∫Xk

P(βk |Xk)bCβ1:k[Xk ]dXk

Small weights are pruned to keep the number of realizations small.

Viewpoint dependent classifier model in P(Zk |Xk ,C , βk) assists ininference DA, and reduces number of realizations when pruned.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 9 / 14

Experiment

Comparison between without and with classifier model.

Highly aliased scenario with 6 identical objects with differentorientations.

Uninformative prior on initial robot pose, causing multiple probablehypotheses.

Figure: Time k = 1, without (left) and with (right) classifier model

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 10 / 14

Experiment

Figure: Time k = 15, without (left) and with (right) classifier model

=⇒ With classifier:

fewer belief components

More accurate localization

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 11 / 14

Experiment

Figure: Time k = 15, without (left) and with (right) classifier model

=⇒ With classifier:

fewer belief components

More accurate localization

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 11 / 14

Experiment

Figure: Weight comparison, times k = 1 (left) and k = 15 (right)

=⇒ With classifier:

fewer belief components

stronger disambiguation

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 12 / 14

Experiment

Figure: Weight comparison, times k = 1 (left) and k = 15 (right)

=⇒ With classifier:

fewer belief components

stronger disambiguation

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 12 / 14

Conclusion and Summary

We propose an approach addressing DA and classificationambiguity that maintains a hybrid belief.

We utilized the coupling between object class and relative viewpointvia a viewpoint dependent classifier model.

Performance improvement in a highly aliased scenario wasdemonstrated for disambiguation and localization.

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 13 / 14

Thank you for listening!

V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 14 / 14