Laboratory for Applications of Remote Sensing Critical Class Oriented Active Learning for Hyperspectral Image Classification Hyperspectral Image Classification

Laboratory for Applications of Remote Sensing

Critical Class Oriented Active Learning for

Hyperspectral Image Classification

School of Civil Engineering, Purdue Universityand


Email: {[email protected], mcrawford2}@purdue.eduJuly 28, 2011

IEEE International Geoscience and Remote Sensing Symposium

Wei Di and Melba Crawford

Outline

BackgroundCritical Class Oriented Active Learning(AL)

Proposed Methods (SVM-CC, SVM-CCMS)– Guided & Active Learning– Critical Class Oriented– Margin Sampling Based

Experimental Results

Conclusions & Future Work


I. BACKGROUND

Motivation

Sampling Strategy

DL PoolTraining Data

Target H

Supervised Classifier

• Achieve better performance• Higher utility, low redundancy

• Economically allocate resources for labeling• Focus on a specific task or requirement

Intelligent sampling strategy

Active Learning

Query Strategy

DL Pool

DU Pool

Supervised

Classifier

New xL

Output Classifier Training

xUf(xu)

Passive Learning

Active Learning (AL) - Iterative learning circle

Uncertainty & Critical Class

Introduction

Active Learning in remote sensing- Classification: Tuia et al. [2009], Patra and Bruzzone

[2011] Demir et al. [2011], Di and Crawford [2011], .- Segmentation: Jun et al. [2010]

Critical Class oriented Active Learning- Shifting hyperplane by pair-wise SVM- Identify “Difficult” Classes- Category based query & margin sampling

GoalProvide concept level guidance for building training set

favoring “difficult” classes


II. PROPOSED METHOD

Key Idea: Shifting Hyperplane

Pair-wise Class A and B

Class AClass B

Hyperplane w

Margin

Support Vectors

Hyperplane

Margin

New Samples

Cumulative Change

Critical Significant Level

Changing Hyperplane

Critical Class Identification

Query-based Regularizer

wk - hyperplane vector by SVM for kth binary

class at the t th query.

Accumulated Margin Instability Measure the cumulative changes

2

2 2 2 21 1

1

NCC

k k k k kk

w w w w

1

tk k

t

Order Statistic Rank class pairs:

2

1

( )Nc

k

C

k ii

Prob. of the kth class pair at critical level CL :

,1

1( ) ( )LC t

k L kt

P C

Critical Class Query

QueryCritical Class Set

Critical Class Pair

Higher probability

Critical Class Identification

,1

1( ) ( )LC t

k L kt

P C

• SVM-CCRandom Query From Critical Class Set

• SVM-CCMS

Query Sample within Critical Class set and closest to margin Critical Class Set

Query


III. EXPERIMENTAL RESULTS

Kennedy Space Center & Botswana Data

• AVIRIS hyperspectral data• Acquired on March, 1996• 176 of total 224 bands• Spectral bandwidth 10nm• Spatial resolution 18m

Data Description

KSC BOT

CLASS NAME No. CLASS NAME No.

1 Scrub 761 Water 361

2 Willow Swamp 243 Primary Floodplain 308

3 Cabbage Palm Hammock* 256 Riparian* 303

4Cabbage Palm/Oak

Hammock* 252 Firescar 335

5 Slash Pine* 161 Island Interior 370

6Oak / Broadleaf

hammock* 229 Woodlands* 324

7 Hardwood Swamp* 105 Savanna 342

8 Graminoid Marsh 431 Short Mopane 299

9 Salt Marsh 419 Exposed Soils 229

10 Water 927

* Denotes the hard classes


Index Class-Pair

KSC18(3,4) Cabbage Palm Hammock;

Cabbage Palm/Oak Hammock

26(4,6) Cabbage Palm/Oak HammockOak / Broadleaf hammock

BOT 18(3,6) RiparianWoodlands

• Accumulated Margin Instability (AMI)

Query Step

Cla

ss P

air

Inde

x

Accumulted Changes of ||W|| by SVM-CC

200 400 600 800

1

6

11

16

21

26

31

36

41 10

20

30

40

50

60

70

80

18

10 20 30 400

0.5

1

1.5

AM

I at

10t

h Q

uery

Class Pair Index

SVM-CC at 10 Query

10 20 30 400

2

4

6

AM

I at

30t

h Q

uery

Class Pair Index

SVM-CC at 30 Query

18 26

10th 30th

AMI as learning process

Query Step

Cla

ss P

air

Inde

xAccumulted Changes of ||W|| by SVM-CC

100 200 300 400

1

6

11

16

21

26

31

36 0

20

40

60

80

100

120

KSC BOT

1826 18


0 100 200 300 400 500 600 700 80075

80

85

90

95

100

Query Steps

Cla

ssifi

catio

n A

ccur

acy

for

DU

RSSVM

MS

SVM-CCMS

0 100 200 300 400 500 600 700 80075

80

85

90

95

Query Steps

Cla

ssifi

catio

n A

ccur

acy

for

DT

RSSVM

MS

SVM-CCMS

DT KSC at 600th query BOT at 400th query

Class Index

CC CCMS SVMMS CC CCMS SVMMS

C1 -0.37 -0.41 -0.27 0 0 0C2 2.57 3.97 1.95 -0.07 0.82 1.10C3 -1.87 -0.80 -2.02 11.64 12.04 9.08C4 7.33 9.24 4.84 -0.24 -0.12 -0.06C5 1.33 4.00 2.11 -5.00 -1.76 -1.91C6 2.64 6.34 2.71 3.68 3.07 -0.43C7 4.79 0.30 7.88 0.67 2.50 2.32C8 -3.86 -2.97 -3.20 0.33 0.60 0.66C9 -0.28 -0.72 -0.08 -1.06 -1.68 -0.27

C10 0 0 0

DT

DU

• Learning Curve

• Per-Class Improvement vs RS


100 200 300 400 500 600 700 8000.3

0.4

0.5

0.6

0.7

0.8

0.9

Query Step

SV

s R

atio

RSSVM

MS

SVM-CCSVM-CC

MS

KSC C1 C2 C3 C4 C5 C6 C7 C8 C9 C10

CC 0.60 0.30 0.73 0.83 0.85 0.74 0.45 0.25 0.24 0.27

CCMS 0.50 0.49 0.8 0.89 0.94 0.78 0.38 0.32 0.21 0.23

SVMMS 0.35 0.60 0.36 0.58 0.67 0.54 0.45 0.54 0.40 0.44

RS 0.48 0.43 0.43 0.45 0.48 0.45 0.42 0.45 0.49 0.45

BOT C1 C2 C3 C4 C5 C6 C7 C8 C9

CC 0.03 0.31 0.89 0.11 0.04 0.88 0.13 0.04 0.13

CCMS 0 0.19 0.95 0.07 0.10 0.95 0.15 0.08 0.04

SVMMS 0.20 0.23 0.39 0.36 0.22 0.44 0.28 0.18 0.20

RS 0.28 0.29 0.29 0.30 0.28 0.26 0.27 0.25 0.29

Per-Class Sampling Ratio• Per-class Sampling Ratio

• Ratio of Support Vectors

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by RS

200 400 600 800

1

2

3

4

5

6

7

8

9

10

20

40

60

80

100

RS

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVMMS

200 400 600 800

1

2

3

4

5

6

7

8

9

10

20

40

60

80

100

SVMMS

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVM-CC

200 400 600 800

1

2

3

4

5

6

7

8

9

10

20

40

60

80

100

CC

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVM-CCMS

200 400 600 800

1

2

3

4

5

6

7

8

9

10

20

40

60

80

100

CCMS

KSC


IV. CONCLUSIONS AND

FUTURE WORK


Conclusions- Shifting Hyperplane – Provides valuable information for

identifying difficult classes. - Critical Class Oriented Margin Sampling – Focuses on

difficult classes, as well as informative samples, improve performance in multi-class problem.

- Support Vectors - Concentrate on samples likely to be support vectors.

Future work- Investigation of feature subspaces for identifying the

critical classes.- Design proper sample-wise utility score to enhance the

category based query.


IV. CONCLUSIONS AND

FUTURE WORK


Conclusions- Shifting Hyperplane – Provides valuable information for

identifying difficult classes. - Critical Class Oriented Margin Sampling – Focuses on

difficult classes, as well as informative samples; improves performance in multi-class problem.

- Support Vectors - Concentrate on samples likely to be support vectors.

Future work- Investigation of the feature subspace for identifying the

critical classes.- Design proper sample-wise utility score to enhance the

category based query.

Thanks very much!

Critical Class Identification Process

Query Step

Cla

ss P

air

Inde

x

Accumulted Changes of ||W|| by SVM-CC

200 400 600 800

1

6

11

16

21

26

31

36

41 10

20

30

40

50

60

70

80

Class Pair Index

Crit

ical

Lev

el

Prob. as Critical Class

10 20 30

1

6

11

16

21

26

31

36 0

0.2

0.4

0.6

0.8

1

• Accumulative Margin Instability

• Critical Class Probability Heat Map

10 20 30 400

0.5

1

1.5

AM

I at

10t

h Q

uery

Class Pair Index

SVM-CC at 10 Query

10 20 30 400

2

4

6

AM

I at

30t

h Q

uery

Class Pair Index

SVM-CC at 30 Query


Class Index

Que

ry S

tep

Acc of Each Class for DT By RS

1 2 3 4 5 6 7 8 9 10

100

200

300

400

500

600

700

8000.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Class Index

Que

ry S

tep

Acc of Each Class for DT By SVM

MS

1 2 3 4 5 6 7 8 9 10

100

200

300

400

500

600

700

8000.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Class Index

Que

ry S

tep

Acc of Each Class for DT By SVM-CC

1 2 3 4 5 6 7 8 9 10

100

200

300

400

500

600

700

8000.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Class Index

Que

ry S

tep

Acc of Each Class for DT By SVM-CC

MS

1 2 3 4 5 6 7 8 9 10

100

200

300

400

500

600

700

8000.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

(a) KSC: RS (b) KSC: SVMMS

(c) KSC: SVM-CC (d) KSC: SVM-CCMS

Per-class Learning Performance


Query Step

Cla

ss I

ndex

No. of SVs for Each Class by RS

100 200 300 400

1

2

3

4

5

6

7

8

910

20

30

40

50

60

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVMMS

100 200 300 400

1

2

3

4

5

6

7

8

910

20

30

40

50

60

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVM-CC

100 200 300 400

1

2

3

4

5

6

7

8

910

20

30

40

50

60

Query Step

Cla

ss I

ndex

No. of SVs for Each Class by SVM-CCMS

100 200 300 400

1

2

3

4

5

6

7

8

910

20

30

40

50

60

RS SVMMS

SVM-CC SVM-CCMS

50 100 150 200 250 300 350 4000.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Query Step

SV

s R

atio

RSSVM

MS

SVM-CCSVM-CC

MS

• BOT

• Ratio of Support Vectors

Documents

Laboratory for Applications of Remote Sensing Critical Class Oriented Active Learning for Hyperspectral Image Classification Hyperspectral Image Classification