Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data

May 16-18, 2005 MultTemp 2005, Biloxi, MS 1

Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data

James C. TiltonMail Code 606*

NASA GSFCGreenbelt, MD 20771

[email protected]

William T. LawrenceNatural Sciences

Bowie State UniversityBowie, MD 20715

[email protected]

*Computational & Information Sciences and Technology Office



Proposal: To develop tools and methods for automated change detection from remotely sensed imagery utilizing a previously developed approach for creating segmentation hierarchies from imagery data.

Step-1 Proposal has been submitted to the ROSES-2005 NRA, Land-Cover/Land-Use Change Element



A set of image segmentations that

i. consist of segmentations at different levels of detail, in which

ii. the coarser segmentations can be produced from merges of regions from the finer segmentations, and

iii. the region boundaries are maintained at the full image spatial resolution.

What is a Segmentation Hierarchy?



Image Analysis is transformed from pixel-by-pixel analysis into object-by-object analysis, allowing the utilization of object shape, texture and context for a more robust and accurate analysis.

A hierarchy of segmentations allows dynamic selection of the appropriate level of segmentation detail for each object of interest.

Advantages of a Segmentation Hierarchy



Collected May 17, 2000 over Baltimore, MD. Four meter spatial resolution. Four spectral bands: blue, green, red and nir. 384x384 pixel sub-section. Twelve-level hierarchical segmentation.

Example: Ikonos Data



OriginalImage



RegionMean Image

with 30Regions



RegionMean Image

with 18Regions



RegionMean Image

with 11Regions



RegionMean Image

with 8Regions



RegionMean Image

with 6Regions



RegionMean Image

with 4Regions



OriginalImage



TwelveLevel

HierarchicalBoundaries



Hierarchical Segmentations produced by RHSEG: RHSEG is a hybrid of Hierarchical Step-Wise

Optimization* region growing with spectral clustering – controlled by spclust_wght parameter.

* J. M. Beaulieu and M. Goldberg, “Hierarchy in picture segmentation: A stepwise optimal approach,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp. 150-163, 1989.

RHSEG and HSEGViewer



Recursive implementation facilitates a highly efficient parallel implementation – a full Landsat TM scene (6500x6500 by 6 bands) can be processed in under 10 minutes with 256 CPUs.

The HSEGViewer program provides a convenient, user-friendly, tool for visualizing and interacting with the image segmentation hierarchies produced by the RHSEG program.




HSEGViewer and demo version of RHSEG are available through http://tco.gsfc.nasa.gov/RHSEG/index.html

More information on RHSEG available at http://cisto.gsfc.nasa.gov/TILTON/index.html


http://tco.gsfc.nasa.gov/RHSEG/index.html

http://tco.gsfc.nasa.gov/RHSEG/index.html

http://cisto.gsfc.nasa.gov/TILTON/index.html

http://cisto.gsfc.nasa.gov/TILTON/index.html



Assembled a multi-season, multi-year test data set from MODIS Terra AM1 platform for initial tests: Bands 1-7 (aggregated to 1KM) Twelve dates: 31 JAN 2003, 19 APR 2003, 09 AUG 2003,

21 OCT 2003, 28 OCT 2003, 18 NOV 2003, 01 FEB 2004, 20 MAR 2004, 11 JUN 2004, 24 SEP 2004, 29 NOV 2004, 28 FEB 2005.

1002x1002 pixels at 1km spatial resolution centered roughly over the Salton Sea.

Southern California fires visible in 28 OCT 2003 scene.

Monitoring Change: First Steps



21 OCT 2003Bands 7, 2 & 1

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



28 OCT 2003Bands 7, 2 & 1

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



01 FEB 2004Bands 7, 2 & 1

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



01 FEB 2004Hierarchical

Boundary Map15 regions



01 FEB 2004Hierarchical

Boundary Map 9 regions



01 FEB 2004Bands 7, 2 & 1

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



Data set Region Label

# of Pixels

Crit. Value

N. Dissim.* vs. HL 0

Hierarchical Levels

31 JAN 2003 1 372749 254 0.248943 32-35

19 APR 2003 1 285261 232 0.204377 33-36

09 AUG 2003 1 331834 212 0.0674519 28-31

21 OCT 2003 1 229176 42 0.0089381 13-33

28 OCT 2003 1 398522 232 0.440007 28-29

18 NOV 2003 1 322248 68 0.0021507 17-30

01 FEB 2004 1 302615 177 0.114273 32-39

20 MAR 2004 1 868747 1059 10.4879 36

20 MAR 2004 6 263434 498 2.36267 35

20 MAR 2004 7 81990 36 0.017641 4-34

11 JUN 2004 3 355635 530 2.86384 32-33

11 JUN 2004 4 116537 108 0.179437 23-31

24 SEP 2004 1 361295 80 0.0461599 22-30

29 NOV 2004 1 387895 84 0.0061176 23-36

28 FEB 2005 1 289606 161 0.160385 30-35* Normalized Dissimilarity vs. Region Mean at Finest Hierarchical Level.



Sum of Masks from Minimum Mean Regions

Histogram0: 644777 1: 2211 2: 1041 3: 872 4: 1127 5: 4703 6: 20366 7: 63794 8: 109386 9: 87193 10: 57075 11: 26905 12: 29126



Land vs. Water Mask

(0-3 designated

as land and 4-12 as water.)



Region Label

# of Pixels Crit. Value

N. Dissim. vs. HL 0

Boundary Npix/Area

Hierarchical Levels

22 23933 187 1.204 0.051 31-37

22 19723 91 0.294 0.058 23-30

22 11565 29 0.0 0.160 0-22

Cloud and Snow Detection/Masking

31 JAN 2003 Data Set (A2003031.1815)

Brightest Region:

Region 22 is clearly a cloud at all hierarchical levels (by inspection)



31 JAN 2003Bands 7, 2 & 1(water masked)

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



31 JAN 2003Cloud Region

(water masked)



Cloud and Snow Detection/Masking31 JAN 2003 Data Set (A2003031.1815)

Selected Region:

Region 51//55 is mountain snow at hierarchical levels 0 through 33

(by inspection – and change in normalized dissimilarity)

Region Label

# of Pixels

Crit. Value

N. Dissim. vs. HL 0

Boundary Npix/Area

Hierarchical Levels

1 580396 866 42.93 0.014 37

1 233974 340 44.17 0.268 36

15 83333 272 34.75 0.098 34-35

51 3055 53.1 1.153 0.305 13-33

55 1358 0.0 0.0 0.602 0-12




Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



31 JAN 2003Cloud and

Snow Regions(water masked)




Selected Region:

Region 64 is mountain snow through hierarchical level 4(by inspection – and change in normalized dissimilarity)

Region Label

# of Pixels

Crit. Value

N. Dissim. vs. HL 0

Boundary Npix/Area

Hierarchical Levels

1 580396 866 26.84 0.014 37

3 193935 165 27.67 0.482 29-36

6 81132 120 23.44 0.379 27-28

13 35237 50.2 22.02 0.645 12-26

50 1904 35.6 12.83 0.463 5-11

64 94 0.0 0.0 0.628 0-4




Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm








Selected Region:

Region 61 is mountain snow through hierarchical level 22(by inspection – and change in normalized dissimilarity)

Region Label

# of Pixels

Crit. Value

N. Dissim. vs. HL 0

Boundary Npix/Area

Hierarchical Levels

1 580396 866 11.44 0.014 37

1 233974 341 8.510 0.268 36

1 150641 214 10.91 0.376 33-35

8 73354 124 11.32 0.415 28-32

14 27527 91.5 11.19 0.374 23-27

61 713 21.5 0.0 0.550 0-22




Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm







31 JAN 2003Bands 7, 2 & 1(water, clouds

& snow masked)

Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm




Band: bandwidth

1: 620- 670nm2: 841- 875nm7: 2105-2155nm



Obtain Cloud Mask data product (MOD 35) for pertinent data set dates and compare with RHSEG results.

If available, obtain Snow Cover data product (MOD 10) and compare with RHSEG results.

Obtain other pertinent MODIS data products (e.g. MOD 12 – Land Cover/Land Cover Change, MOD 14 – Thermal Anomalies, Fires & Biomass Burning, MOD 13 – Gridded Vegetation Indices, …) for analysis and comparison.

Monitoring Change: Next Steps



Develop more flexible tools for analyzing RHSEG segmentation hierarchies, including improvements to HSEGViewer.

Implement other dissimilarity criteria in RHSEG, such as the “Spectral Angle Mapper” criterion.

Implement tools to evaluate various spatial features for use in analyzing the RHSEG segmentation hierarchies, such as convex_area, solidity, and extent, as well as texture and fractal measures.

Implement tools to find and track corresponding regions across multi-temporal data sets.

Monitoring Change: Next Steps



Automate process to flag areas with intra-data change

Create a rule-based automated classification system to label regions

Create a system to evaluate change as “expected” or “unexpected”

Use a rules-based system to flag areas of change that are not expected

Automated evaluation of change would facilitate (human) follow-up for change mediation/intervention

Monitoring Change: Future Plans

Documents

Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data