May 16-18, 2005 MultTemp 2005, Biloxi, MS 1
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
James C. TiltonMail Code 606*
NASA GSFCGreenbelt, MD 20771
William T. LawrenceNatural Sciences
Bowie State UniversityBowie, MD 20715
*Computational & Information Sciences and Technology Office
May 16-18, 2005 MultTemp 2005, Biloxi, MS 2
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Proposal: To develop tools and methods for automated change detection from remotely sensed imagery utilizing a previously developed approach for creating segmentation hierarchies from imagery data.
Step-1 Proposal has been submitted to the ROSES-2005 NRA, Land-Cover/Land-Use Change Element
May 16-18, 2005 MultTemp 2005, Biloxi, MS 3
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
A set of image segmentations that
i. consist of segmentations at different levels of detail, in which
ii. the coarser segmentations can be produced from merges of regions from the finer segmentations, and
iii. the region boundaries are maintained at the full image spatial resolution.
What is a Segmentation Hierarchy?
May 16-18, 2005 MultTemp 2005, Biloxi, MS 4
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Image Analysis is transformed from pixel-by-pixel analysis into object-by-object analysis, allowing the utilization of object shape, texture and context for a more robust and accurate analysis.
A hierarchy of segmentations allows dynamic selection of the appropriate level of segmentation detail for each object of interest.
Advantages of a Segmentation Hierarchy
May 16-18, 2005 MultTemp 2005, Biloxi, MS 5
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Collected May 17, 2000 over Baltimore, MD. Four meter spatial resolution. Four spectral bands: blue, green, red and nir. 384x384 pixel sub-section. Twelve-level hierarchical segmentation.
Example: Ikonos Data
May 16-18, 2005 MultTemp 2005, Biloxi, MS 6
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
OriginalImage
May 16-18, 2005 MultTemp 2005, Biloxi, MS 7
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 30Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 8
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 18Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 9
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 11Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 10
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 8Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 11
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 6Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 12
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
RegionMean Image
with 4Regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 13
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
OriginalImage
May 16-18, 2005 MultTemp 2005, Biloxi, MS 14
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
TwelveLevel
HierarchicalBoundaries
May 16-18, 2005 MultTemp 2005, Biloxi, MS 15
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Hierarchical Segmentations produced by RHSEG: RHSEG is a hybrid of Hierarchical Step-Wise
Optimization* region growing with spectral clustering – controlled by spclust_wght parameter.
* J. M. Beaulieu and M. Goldberg, “Hierarchy in picture segmentation: A stepwise optimal approach,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp. 150-163, 1989.
RHSEG and HSEGViewer
May 16-18, 2005 MultTemp 2005, Biloxi, MS 16
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Recursive implementation facilitates a highly efficient parallel implementation – a full Landsat TM scene (6500x6500 by 6 bands) can be processed in under 10 minutes with 256 CPUs.
The HSEGViewer program provides a convenient, user-friendly, tool for visualizing and interacting with the image segmentation hierarchies produced by the RHSEG program.
RHSEG and HSEGViewer
May 16-18, 2005 MultTemp 2005, Biloxi, MS 17
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
HSEGViewer and demo version of RHSEG are available through http://tco.gsfc.nasa.gov/RHSEG/index.html
More information on RHSEG available at http://cisto.gsfc.nasa.gov/TILTON/index.html
RHSEG and HSEGViewer
May 16-18, 2005 MultTemp 2005, Biloxi, MS 18
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Assembled a multi-season, multi-year test data set from MODIS Terra AM1 platform for initial tests: Bands 1-7 (aggregated to 1KM) Twelve dates: 31 JAN 2003, 19 APR 2003, 09 AUG 2003,
21 OCT 2003, 28 OCT 2003, 18 NOV 2003, 01 FEB 2004, 20 MAR 2004, 11 JUN 2004, 24 SEP 2004, 29 NOV 2004, 28 FEB 2005.
1002x1002 pixels at 1km spatial resolution centered roughly over the Salton Sea.
Southern California fires visible in 28 OCT 2003 scene.
Monitoring Change: First Steps
May 16-18, 2005 MultTemp 2005, Biloxi, MS 19
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
21 OCT 2003Bands 7, 2 & 1
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 20
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
28 OCT 2003Bands 7, 2 & 1
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 21
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
01 FEB 2004Bands 7, 2 & 1
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 22
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
01 FEB 2004Hierarchical
Boundary Map15 regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 23
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
01 FEB 2004Hierarchical
Boundary Map 9 regions
May 16-18, 2005 MultTemp 2005, Biloxi, MS 24
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
01 FEB 2004Bands 7, 2 & 1
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 25
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Data set Region Label
# of Pixels
Crit. Value
N. Dissim.* vs. HL 0
Hierarchical Levels
31 JAN 2003 1 372749 254 0.248943 32-35
19 APR 2003 1 285261 232 0.204377 33-36
09 AUG 2003 1 331834 212 0.0674519 28-31
21 OCT 2003 1 229176 42 0.0089381 13-33
28 OCT 2003 1 398522 232 0.440007 28-29
18 NOV 2003 1 322248 68 0.0021507 17-30
01 FEB 2004 1 302615 177 0.114273 32-39
20 MAR 2004 1 868747 1059 10.4879 36
20 MAR 2004 6 263434 498 2.36267 35
20 MAR 2004 7 81990 36 0.017641 4-34
11 JUN 2004 3 355635 530 2.86384 32-33
11 JUN 2004 4 116537 108 0.179437 23-31
24 SEP 2004 1 361295 80 0.0461599 22-30
29 NOV 2004 1 387895 84 0.0061176 23-36
28 FEB 2005 1 289606 161 0.160385 30-35* Normalized Dissimilarity vs. Region Mean at Finest Hierarchical Level.
May 16-18, 2005 MultTemp 2005, Biloxi, MS 26
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Sum of Masks from Minimum Mean Regions
Histogram0: 644777 1: 2211 2: 1041 3: 872 4: 1127 5: 4703 6: 20366 7: 63794 8: 109386 9: 87193 10: 57075 11: 26905 12: 29126
May 16-18, 2005 MultTemp 2005, Biloxi, MS 27
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Land vs. Water Mask
(0-3 designated
as land and 4-12 as water.)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 28
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Region Label
# of Pixels Crit. Value
N. Dissim. vs. HL 0
Boundary Npix/Area
Hierarchical Levels
22 23933 187 1.204 0.051 31-37
22 19723 91 0.294 0.058 23-30
22 11565 29 0.0 0.160 0-22
Cloud and Snow Detection/Masking
31 JAN 2003 Data Set (A2003031.1815)
Brightest Region:
Region 22 is clearly a cloud at all hierarchical levels (by inspection)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 29
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 30
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Cloud Region
(water masked)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 31
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking31 JAN 2003 Data Set (A2003031.1815)
Selected Region:
Region 51//55 is mountain snow at hierarchical levels 0 through 33
(by inspection – and change in normalized dissimilarity)
Region Label
# of Pixels
Crit. Value
N. Dissim. vs. HL 0
Boundary Npix/Area
Hierarchical Levels
1 580396 866 42.93 0.014 37
1 233974 340 44.17 0.268 36
15 83333 272 34.75 0.098 34-35
51 3055 53.1 1.153 0.305 13-33
55 1358 0.0 0.0 0.602 0-12
May 16-18, 2005 MultTemp 2005, Biloxi, MS 32
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 33
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Cloud and
Snow Regions(water masked)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 34
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking31 JAN 2003 Data Set (A2003031.1815)
Selected Region:
Region 64 is mountain snow through hierarchical level 4(by inspection – and change in normalized dissimilarity)
Region Label
# of Pixels
Crit. Value
N. Dissim. vs. HL 0
Boundary Npix/Area
Hierarchical Levels
1 580396 866 26.84 0.014 37
3 193935 165 27.67 0.482 29-36
6 81132 120 23.44 0.379 27-28
13 35237 50.2 22.02 0.645 12-26
50 1904 35.6 12.83 0.463 5-11
64 94 0.0 0.0 0.628 0-4
May 16-18, 2005 MultTemp 2005, Biloxi, MS 35
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 36
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Cloud and
Snow Regions(water masked)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 37
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking31 JAN 2003 Data Set (A2003031.1815)
Selected Region:
Region 61 is mountain snow through hierarchical level 22(by inspection – and change in normalized dissimilarity)
Region Label
# of Pixels
Crit. Value
N. Dissim. vs. HL 0
Boundary Npix/Area
Hierarchical Levels
1 580396 866 11.44 0.014 37
1 233974 341 8.510 0.268 36
1 150641 214 10.91 0.376 33-35
8 73354 124 11.32 0.415 28-32
14 27527 91.5 11.19 0.374 23-27
61 713 21.5 0.0 0.550 0-22
May 16-18, 2005 MultTemp 2005, Biloxi, MS 38
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 39
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Cloud and
Snow Regions(water masked)
May 16-18, 2005 MultTemp 2005, Biloxi, MS 40
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water, clouds
& snow masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 41
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
31 JAN 2003Bands 7, 2 & 1(water masked)
Band: bandwidth
1: 620- 670nm2: 841- 875nm7: 2105-2155nm
May 16-18, 2005 MultTemp 2005, Biloxi, MS 42
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Obtain Cloud Mask data product (MOD 35) for pertinent data set dates and compare with RHSEG results.
If available, obtain Snow Cover data product (MOD 10) and compare with RHSEG results.
Obtain other pertinent MODIS data products (e.g. MOD 12 – Land Cover/Land Cover Change, MOD 14 – Thermal Anomalies, Fires & Biomass Burning, MOD 13 – Gridded Vegetation Indices, …) for analysis and comparison.
Monitoring Change: Next Steps
May 16-18, 2005 MultTemp 2005, Biloxi, MS 43
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Develop more flexible tools for analyzing RHSEG segmentation hierarchies, including improvements to HSEGViewer.
Implement other dissimilarity criteria in RHSEG, such as the “Spectral Angle Mapper” criterion.
Implement tools to evaluate various spatial features for use in analyzing the RHSEG segmentation hierarchies, such as convex_area, solidity, and extent, as well as texture and fractal measures.
Implement tools to find and track corresponding regions across multi-temporal data sets.
Monitoring Change: Next Steps
May 16-18, 2005 MultTemp 2005, Biloxi, MS 44
Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data
Automate process to flag areas with intra-data change
Create a rule-based automated classification system to label regions
Create a system to evaluate change as “expected” or “unexpected”
Use a rules-based system to flag areas of change that are not expected
Automated evaluation of change would facilitate (human) follow-up for change mediation/intervention
Monitoring Change: Future Plans