Quality Assessment of

Shuttle Radar Topography Mission

Digital Elevation Data

Ashton Shortridge

Dept. of Geography

Michigan State University

ashton@msu.edu

Third International Conference on

Geographic Information Science

College Park, Maryland, October 20-23

Thanks to

Scott Oppman, Oakland County (Michigan)

Information Technology Dept for data!

NASA personnel and many others for flying the

SRTM & processing all that data!

SRTM

Shuttle Radar Topography Mission

Flown in February, 2000

Collected data over 80% Earth's land area

All land between 60 degrees N, 56 degrees S

Data released at 1 arc second interval for US

Released at 3 arc second interval for roW

SRTM Data Collection

Radar signals transmitted from Shuttle

Received back at two antennas

One in shuttle bay

One on end of 60m boom

Difference between two

signals used to reconstruct

elevation

http://www.jpl.nasa.gov/srtm/missionoverview.html

Data Resources

http://photojournal.jpl.nasa.gov/catalog/PIA02735

Information: http://srtm.usgs.gov/

Americas Download: http://seamless.usgs.gov/

Global Download:

http://glcfapp.umiacs.umd.edu:8080/esdi/index.jsp

SRTM Galapagos

GTOPO 30 Galapagos

~1 km cells

Quality?

Find a data-rich location

Examine error using careful methods

Quantify Error and correlate with other

characteristics

How to consider data quality when this is the best available data for

most everywhere?!

Outline

Case Study: part of Oakland County, MI

Available data

A non-raster based methodology for evaluating

raster data accuracy

Relationship between SRTM error & land cover

Ortonville and the Shuttle Mission

Study site: northern Oakland County, MI

Ortonville (population 1,535) and environs

8.4 km x 6.7 km region

Area facing rapid development at Detroit urban

fringe

Diverse land cover

Varied topography (for MI!)

Survey Elevations

Oakland County GIS contracted for detailed high

accuracy countywide DEM from Woolpert LLC

Aerial Photography collected in April 2000

Derived Points and Breaklines

Stated accuracy

1 foot vertical

2.5 ft horizontal

Survey Elevations

46,065 points

Range:

277 – 360 m.

mean: 310.6

6.6 km

8.3 km

Oakland Data Characteristics

Irregular postings

Michigan State Plane, southern zone

NAD 83, Units int'l feet

Vertical units: International Ft. (NAVD 88)

DEM Data

Data obtained from http://seamless.usgs.gov/

1” NED

NAD83, vertical units meters, NAVD 88

3” SRTM (to match with non-US product)

WGS84, vertical units meters

SRTMAs DEMs

NED

SRTM

SRTM

Land Cover Data

1” 1992 NLCD Land Cover

Modified Anderson Level 2

from Landsat TM

NAD83, from Seamless

30 meter 2001 Land Cover

Michigan GAP, Multiple Anderson Levels

from Landsat TM

Michigan GeoRef (oblique Mercator)

Relatively Incompatible

1992 NLCD

: 11 (Water)

: 21 (LI Resid)

: 22 (HI Resid)

: 23 (Com/Ind)

: 41 (Dec. Forest)

: 42 (Evg. Forest)

: 43 (Mix Forest)

: 81 (Pasture)

: 82 (Row Crops)

: 91 (Wood. WL)

: 92 (Em. H. WL)

2001 GAP-IFMAP

: 11 (Water)

: 21 (LI Resid)

: 22 (HI Resid)

: 23 (Com/Ind)

: 41 (Dec. Forest)

: 42 (Evg. Forest)

: 43 (Mix Forest)

: 81 (Pasture)

: 82 (Row Crops)

: 91 (Wood. WL)

: 92 (Em. H. WL)

Methods

How to integrate this data?

Different datums, coordinate systems, vertical

units, spatial resolutions....

Identify a method that is gentlest on the original

data

vs

A Raster Methodology

Decide upon a common system

Datum / Projection / Coordinate System

Origin, Dimensions, Cell Size

“Preprocess” data to that system

Project, Resample, Clip rasters

Project, Convert Oakland Co. points to raster

Subtract TRUE from SRTM & Intersect with

Landcover

Alternative, Point-Based

Methodology (I)

Assume elevations are gridded spot heights

Not areal averages

Decide upon a common system

Datum / Projection / Coordinate System

Locations at which to conduct analysis

I chose to compare at the DEM locations

“Preprocess” data to that system

Convert rasters to points, project the points

Project Oakland Co. points

Alternative, Point-Based

Methodology (II)

Interpolate 'True' heights at SRTM and NED spot

locations

IDW, power 2, closest 6 neighbors

Interpolate land cover classes at SRTM and NED

spot locations

Nearest - Neighbor

Subtract 'True' from 'DEM' & Intersect with

Landcover

Platforms

Methodology 1 (Raster) implemented in Arc 8.2

(ESRI)

Methodology 2 (Point) implemented in R 1.9.1

(Open Source Statistics Software)

Descriptive Statistics & LC Correlations in R for

both approaches

Results - Error

Raster Method

SRTM Error Statistics

Mean: 2.92 m.; SD: 3.79 m.; RMSE: 4.78 m

NED Error Statistics

Mean: 1.06 m.; SD: 1.49 m.; RMSE: 1.83 m

Points Method

SRTM Error Statistics

Mean: 2.95 m.; SD: 3.93 m.; RMSE: 4.92 m

NED Error Statistics

Mean: 1.07 m.; SD: 1.51 m.; RMSE: 1.85 m

NED Error (Raster)

-9.4 – 15.1 m.

DEM Error (Point)

SRTM Error

-9 – 22.7 m.

1992 NLCD

: 11 (Water)

: 21 (LI Resid)

: 22 (HI Resid)

: 23 (Com/Ind)

: 41 (Dec. Forest)

: 42 (Evg. Forest)

: 43 (Mix Forest)

: 81 (Pasture)

: 82 (Row Crops)

: 91 (Wood. WL)

: 92 (Em. H. WL)

Land Cover and SRTM Error

Error split by overlying land cover type

Significant difference (p-value < 2.2e-16) in mean

error between Land Cover Classes

One-way test of means

Kruskal-Wallis rank sum test

Forest Classes associated with substantial positive error

bias

SRTM too high

SRTM Error by LC Class (2001)

Upland Oak Forest Mixed Dec. Pines Upland Mixed For.

SRTM Error by LC Class (1992)

Forest

NED Error by LC Class (2001)

NED Error by LC Class (1992) Wilcoxan Rank-Sum Test (SRTM)

mu=0 mu=3 mu=4

Class p-value p-value p-value Desc

11 0.398 1 1 Open Water

21 *** 1 1 Low Intensity Residential

22 0.158 1 1 High Intensity Residential

23 0.622 1 1 Commercial/Indust/Transport

41 *** *** *** Deciduous Forest

42 *** *** *** Evergreen Forest

43 *** 0.003 0.075 Mixed Forest

81 *** 1 1 Pasture/Hay

82 *** 1 1 Row Crops

91 *** 0.033 1 Woody Wetlands

92 *** 1 1 Emergent Herb. Wetlands

*** indicates << 0.0001

Discussion - Methods

Point-based method minimized change to

elevations

Interpolation must occur to evaluate error at each node

in projected NED and SRTM

Differences with raster method were slight

Elevation differences reduced

RMSE ~10th meter lower

Effect of forested land cover reduced

Still highly significantly biased

Discussion – SRTM Error

RMSE is well within SRTM specifications

< 16 meters (4.9 m. for study area)

Error significantly higher than zero

Average ~ 3 m.

SRTM is too high

Significantly more bias in forested areas

Means in the 4-6 meter range

Discussion – SRTM Error (II)

RMSE magnitude strongly linked to forested land

cover

Returns not striking the ground

RMSE 6-8 meter range

Opportunity for statistical error models

Employ landcover characteristics to adjust (co)variance

models

Identify canopy height?

Conclusions

SRTM meets basic specs

But mean error is positive (biased – too high)

And variation of error is correlated with landcover

Forests and Error

Forests introduce positive bias

Evergreen forests may be more error-prone

Expect regions not experiencing leaf-off conditions to

have higher error than Michigan

Preprocessing choices make slight difference

SRTM Split into Land Cover Classes

(1992 NLCD Land Cover Classification System)

Error (m.)

Code NPts Mean STD RMSE Description

11 74 -0.38 3.16 3.16 Open Water

21 202 1.35 2.39 2.74 Low Intensity Residential

22 24 0.52 2.36 2.37 High Intensity Residential

23 64 0.04 1.84 1.82 Commercl/Indust/Trans

41 2740 4.75 4.17 6.32 Deciduous Forest

42 252 6.08 4.13 7.35 Evergreen Forest

43 28 5.04 4.05 6.42 Mixed Forest

81 1447 1.34 2.72 3.03 Pasture/Hay

82 1289 0.91 2.82 2.96 Row Crops

91 722 3.44 3.40 4.83 Woody Wetlands

92 202 1.08 3.21 3.37 Emergent Herb. Wetlands