Upload
melvin-dennis
View
221
Download
0
Embed Size (px)
Citation preview
IKONOS V&V
1
Stennis Space Center
NASA Scientific Data Purchase
IKONOS V&V Overview
Mary Pagnutti
IKONOS V&V
2
Stennis Space CenterStennis Space Center
NASA Scientific Data Purchase (SDP)
• Began with $50 Million appropriation for NASA purchase of scientific data sets for commercial sources
• Goal: Test new way of doing business; tap new sources of data
• Managed by NASA Commercial Remote Sensing Program, Stennis Space Center, MS
• Phased Approach– Phase I - simulated / prototypical data sets
received and reviewed– Phase II - selection of products and data purchase
over 3 years
• Firm fixed price contracts - cash on delivery of data
IKONOS V&V
3
Stennis Space Center
• Perspective: SDP is an experiment– Can the commercial sector provide useful, cost effective data to the
science community?
– Only one satellite (no backup), buy data while we can
• Assumptions – Space Imaging will attempt to provide high quality data
– NASA Stennis will trust but verify
– Releasing data in a timely fashion is important
– Feedback from users is important
• Competition is developing– Earth Watch, Orbital, Russian Spin Data, West Indian Aerospace
– Methodologies developed will be used again.
IKONOS SDP V&V Philosophy
IKONOS V&V
4
Stennis Space Center
IKONOS V&V Activities
Performance Product V&V Characterization
(Sample V&V data sets)
Shipment Verification
(All acquired data sets)
• Data Format• Meta File• Location / Coverage• Cloud Cover• General Statistics
– mean– standard deviation– pixel histogram– fraction of saturated pixels
• Auto Report generation • Thumbnail generation• Band-to-band registration• MODTRAN / radiance intercomparison • NIIRS evaluation when appropriate
Product Characterization
(Sample V&V data sets)
• Image Quality
–Spatial–SNR
• Geometric Accuracy–Positional
Accuracy–DEM evaluation
• Radiometric Performance
–SpectralCharacterization
Applications Utility
(Sample V&V data sets)
• University of Maryland assessment• Obtain feedback from ESE Researchers
IKONOS V&V
5
Stennis Space Center
Joint Agency Commercial Imagery Evaluation (JACIE) Team Formed
IKONOS one meter panchromatic image of Jefferson Memorial in Washington, D.C.
• Joint NASA / NIMA / USGS team focused on characterizing commercial remote sensing system performances
• Overall performance assessment to meet each agency’s mission requirements
– NASA - System Characterization– NIMA- Photogrammetry– USGS - Cartographic Assessment
• Improved communications with industry
– More productive interchanges between industry and government
– Minimize duplication of effort– Reduction in costs incurred by industry and
government
IKONOS V&V
6
Stennis Space Center
SCIENCE USERS
JACIE TEAM
EOS Validation
Teams U of Maryland U of Arizona
S.Dakota State U
SDP IKONOS Characterization Team
IKONOS V&V
7
Stennis Space Center
IKONOS Validation Sites
SDSU
SSC
BeltsvilleKonza Prairie
Sevilleta
WSMR
Maricopa
U of A
Morrison Quad.
Ivanpah
Railroad Valley
Lunar Lake
Lake Tahoe
Stillman Creek
USGS-EDC
Kaintuck Hollow Quad
Big Springs
IKONOS V&V
8
Stennis Space Center
IKONOS Validation Sites Overview
IKONOS V&V
9
Stennis Space Center
NASA Stennis Space Center, Mississippi
Site: Scattered buildings within heavily wooded area, man-made reservoirs and canal.Elevation 5.5m - 10m.30.388 degrees N, 89.61 degrees W
Targets: • 2 orthogonal concrete edge targets (70x80m oriented E-W, 80x80m oriented N-S) 130m concrete radial target (96.4 deg arc, 0.08-5.0m range), 4 step concrete grayscale target (35x40m each)• over 40 known GPS markers known to within 2 cm• 4 step gray tarps (20mx20m each)
In-Situ Instrumentation: Cimel Sunphotometer, Shadowband Radiometer, ASD and GER Spectro- radiometers, SSC Weather Station, Radiosondeballoon launch,Pyronometer, Goniometer.
Coincident Collects: Landsat 7
Purpose: Spatial, geometric and radiometric characterization, BRDF evaluation.
Concrete edge, radial and 4 step grayscale targets
IKONOS Image Area
ATLAS image and showing known GPS points at SSC
Radiometric Tarp Coupons
IKONOS V&V
10
Stennis Space CenterImage Quality Characterization
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH SIOn-
Orbit
Funct.Collect
Std.Collect
TYPE
Panchromatic MTF = 0.1 atNyquist frequency
Review on-boardcalibrations provided by SI
Optimize and prepare SSCedge targets
Procure tarps Analyze images over
various targets to obtainMTF
Validate periodicallythroughout the data buy
X X Std OrigPrec Orig
SI data
Multispectral MTF = 0.24 atNyquist frequency Same as above X X
Std OrigPrec OrigSI data
Panchromatic GSD = 1.0 mMultispectral GSD = 4.0 m
Review on-boardmeasurements provided bySI
Analyze images overvarious targets to obtainGSD
Validate prior to datapurchase
X X Std OrigPrec Orig
SI data
IKONOS V&V
11
Stennis Space Center
Big Springs, TX Edge Target
• Three 15 m x 15 m squares
• Northing and easting edges allow MTF analyses in two directions
• Edges are aligned approx 8 deg off true north and true east
IKONOS Panchromatic Band
IKONOS V&V
12
Stennis Space CenterImage Quality Characterization, cont’d.
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH SIOn-
Orbit
Funct.Collect
Std.Collect
TYPE
SNR = 10 at Nyquistfrequency (Based on: 10%target reflectance 2:1 target-to-background ratio, 30 degsolar elevation)
Review on-boardcalibrations provided by SI
Optimize and prepare SSCedge targets
Analyze images overvarious targets to obtainSNR
Validate periodicallythroughout the data buy
X X Std OrigPrec Orig
SI data
SNR = 94 at zero spatialfrequency Same as above X X
Std OrigPrec OrigSI data
IKONOS V&V
13
Stennis Space Center
Uniform Sites Selection for SNR
• Utilize NASA Scientific Data Purchase EarthSat Databuy
– TM data for Africa– 30 meter GSD data– Develop algorithms to find most
uniform scenes for Radiometry & SNR estimates
RGB 321
50 100 150 200 250
100
200
300
400
500
600
700
0.5 1 1.50
2
4
6
8
10x 10
4 TM1 std=3.3125%
0.5 1 1.50
2
4
6
8
10x 10
4 TM2 std=4.7671%
0.5 1 1.50
2
4
6
8
10x 10
4 TM3 std=5.4742%
0.5 1 1.50
2
4
6
8
10x 10
4 TM4 std=6.2394%
Libya TM Scene
Tight Gaussian Histogram Indicate Uniform Scenes
African Map with Sites
IKONOS V&V
14
Stennis Space CenterGeometric Accuracy Characterization
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH Lab Funct.Collect
Std.Collect
TYPE
Absolute Horizontal GeometricAccuracy* ± 250 m
Analyze images overground control points toobtain accuracy
Validate periodicallythroughout the data buy
XStd Orig
Absolute Horizontal GeometricAccuracy* ± 12.2 m Same as above X Std Master
Absolute Horizontal GeometricAccuracy* ± 3 m Same as above X Pan Prec Orig
Absolute Horizontal GeometricAccuracy* ± 2 m Same as above X Pan Prec Master
Absolute Horizontal GeometricAccuracy* ± 5 m Same as above X
MS Prec OrigMS Prec Master
* 90% circular error
IKONOS V&V
15
Stennis Space Center
Geometric Accuracy Assessment
• USGS EROS Data Center– Brookings, South Dakota (SDSU)– Sioux Falls, South Dakota (EDC)– Each site surveyed 50+ points throughout 11x11 km scene
• USGS– Morrison Quad– Kaintuck Hollow Quad– Utilize USGS database for both sites
• NASA CRSP– Stennis Space Center– Utilize site survey & Geodetic Targets
Brookings South Dakota
Fast Static SurveyChecked to less than five cm. horizontal using two Trimble 4000ssi units over B Order monuments.
IKONOS V&V
16
Stennis Space CenterRadiometric Performance Characterization
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH SIOn-
Orbit
Funct.Collect
Std.Collect
TYPE
Absolute radiometric accuracyto within ± 10% over the entireimaging exposure dynamicrange (temporal stability)
Review on-board calibrationsprovided by SI
Analyze images overradiometric targets to obtainaccuracy
Validate periodicallythroughout the data buy
X X All
SI data
Relative radiometric accuracyto within ± 5% (pixel to pixel)
Same as above X X All
SI data
IKONOS V&V
17
Stennis Space CenterRadiometric Performance Characterization, cont’d.
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH SIOn-
Orbit&
Lab
Funct.Collect
Std.Collect
TYPE
97% of all detectors should be within5% of the mean quantum efficiency anddark current
Review on-board calibrationsprovided by SI X SI data
Less than 1% inoperable detectorsprior to launch and less than 3% after 5years MMD. No more than 3 adjacentdetectors inoperable.
Review laboratorymeasurements provided by SI
Perform general statisticalanalysis of data
X SI data
IKONOS V&V
18
Stennis Space CenterRadiometric Performance Characterization, contd.
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH Lab Funct.Collect
Std.Collect
TYPE
Linearity to within ± 5% over the entireimaging exposure dynamic range
Review laboratorymeasurements provided by SI
Analyze images overradiometric targets to obtainaccuracy
Validate periodicallythroughout the data buy
X X All
Lab data
Dynamic range of either 8 or 11 bits Validate periodicallythroughout the data buy X X All
IKONOS V&V
19
Stennis Space Center
Radiometric Vicarious Calibration Approaches
• Reflectance Method– Measurement of ground reflectance and atmospheric properties
allows prediction of Top-of-the-Atmosphere (TOA) radiance estimates
– Typically require ground teams and a great deal of coordination
• Radiance Method– Coincident collect with well characterized airborne or satellite
system
IKONOS V&V
20
Stennis Space Center
Reflectance Method
Method: Utilize ground reflectance measurements (Spectroradiometer) and atmospheric measurements (Sun Photometer & Radiosonde) to determine radiometric accuracy of new commercial satellite systems (IKONOS)
Sun Photometer
Spectroradiometer
Commercial Satellite
Radiosonde Balloon
IKONOS V&V
21
Stennis Space Center
Radiance Method
Method: Utilize coincident airborne or spaceborne calibrated imagery (e.g. Landsat 7) to determine radiometric accuracy of new commercial satellite systems (IKONOS)
Commercial Satellite
Well Characterized System (Aircraft or Satellite)
IKONOS V&V
22
Stennis Space Center
IKONOS Characterization Status
• Spatial Characterization – Three acquisitions have been made (received std original imagery only)– Four SI archive images have been acquired– Two acquisitions are pending collection– Awaiting two precision original images– All but one image has been evaluated
• Geometric Characterization– Three acquisitions have been made (received std original imagery only)– Two acquisitions are pending collection (same as above)– Two DEM acquisitions scheduled for spring 2001– Two have been evaluated
• Radiometric Characterization– Six acquisitions have been made and evaluated (received std original imagery
only)– Two acquisitions are pending collection (same as above)
IKONOS V&V
23
Stennis Space Center
IKONOS Characterization Summary
Preliminary Findings:• Geometric Characterizations show imagery is
well within specification• Spatial Characterizations indicate imagery meets
MTF and SNR specifications• Radiometric Characterizations indicate potential
concerns with the red and NIR bands• No serious issues associated with on-board
compression have been found
IKONOS V&V
24
Stennis Space Center
Backup
IKONOS V&V
25
Stennis Space CenterIKONOS Sensor Facts from SI Web Site
IKONOS V&V
26
Stennis Space Center
Spectral Bands
Swath Widths & Scene Sizes
Nominal swath width:13 km at nadir
Areas of interest:A nominal single image at 13 km x 13 km Strips of 11 km x 100 km up to 11 km x 1000 km Image mosaics of up to 12,000 square km. Up to two 10,000 square kilometer contiguous
areas in a single pass within a region
Metric Accuracy
12 meter horizontol and 10 meter vertical accuracywith no ground control
2 meter horizontal and 3 meter vertical accuracywith ground control
These are specified as 90% CE (circular error) forthe horizontal and 90% LE (linear error) for thevertical
IKONOS Sensor Facts from SI Web Site, cont’d.
IKONOS V&V
27
Stennis Space Center
Orbital Information
Altitude 423 miles / 681 kilometers
Inclination 98.1 degrees
Speed 4 miles per second / 7 kilometers per second
Descending Nodal Crossing Time 10:30 am
Revisit Frequency2.9 days at 1 meter resolution;1.5 days at 1.5 meter resolution
These values are for targets at 40 degreeslatitude. The revisit times will be more frequentfor higher latitudes and less frequent forlatitudes closer to the equator.
Orbit Time 98 minutes
Orbit Type Sun – synchronous
Viewing Angle Agile spacecraft – in-track and cross-trackpointing
Weight 1600 pounds
IKONOS Sensor Facts from SI Web Site, cont’d.
IKONOS V&V
28
Stennis Space Center
CameraSystem
Designed and built by Eastman Kodak Total system weight: 376 lbs. (171 kg.) Total system power consumption: 350 watts
OpticalTelescopeAssembly
Assembly size: 61 in. long x 31 in. wide Assembly weight w/out focal plane unit: 240 lbs. Optical design: Three mirror anastigmat Focal length/ focal ratio: 10m / f14.3 Image resolution (at nadir): 1m panchromatic, 4m multispectral Primary mirror: 27.5in. diameter x 4in. thick, 29.5 lbs.
ImagingSensors &Electronics
Focal plane unit Unit size: 10in. x 9in. x 9in. Panchromatic sensor: 12 micron pixel pitch, 13,500 pixels Multispectral sensor: 48 micron pixel pitch, 3375 pixels
DigitalProcessingUnit
Unit size 18in x 7.5in. x 12.4in. Compression rate:11 bits per pixel compressed to2.6 bpp Compression speed: 4 million pixels per second per
processing channel.
PowerSupply Unit Unit size: 7in. x 8in. x 16in.
Launch Date September 24, 1999 (11:21:08 am PDT)
LaunchVehicle
Athena II
LaunchVehicleManufacterer
Lockheed Martin
IKONOS Sensor Facts from SI Web Site, cont’d.
IKONOS V&V
29
Stennis Space CenterStennis Target Set
Positive Systems ImageExpanded View of Target
EdgeTarget: 70x80 meters and 70x70 meters 8:1 contrast ratio
IKONOS V&V
30
Stennis Space Center
Edge Response
x
:response edge the get
to x vs. value pixel plot
tanGSDx
edge and line scan between angle :
Foot Print
0 2 4 6 8 10 12 1460
80
100
120
140
160
180
200
220
240
260Edge Response ADAR 5500 -- Blue band 1m
Meters
Edg
e R
espo
nse
¬ Undersampled
Focal plane array undersamples edge with classical cross section samplingSolution: Sample tilted edge to improve sampling
IKONOS V&V
31
Stennis Space Center
Edge Response Relationship to Line Spread Function
dx
d)x(E
)x(l
x x
)x(lFT)0,(MTF
)y,x(PSFFT),(MTF
IKONOS V&V
32
Stennis Space CenterModulation Transfer Function
• MTF is a measure of an imaging system’s ability to recreate the spatial frequency content of scene
MTF is the magnitude of the Fourier Transform ofthe Point Spread Function / LineSpread Function.
1.0
Cut-off
Spatial frequency
MTF
IKONOS V&V
33
Stennis Space Center
SNR Estimate Example Results
0 50 1000
2000
4000
6000
8000
10000TM1
Pseudo SNR0 50 100
0
2000
4000
6000
8000
10000TM2
Pseudo SNR
0 50 1000
2000
4000
6000
8000
10000TM3
Pseudo SNR0 50 100
0
2000
4000
6000
8000
10000TM4
Pseudo SNR
Histograms of Mean/Std of 5x5 Pixel Blocks
TM data
Location of Histogram Peak is a measure of the SNR
IKONOS V&V
34
Stennis Space CenterGPS Accuracies
• Trimble Pathfinder or Geo Explorer – Raw - 10 meters
– Coarse mode corrected - 5 meters per measurement reading
– Coarse mode corrected - averaging data 2 meter
– Fine mode corrected - averaging 10 minutes of differentially corrected data gives sub-meter
Geo Explorer
Pathfinder
IKONOS V&V
35
Stennis Space Center
Radiometric Error Associated with Finite Size Tarp
Insert Measured Reflectances Figures
5 10 15 20 25
5
10
15
20
25
-6 -4 -2 0 2 4 60
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
GSD
Tar
p R
espo
nse
Tarp/GSD=5 MTF@Nyquist 0.1
For typical background/tarp reflectances, the error at the center pixels will be less than a few percent
Cross Sectional Plot of Response
Finite Size of Tarp will limit Radiometric Accuracy
Simulated 20 m Tarp Image with 4 m GSD with MTF @ Nyquist of 0.1
IKONOS V&V
36
Stennis Space Center
CRSP Tarp Sample Data
50% Tarp 30% Tarp 20% Tarp <5% Tarp
Vendorsupplied mean:420-1050nm
52.78% 38.27% 23.60% 2.46%
Measuredmean:420-1050nm
45.56% 32.69% 20.90% 2.37%
Tarp Sample DataReflectance Data
300 400 500 600 700 800 900 1000 11000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Wavelength (nm)
Ref
lect
ance
(%
)
Reflectivity of tarp samples
50% tarp30% tarp20% tarp<5% tarp
Tarp Samples are Spectrally Flat Over Region of Interest
IKONOS V&V
37
Stennis Space CenterGPS Accuracies
• Trimble 4000ssi Survey grade GPS units– Static survey- 5mm + 1ppm in base line distance.
– Fast Static - Based on conditions and Satellite geometry
– Real Time Kinematic (RTK) - less than five centimeters horizontal and vertical
Brookings South Dakota
Fast Static SurveyChecked to less than five cm. horizontal using two Trimble 4000ssi units over B Order monuments.
IKONOS V&V
38
Stennis Space Center
Radiometric Vicarious Calibration Background
• Radiometric stability of remote sensing data is critical for long term studies
• On-orbit calibration sources are not always stable or reliable
• Methods which independently verify the radiometric calibration of remote sensing data are essential
• Vicarious calibrations use natural well characterized scenes
IKONOS V&V
39
Stennis Space Center
CRSP Radiometric Tarp Sample Examination
Samples were obtained of the 4 grayscale radiometric tarps procured by CRSP. These samples were tested in the laboratory under conditions similar to those encountered in the field. These tests were performed to evaluate specified reflectivity values.
4” tarp Samples50%
<5% 20%
30%
IKONOS V&V
40
Stennis Space Center
Desired Calibration Site Characteristics
• Bright and Uniform– High reflectance minimizes atmospheric uncertainties
– Uniformity allows statistically meaningful results
• Stable– Long term studies are facilitated with stability
• Low Precipitation and Cloud Cover– Maintains stability and increases chance of acquisition
• Large– Several swath widths in extent or more to avoid adjacency effects
IKONOS V&V
41
Stennis Space Center
Tarp Deployment at SSC
IKONOS V&V
42
Stennis Space Center
SDP Shipment Verification
Shipment Ver ification ReportContract No: Tracking ID No: Ve ndor: Task No: Mission No:
Transmittal Date: V&V Date Received Product Descriptor: Location:
Ve rify Shipment Contents. Product/Task # Images Volume (GB)
Observed Shipment Content and Count:
Are any of the materials listed in the Transmittal Letter missing?
Are any of the deliverables missing? (Calibration Report? Mission Report? Index Maps? Ground Truth Data?)
Ve rify Shipment Quality.
Is there any indication in the mission documents, that data with the wrong specifications was delivered?
Does geographic coverage match Delivery Order? Does acquisition date match Delivery Order?
List File Readability Problems or NA: Ingest Date: % Sampled:
List Image Quality Problems or NA: Method: % Sampled:
Discrepancy Report No.: Disposition of NC Products:
I have verified this shipment according to CRSP-WI-22 procedures.Primary Reviewer: Date:
Secondary Reviewer: Date:
CRSP Form 021 (2/2000)
• Shipment content and media count
• Mission document review
• Approximate geographic coverage
• Comparison with original task order
• File readability
• General image quality
• 100% visual inspection of JPEG quicklooks
• Discrepancy report (As Needed)
• Shipment tracking database
• Shipment verification report
IKONOS V&V
43
Stennis Space Center
NASA IKONOS V&V FocusIKONOS Data Product Characterization
• MTF and spatial resolution
• SNR estimates
• Geometric accuracies of imagery
• Radiometric accuracies of products
• Band-to-band registration of multispectral products
• Intercomparison of IKONOS, Landsat 7 and other satellite imagery
Quantitative System Characterizations:
1m GSD PAN 4m GSD MS
IKONOS V&V
44
Stennis Space Center
IKONOS Spectral Characterization
VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION
APPROACH Lab Funct.Collect
Std.Collect
TYPE
Band edge points at 50% peakresponse shall be within ± 0.01microns
Review laboratorymeasurements providedby SI
X Lab data
Slope through the 50% pointshall be at least 20% / 0.02microns (NIR long edge slope <20% / .04 microns)
Review laboratorymeasurements providedby SI
X Lab data
Out of band filter response <5.5% of total integratedtransmittance within 5%transmission points of that band
Review laboratorymeasurements providedby SI
X Lab data
The response for 70% of thedata centered on the peakresponse shall be within 20% ofthe maximum value. (NIRresponse for 35% of the datacentered on the peak responseshall be within 20% of themaximum value.
Review laboratorymeasurements providedby SI
X Lab data
IKONOS V&V
45
Stennis Space Center
IKONOS Spectral Bandpasses
IKONOS2 Relative Spectral Response
0
0.2
0.4
0.6
0.8
1
350 550 750 950
Wavelength (nm)
Rel
ativ
e S
pec
tral
R
esp
on
sivi
ty
Pan
Blue
Green
Red
NIR
Provided by Space Imaging
IKONOS V&V
46
Stennis Space Center
Well Characterized Site Method
Method: Utilize well characterized satellite systems (Landsat-7), not necessarily coincident, and well characterized sites (African deserts) to determine the radiometric accuracy of new commercial satellite systems (IKONOS)
Commercial Satellite
Calibrated Satellite
IKONOS V&V
47
Stennis Space Center
NASA CRSP Tarps
• Portable radiometric characterization of 4 m class GSD systems and smaller
• MTF characterization of 1 m class GSD systems and smaller
• SNR estimation on 1 m class GSD systems• Fiducial markers for remote uniform sites
IKONOS V&V
48
Stennis Space Center
CRSP Tarp Specifications
• Size and Shape– 20 m x 20 m
• Reflectivity– Spectral Range 400-1000 nm
– Tarp 1 < 5 %
– Tarp 2 20-25 %
– Tarp 3 30-40 %
– Tarp 4 50-55 %
• Spatial Uniformity – Standard deviation of 9 equally
space reflectivity measurements <1% of absolute reflectance
• Spectral Uniformity– <10% variation/100 nm band
• Deployability– Less than 2 hours with 2 people
in less than 3m/s wind
• Maintainability– Meet Specifications for 60
Deployments or two years which ever comes first