Upload
darren-oconnor
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Validated data and removal of bias through
Traceability to SI units
Nigel Fox
Centre for Optical and Analytical Measurement
Dec 03
Resolution adopted by CEOS Plenary 14 (Nov 2000)
1/ All EO measurement systems should be verified traceable to SI units for all appropriate measurands.
2/ Pre-launch calibration should be performed using equipment and techniques that can be demonstrably traceable to and consistent with the SI system of units, and traceability should be maintained throughout the lifetime of the mission.
Traceability – Property of the result of a measurement or the value of a standard whereby it can be related to stated references, usually through an
unbroken chain of comparisons all having stated uncertainties
Vocabulary for International Metrology (VIM)
SI units – The coherent system of units adopted and recommended by the General Conference of Weights and Measures (CGPM).
Reproducibility of results of measurements – Closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurement.
Accuracy of measurement – Closeness of the agreement between the result of a measurement and a true value of the measurand.
Precision – No metrological definition except to state that it should never be used in the context of “accuracy” and, because of possible confusion its use, should normally be avoided in metrological applications.
Repeatability of results of measurements – closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurement.
Uncertainty of measurement – Parameter, associated with the result of a measurement, that characterises the dispersion of the values that could reasonably be attributed to the measurand.
Error of measurement – Result of a measurement minus a true value of the measurand Stability – Ability of a measuring
instrument to maintain constant its metrological characteristics with time.
Traceability – Property of the result of a measurement or the value of a standard whereby it can be related to stated references, usually through an unbroken chain of comparisons all having stated uncertainties.
Precision – No metrological definition except to state that it should never be used in the context of “accuracy” and, because of possible confusion its use, should normally be avoided in metrological applications.
Uncertainty of measurement – Parameter, associated with the result of a measurement, that characterises the dispersion of the values that could reasonably be attributed to the measurand.
Convention of the Meter – established 1875
RM ORegional M etro logy Organisation
BIPMBureau de In ternational
des Poids et M esures
CCPRConsultative Com m ittee
for Photom etry and Radiom etry
CCTConsultative Com m ittee
for Therm om etry
CCMC onsultative C om m ittee for
M ass and R elated Q uantitiesCC...
C IPMCom ite In ternational des Poids et M easures
CPGMConference Generale des Poids et M esures
66 m em ber & associate states
SI Traceability: The Mututal Recognition Arrangement (MRA)
SIM
CCPR Key comparisons
• Spectral Irradiance
• Spectral Responsivity
• Luminous intensity
• Luminous Flux
• Spectral transmittance
• Spectral diffuse reflectance
(total hemispherical)
CCPR
Monitoring and interpreting the Earth’ systems
Solar Reflected RadiationAtmosphere - Aerosol (size & distribn)
- Clouds - Pollution (impact on health)
Water - Pollution (originator)- Algae plumes
Land - Useage / condition - Type/quantity of vegetation
- Minerals - Carbon & hydrological cycles
Governments – Treaties, Tax, Planning
Spatial variability requires good stability and SNR (signal to noise ratio) from a single sensor - but long term studies “climate change” need accuracy and consistency
Engineering specification of SNR should equate to accuracy
Thermal Emitted RadiationAtmosphere – Atmospheric chemistry Water – Temperature Land – Fires, Volcanoes, Pollution,
Incoming Solar RadiationDrives all the processes of the Earth
System and potentially damaging (UV) to Biosphere (Human health)
Need for improved Quality Assurance
Requirement - baseline for climate studies
- improve models- prediction of weather systems- identify crops from weeds
- global warming - Man or Nature?
- detection of change
- monitoring the treaties - auditing carbon sinks - efficiency of carbon sinks
- QA of operational services (GMES)
- instrument synergy
Difficulties - bias between sensors
- instruments change on launch and degrade in-orbit (gain and spectral)
To strengthen the evidence
Need for improved Quality Assurance
- application of correction for atmosphere loss
- lack of cohesion between networks and “ground truth” validation data (atmosphere an exception)
- models inadequate
- no consistent statements of uncertainty or
confidence.
Anomalies in NOAA/AVHRR data
N-6 N-11N-7 N-9 N-14
Normalised Difference Vegetation Index (NDVI) over “stable” desert as measured by AVHRR
–Demonstrates both in-flight “ageing” and initial calibration biases
Temporal change difficult to identify even using “identical” instrumentation without normalisation
Total solar Irradiance (Solar constant) –only normalisation allows a long term record to be established (Biases are 10 X larger than necessary to detect impact on climate change)
LOS 1998 IEEE Trans G & RS
Traceability chain for optical radiation measurement
Optical power =Po
Electrical Heater Power = PEAbsorbing black coating
Copper disk
When thermometer temperature T=To=TE then Po=PE
Electrical Substitution Radiometry a 100 yr old technology
Mechanical cryogenic cooler “Fridge” (T = 20 K)
Principle of Cryogenic radiometry
Optical power =Po
Absorbing cavity (~ 0.99999)Electrical Heater Power = PE
Cooling improves sensitivity by 1000 X
Thermal shroud When T =To=TE then Po=PE
Shutter
25 yrs of cryogenic radiometry at NPL
Primary standard lamp
Working standard lamp
Cal Lab Primary lamp
Cal Lab working std Lamp
User Cal Lamp
User Instrument
Spectroradiometer
Spectroradiometer
Spectroradiometer
Spectroradiometer
Fundamental constants (SI)
Primary standard cryogenic radiometer
LaserCal interval ~100nm
LaserCal interval ~0.1 nm
Photodiode (spectral responsivity
Filter Radiometer
Radiance Temperature
Ultra High Temperature Black Body (3500 K)
Radiance continuum (Planck)
Spectroradiometer (multi-band filter radiometer
Spectral Radiance/Irradiance calibrations LAND OCEAN ATMOSPHERE
Satellite Pre-flight Calibration
Traceability ??
Satellite In-flightCalibration
Data products
Atmosphere/Model
Vicarious
Lamp Solar illuminated Diffuser
Main uncertainty components: k = 1 confidence level
Uncertainty sources with an effect on irradiance less than 0.03%- Blackbody temperature:
- Size of source effect- Mathematical approximations- Geometric Factor- Electronics- Filter radiometer relative shape
-Blackbody absorption (apart from ~380 nm)-Blackbody-integrating sphere geometry-SRIPS linearity-Monochromator wavelength error-Monochromator bandwidth and subsequent mathematical approximations-Lamp current control
Lamp alignment
SRIPS repeatability
Blackbody Emissivity
Lens Transmission
FR absolute responsivity
Blackbody Stability
Blackbody Uniformity
0.06%0.06%0.06%
0.07%0.08%0.26%
0.02%0.03%0.05%0.089 K
0.02%0.03%0.05%0.103 K
0.03%0.03%0.06%0.115 K
0.06%0.07%0.13%0.258 K
0.07%0.09%0.17%0.327 K
T
E
M
P.
700 nm550 nm300 nmIndividual
uncertainty
Effective uncertainty, 3050 K BB
Uncertainty component
S R IP S p r im a r y s c a le u n c e r ta in ty : O v e r a l l a t 9 5 % c o n f id e n c e le v e l
0 .0 %
0 .5 %
1 .0 %
1 .5 %
2 .0 %
2 .5 %
3 .0 %
3 .5 %
4 .0 %
4 .5 %
2 5 0 5 0 0 7 5 0 1 0 0 0 1 2 5 0 1 5 0 0 1 7 5 0 2 0 0 0 2 2 5 0 2 5 0 0W a v e le n g th (n m )
Radi
ance
unc
erta
inty
(%)
P r im a ry S c a le
B B ra d ia n c e
S R IP S R e p e a ta b il i ty
U V F W m o d e ll in g
L a m p p e r f o r m a n c e : F E L s ( a p p r o x 1 in 3 s h o w a d e v ia t io n )
- 1 4 .0 0 %
-1 2 .0 0 %
-1 0 .0 0 %
-8 .0 0 %
-6 .0 0 %
-4 .0 0 %
-2 .0 0 %
0 .0 0 %
2 .0 0 %
4 .0 0 %
2 5 0 5 0 0 7 5 0 1 0 0 0 1 2 5 0 1 5 0 0 1 7 5 0 2 0 0 0 2 2 5 0 2 5 0 0
W a v e le n g th
Dif
fere
nc
e
Traceability for in-flight / in-situ / vicarious calibration
Spectral Radiance
- lamp illuminated spheres
- Filter radiometers (spectroradiometers)
- Irradiance source + diffuser
Lamp + spectralon or ….
Sun + spectralon or ….
Sun + Moon
Spectral Reflectance
- in-situ absolute ratio (using radiometers)
- Ratio to “standard” reflector/diffuser
Via models / atmosphere correction
to satellite for cal/val
(radiances)
To bio/geophysical quantities (refelectances)
The NPL diffuse reflectance scale is derived goniometrically for the spectral region 300 to 2500 nm
Uncertainty of <0.2 % in the visible and shown equivalence with NIST
Diffuse reflectance (BRDF)
SampleDetectorSpectralon 400 nm
0.65
0.75
0.85
0.95
1.05
-90 -60 -30 0 30 60 90
radi
ance
fact
or
Beta-NPL (400 nm), Tile 1
Beta-PTB (400 nm), Tile 1
Beta-NPL (400 nm), Tile 2
Beta-NIST (400 nm), Tile 2
Validated data products require all processing steps and data to be QA – Accredited?
Pre-flight- User specification- Instrument build compliance- Calibration?
Post-launch- In-flight checks- Ground “Truth” comparison- Inter-sensor cross
calibration Processed data released
- “validated”- Uncertainty statement?
Rare for all these activities to have been independently reviewed and/or audited
Global Monitoring Environment and Security (GMES)Joint initiative of ESA and EU
Aim: to establish “operational services” for Earth Observation data to meet needs of key stakeholders , public services, private industry,
academia and the citizen with a view to financial self-sufficiency.
Robust evidence requires robust QA
Success requires: - Combination of data from many sources, (satellites, in-situ, aircraft) - Efficient production of cost effective, reliable, data products / maps - Data must provide the evidence to allow decisions to be taken with confidence. - Innovation in measurement and analysis
Reliability: Implies Quality assurance and statements of confidence associated with data (not only for end users but also
“operational service” providers
Users generally assume QA
Infrastructure for innovation in measurement, validation and QA of EO data
Public sector
Private Industry
Academia
QA
Advice
Calibration
TraceabilityAudit Validation
• Transfer standards
• Comparisons
• Innovation on techniques
• Measurement & test protocols
• International link
• Independence
In-situ
Pre-flight
airbornePost-launch
Modelling & Data processing
NPL ++
NIST ++
Summary Primary scales, transfer standards and techniques now allow high
accuracy to be achieved for both pre-flight and vicarious calibration (particularly for radiance)
All aspects/steps of producing EO data products needs validation and traceability (instrument calibration and algorithms/models)
Consistent presentation and breakdown of uncertainty budgets
Flexibility to allow innovation
Regular comparisons to evaluate biases
Establish well characterised ground targets as “reference standards”
Develop improved “in-flight” calibration methods e.g TRUTHS (Fox et al Proc. SPIE 4881, p395 2003)
For Earth Observation to provide the evidence to support policy
requires the industry and its data to be as robust as traditional industries