Upload
ted-habermann
View
551
Download
3
Embed Size (px)
DESCRIPTION
The NASA Earth Science Data and Information System (ESDIS) is migrating documentation for their data and products towards International Standards developed by ISO Technical Committee 211 (ISO/TC211). In order to do this effectively, NASA must understand and participate in the ISO process. This presentation was given at a NASA ISO Seminar during November 2012. It outlines the ISO standards process and describes some extensions to the ISO standards that are being proposed to address ESDIS requirements not addressed in the original standard.
Citation preview
ESDIS and International StandardsTed Habermann, NASA ISO Metadata WorkshopNovember, 2012
ISO Process
Preliminary Proposal Preparatory Committee Enquiry Approval Publication
New Work Item
Proposal
Draft New Work Item Proposal
Working Draft
or PAS
Technical Consensus
(DIS)
Final Draft International
Standard(FDIS)
International Standard
(IS)
5 months 2 months
0-6 months 6 months 12 months 9 months 3 months? months
ISO Project Options
Stage Proposal Preparatory Committee Enquiry Approval Publication
Develop and accept
committee draft
Develop and accept
enquiry draft
Publication of International
Standard
NormalProcedure
Acceptance of Proposal
Preparation of working
draft
Approval of FDIS
Draft With Proposal
FastTrack
Technical Specification
Technical Report
Publically Available Specification
Acceptance of Proposal
Acceptance of Proposal
Publication of International
Standard
Publication of International
Standard
Publication of Technical
Specification
Acceptance of Proposal
Acceptance of Proposal
Approval of FDIS
Approval of FDIS
Study by Working
Group
Prepare Draft Accept Draft
Develop and accept
committee draft
Develop and accept
enquiry draft
Accept enquiry draft
Approve draft PAS
Publication of Technical
Report
Publication of PAS
Accept Draft
ISO Timeline
P PAEC P PAEC
3 years 3-5 years 3 years
6-8 yearsImplementation
19115-1 (Metadata Conceptual Model: IS 2013/06)*19115-3 (Metadata Schema, Transform, Schematron: IS 2015/05)*
19115-2 (Metadata Acquisition / Images Conceptual Model)*19139-2 (Metadata Acquisition / Images Schema: TS 2012/10)*
19157 (Data Quality Conceptual Model: IS 2013/04) *19157 (Data Quality Schema)*
TS 19130 Parts 1-2 (Imagery Sensor Models for Geopositioning Conceptual Model: TS 2012/12)*
Conceptual Models and Implementations
191152003-05
191392006-083.3 Years
19115-22009-02
19139-22012-103.6 Years
19115-12013-06
19115-32015-051.9 Years
191572013-04
19157-2??
~50% Decrease
Now creating XML schemas directly from UML models using software developed in OGC Testbed. This capability is also being added directly into the tool by the vendor.
Test schemas for 19115-1 and 19157 with proposed NASA extensions are available now.
191302012-12
???
Model SchemaDelay
5.8 Years
Conventions and Extensions
Conventions are agreements that ensure consistent usage of the standard and facilitate interoperability across communities. They are expressed as guidance and best practices for the community.
Requirements not considered during the development of the original standard or that emerge during the implementation period can be addressed using extensions to the standard.
These extensions should be considered as candidate changes when the standard is revised every five years.
Developing NASA conventions and extensions:1) makes it possible to use the standard now and 2) increases the chances of addressing NASA needs in the next version of the
standard.
P PAEC P PAEC
Conventions and ExtensionsImplementation
NASA Documentation Requirements
ISO Standards
Users
GCMD
ECHO
DAACs/ECS
Missions
Science Teams
NASA Documentation Requirements
(Some) Existing Documentation Requirements
More than 2600 Collections have been described using the ECHO metadata dialect.
Almost 6000 DIFs include the word NASA.
ECHO Dialect
~150 Elements with content. Keywords, identifiers, and contact information are the most commonly occurring fields.
324 AdditionalAttributes occur over 3000 times
Most Common: PROCESSVERSION QAPERCENTGOODQUALITY QAPERCENTOTHERQUALITY QAPERCENTNOTPRODUCEDCLOUD QAPERCENTNOTPRODUCEDOTHER
DIF Dialect
~130 elements with content. Keywords, identifiers, and contact information are the most commonly occurring fields.
99% translation to ISO, ~79% translation to ECHO
More Overlap Than Difference
DIF
ISO
ECHO
The metadata dialects currently used by ESDIS have much more overlap than difference.
The mappings are generally well understood: we are in a tweaking stage.
The translations can be implemented using well-known, standard tools that are designed for XML processing.
These are different from the programming languages generally used for scientific data processing.
Additional AttributesThe ECHO AdditionalAttributes are an undifferentiated pile of information.
They can be translated to ISO, but we need to know where to put them. In order to do that, we need to identify their types.
AdditionalAttributes and ISO
citation.datecitation.identifiercitation.pointOfContactcontentInformationdescriptiveKeywordsdistributionInformationextentgeographicIdentifierinstrumentInformationlineageprocessingInformationqualityInformation
citation.datecitation.identifiercitation.pointOfContactcontentInformationdescriptiveKeywordsdistributionInformationextentgeographicIdentifierinstrumentInformationlineageprocessingInformationqualityInformation
Many of the AdditionalAttributes fit into standard ISO elements.
Others are implementation specific elements of existing ISO types.
Rules:Percent/Pct > contentInformationQuality > qualityInformationGain/Bias > instrumentInformation
75% of the attributes, 81% of content
citation.datecitation.identifiercitation.pointOfContactcontentInformationdescriptiveKeywordsdistributionInformationextentgeographicIdentifierinstrumentInformationlineageprocessingInformationqualityInformation
<eos:otherPropertyType> <gco:RecordType xlink:href="http://www.echo.nasa.gov/ingest/schemas/operations/Collection.xsd# xpointer(//element[@name='AdditionalAttribute'])">Echo Additional Attribute</gco:RecordType></eos:otherPropertyType><eos:otherProperty> <gco:Record xmlns:echo="http://www.echo.nasa.gov/ingest/schemas/operations"> <echo:AdditionalAttribute> <echo:Name>ScanAngle</echo:Name> <echo:DataType>float</echo:DataType> <echo:Description>The angle between the sensor view vector and the nadir axis</echo:Description> <echo:MeasurementResolution/> <echo:ParameterRangeBegin/> <echo:ParameterRangeEnd/> <echo:ParameterUnitsOfMeasure>degrees</echo:ParameterUnitsOfMeasure> <echo:ParameterValueAccuracy/> <echo:ValueAccuracyExplanation/> <echo:Value>47.5</echo:Value> </echo:AdditionalAttribute> </gco:Record></eos:otherProperty>
RecordType/Record - ISO with XML Reference
RecordType
Record
gmi:MI_AcquisitionInformation
MD_Metadata
MI_Instrument
+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString
MI_EnvironmentalRecord
+ averageAirTemperature : Real+ maxRelativeHumidity : Real+ maxAltitude : Real+ meteorologicalConditions : CharacterString
+ environmentalConditions 0..*
+ operation
0..*
MI_Platform
+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ description[0..1] : CharacterString+ sponsor[0..*]: CI_ResponsibleParty
MI_Plan
+ type[0..1] : MI_GeometryTypeCode+ status : MD_ProgressCode+ citation[0..*] : CI_Citation
MI_Operation
+ description[0..1] : CharacterString+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ status : MD_ProgressCode+ type[0..1] : MI_OperationTypeCode0
MI_AcquisitionInformation
+ platform 0..*
+ acquisitionPlan 0..*
+ instrument 0..*
+ objective 0..* + requirement
0..*
MI_Objective
+ identifier[1..*] : MD_Identifier+ priority[0..1] : CharacterString+ type[0..*] : MI_ObjectiveTypeCode+ function[0..*] : CharacterString+ extent[0..*] : EX_Extent
MI_Requirement
+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ requestor[1..*] : CI_ResponsibleParty+ recipient[1..*] : CI_ResponsibleParty+ priority : MI_PriorityCode+ requestedDate : MI_RequestedDate+ expiryDate : DateTime
+ mountedOn 0..1
eos:MI_AcquisitionInformation
MD_Metadata
MI_Instrument
+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record
MI_EnvironmentalRecord
+ averageAirTemperature : Real+ maxRelativeHumidity : Real+ maxAltitude : Real+ meteorologicalConditions : CharacterString
+ environmentalConditions 0..*
+ operation
0..*
MI_Platform
+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ description[0..1] : CharacterString+ sponsor[0..*]: CI_ResponsibleParty+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record
MI_Plan
+ type[0..1] : MI_GeometryTypeCode+ status : MD_ProgressCode+ citation[0..*] : CI_Citation
MI_Operation
+ description[0..1] : CharacterString+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ status : MD_ProgressCode+ type[0..1] : MI_OperationTypeCode0
MI_AcquisitionInformation
+ platform 0..*
+ acquisitionPlan 0..*
+ instrument 0..*
+ objective 0..* + requirement
0..*
MI_Objective
+ identifier[1..*] : MD_Identifier+ priority[0..1] : CharacterString+ type[0..*] : MI_ObjectiveTypeCode+ function[0..*] : CharacterString+ extent[0..*] : EX_Extent
MI_Sensor
+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record
MI_Requirement
+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ requestor[1..*] : CI_ResponsibleParty+ recipient[1..*] : CI_ResponsibleParty+ priority : MI_PriorityCode+ requestedDate : MI_RequestedDate+ expiryDate : DateTime
+ sensor 0..*
+ mountedOn 0..1
+ mountedOn 0..1
gmi:DQ_Lineage
MD_Metadata
DQ_DataQuality
+ scope : DQ_Scope+ lineage 0..1
LI_Lineage
+ statement [0..1] : CharacterString
LE_Source
+ description [0..1] : CharacterString+ scaleDenominator [0..1] : MD_RepresentativeFraction+ sourceReferenceSystem [0..1] : MD_ReferenceSystem+ sourceCitation [0..1] : CI_Citation+ sourceExtent [0..*] : EX_Extent+ processedLevel[0..1] : MD_Identifier+ resolution[0..1] : LE_NominalResolution
LE_ProcessStep
+ description : CharacterString+ rationale [0..1] : CharacterString+ dateTime [0..1] : DateTime+ processor [0..*] : CI_ResponsibleParty
+ source 0..*+ processStep 0..*
+ output, source0..*
If(count(source) + count(processStep) =0) and(DQ_DataQuality.cope.level = 'dataset' or 'series')then statement is mandatory
LE_Processing
+ identifier : MD_Identifier+ softwareReference[0..*] : CI_Citation+ procedureDescription[0..1] : CharacterString+ documentation[0..*] : CI_Citation+ runTimeParameters[0..1] : CharacterString
LE_Algorithm
+ citation: CI_Citation+ description : CharacterString
+ processingInformation0..*
+ algorithm 0..*LE_ProcessStepReport
+ name : CharacterString+ description[0..1] : CharacterString+ fileType[0..1] : CharacterString
+ report 0..*
+ sourceStep0..*
eos:DQ_Lineage
MD_Metadata
DQ_DataQuality
+ scope : DQ_Scope+ lineage 0..1
LI_Lineage
+ statement [0..1] : CharacterString
LE_Source
+ description [0..1] : CharacterString+ scaleDenominator [0..1] : MD_RepresentativeFraction+ sourceReferenceSystem [0..1] : MD_ReferenceSystem+ sourceCitation [0..1] : CI_Citation+ sourceExtent [0..*] : EX_Extent+ processedLevel[0..1] : MD_Identifier+ resolution[0..1] : LE_NominalResolution
LE_ProcessStep
+ description : CharacterString+ rationale [0..1] : CharacterString+ dateTime [0..1] : DateTime+ processor [0..*] : CI_ResponsibleParty
+ source 0..*+ processStep 0..*
+ output, source0..*
If(count(source) + count(processStep) =0) and(DQ_DataQuality.cope.level = 'dataset' or 'series')then statement is mandatory
LE_Processing
+ identifier : MD_Identifier+ softwareReference[0..*] : CI_Citation+ procedureDescription[0..1] : CharacterString+ documentation[0..*] : CI_Citation+ runTimeParameters[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record
LE_Algorithm
+ citation: CI_Citation+ description : CharacterString+ additionalCitation: CI_Citation+ version: CharacterString
+ processingInformation0..*
+ algorithm 0..*LE_ProcessStepReport
+ name : CharacterString+ description[0..1] : CharacterString+ fileType[0..1] : CharacterString
+ report 0..*
+ sourceStep0..*
NASA Documentation Requirements
ISO Standards
Users
GCMD
ECHO
DAACs/ECS
Missions
Science Teams
Granules/HDF
Identifiers
Semantic Mashups
User Input
Lineage / PROVData Quality
Preservation Objects
GIS
ServicesNASA Documentation
Requirements
Questions?