Metadata/Time-Date Tools (Toolkit_MTD)

Metadata/Time-Date Tools

(Toolkit_MTD)

Abe Taaheri , Megan Donovan Spencer , Larry Klein(Space Applications Corp.)

Doug Ilg (RSTX) September 23, 1998

EOS

NASA'S MISSION TO PLANET EARTH

EARTH OBSERVING SYSTEM

EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-2

Overview

• Science Data Production (SDP) TOOLKIT

• Metadata/Time-Date Tools (TOOLKIT_MTD)

• Metadata Tools

• Time/Date Tools

• Example (call sequences)

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-3

SDP TOOLKIT

• SDP TOOLKIT is an interface between EOS instrument data

producing software and the ECS production archival system.

• Besides the tools handling metadata and time/date conversion, it

includes many other tools such as :

- File I/O tools (IO)

- Error/Status Reporting (SMF)

- Process Control Tools (PC)

- Shared Memory Management Tools (MEM)

- Spacecraft Ephemeris and Attitude Data Access Tools (EPH)

- Digital Elevation Model Tools (DEM)

- Ancillary Data Tools (AA)

- Celestial Body Position Tools (CBP)

- Geo-Coordinate Transformation Tools (GCT)

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-4

Science Data Production andArchival Scenario

Science Data Processing

SDP Toolkit

Product Generation

Executable (PGE)

(Generated by ScienceData Server)

Metadata

ConfigurationFile

2

(ASCII File)

InventoryMetadata

(HDF)

Product

Scientific Data Set

Structural Metadata

(HDF-EOS Only)

Inventory Metadata

Archive Metadata

Scientific Data Set

(Generated byProcessing)

PCFFile

Inputs

4

3

1

5

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-5

SDP TOOLKIT (cont.)

• EOSDIS extended data providers will not need the entire TOOLKIT to

handle metadata and data structures (HDF-EOS, HDF)

• This necessitated creating a smaller package to allow these

providers the capability of formatting their products in ECS standard

formats, without requiring the entire SDP toolkit package

• The smaller package could also be used by application developers

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-6

Toolkit_MTD

• Tools relevant to metadata and time/date conversion tools areextracted from Science Data Processing (SDP) Toolkit

• Process Control File, that contains Process Control informationfor I/O, is replaced by a short file mapping logical identifiers tophysical file names

• Errors are reported as in HDF and HDFEOS

• Installation Scripts are modified for quick installation of HDF,HDFEOS, and Toolkit_MTD

• New functionality added to read Metadata from ASCII file

• Tested on SUN and SGI platforms, will be ported to other unixplatforms; HP, DEC, IBM, and also Windows and Mac

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-7

Metadata Tools

• PGS_MET_SetFileId()

Sets logical IDs assigned for the user defined files.

• PGS_MET_GetFileId(char *filename)

Retrieves logical ID assigned for a file.

• PGS_MET_Init(PGSt_PC_Logical fileId, PGSt_MET_all_handles mdHandles)

Initializes MCF file containing metadata.

• PGS_MET_Init_NonMCF(PGSt_PC_Logical fileId, PGSt_MET_all_handles mdHandles)

Initializes an ASCII file containing metadata.

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-8

Metadata Tools (cont.)

• PGS_MET_SetAttr(PGSt_MET_handle mdHandle,char *attrNameStr,void *attrValue)

Assigns values to metadata attributes (after initializing MCF file).

• PGS_MET_GetSetAttr(PGSt_MET_handle mdHandle, char* attrNameStr, void* attrValue)

Retrieves values that are set automatically, or by the user.

• PGS_MET_GetPCAttr(PGSt_PC_Logical fileId, PGSt_integer version, char *hdfAttrName, char* Name, void * parmValue)

Reads metadata parameters held in HDF attributes or in aseparate ASCII file.

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-9

Metadata Tools (cont.)

• PGS_MET_GetConfigData(char* attrName, void* attrValue)

Certain configuration parameters are held in theconfigfile as “10220|REMOTEHOST|sandcrab”.

This tool would retrieve the value "sandcrab" from theconfigfile given the name of the parameter "REMOTEHOST".

• PGS_MET_Write(PGSt_MET_handle mdHandle, char * hdfAttrName, PGSt_integer hdfFileId)

Writes metadata after all the metadata parameters are set inmemory. Tool checks that all the mandatory parameters are set.

• PGS_MET_Remove()Removes ODL representation of all MCF files and someinternal files used by the MET tools.

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-10

Time/Date Tools

• PGS_TD_UTCtoTAI( char asciiUTC[28],

PGSt_double *secTAI93 )

Converts UTC time in CCSDS ASCII Time Code (A or B format)

to Toolkit internal time (real continuous seconds since

12AM UTC 1-1-93).

• PGS_TD_TAItoUTC( PGSt_double secTAI93,

char asciiUTC[28])

Converts Toolkit internal time (real continuous seconds

since 12AM UTC 1-1-93) to UTC time in CCSDS ASCII

Time Code A format.

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-11

Time/Date Tools (cont.)

• PGS_TD_TAItoTAIjd( )

• PGS_TD_TAIjdtoTAI( )

• PGS_TD_TAItoGAST( )

• PGS_TD_UTC_to_Sctime( )

• PGS_TD_SCtime_to_UTC( )

• PGS_TD_ASCIItime_AtoB( )

• PGS_TD_ASCIItime_BtoA( )

• PGS_TD_UTCtoGPS( )

• PGS_TD_GPStoUTC( )

• PGS_TD_UTCtoTDTjed( )

• PGS_TD_UTCtoTDBjed( )

• PGS_TD_TimeInterval( )

• PGS_TD_UTCtoUTCjd( )

• PGS_TD_UTCjdtoUTC( )

• PGS_TD_UTCtoUT1( )

• PGS_TD_UTCtoUT1jd( )

• PGS_TD_LeapSec( )

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-12

Example

/* include files */

#include <PGS_MET.h>

#include <PGS_tk.h>

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <hdf.h>

#include <mfhdf.h>

#include <PGS_SMF.h>

#define INVENTORYMETADATA 1

#define ARCHIVEDMETADATA 2

#define ODL_IN_MEMORY 0

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-13

Example (cont.)

int main()

{

/***** Declarations *****/

PGSt_MET_all_handles mdHandles;

PGSt_SMF_status ret = PGS_S_SUCCESS;

PGSt_integer fileId, sdid, value,rtid;

PGSt_integer i, ival;

PGSt_integer version;

PGSt_double dval;

char *sval[5];

char *start_time = "";

char my_HDF_file[PGSd_MET_FILE_PATH_MAX]="";

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-14

Example (cont.)

/**********************************

Associate logical IDs with physical

filenames

***********************************/

ret = PGS_MET_SetFileId();

/* get values from Config file */

ret = PGS_MET_GetConfigData(“Runtime_ID”,&rtid);

ret = PGS_MET_GetConfigData(“VERSION_NUMBER”,&ival);

ret = PGS_MET_GetConfigData(“STARTTIME”,

&start_time);

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-15

Example (cont.)

/* get value from metadata block of input file */

ret = PGS_MET_GetPCAttr(10265,1,INVENTORYMETADATA,

"EquatorCrossingLongitude",&dval);

/* get value from ASCII metadata file */

ret = PGS_MET_GetPCAttr(5731,1,”coremetadata”,

”TestingAttribute1",&sval);

for(i=0; i<1; i++) printf(“%s”,sval);

printf(“\n”);

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-16

Example (cont.)

/* Initialize MCF file */

fileId = 10250;

ret = PGS_MET_Init(fileId,mdHandles);

if (ret !=PGS_S_SUCCESS)

{

printf("initialization failed\n");

exit(1);

}

/* assign value to attribute in MCF */

ret = PGS_MET_SetAttr(mdHandles[1],

"LocalVersionID",&ival);

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-17

Example (cont.)


”EquatorCrossingLongitude",&dval);


”RANGEBEGININGDATETIME",&start_time);


"ParameterValue.1",57.5)

/* assign value to attribute in MCF

in Archive block */


"Runtime_ID",&rtid);

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-18

Example (cont.)

/* Read back in value of attribute in memory */

ret = PGS_MET_GetSetAttr(mdHandles[2],

"Runtime_ID",&rtid);

/* Write metadata to HDF and ASCII files */

version =1;

fileId = 5039;

ret = PGS_PC_GetReference(fileId,

&version, my_HDF_file);

if (ret == PGS_S_SUCCESS)

{

sdid = SDstart(my_HDF_file,DFACC_CREATE);

}

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-19

Example (cont.)

/* write INVENTORYMETADATA to HDF file */

ret = PGS_MET_Write(mdHandles[INVENTORYMETADATA],

"coremetadata",sdid);

/* write ARCHIVEDMETADAT to HDF file */

ret = PGS_MET_Write(mdHandles[ARCHIVEDMETADATA],

"archivemetadata",sdid);

/* write to non-HDF file */

fileId = 5804;

ret = PGS_MET_Write(mdHandles[ODL_IN_MEMORY],

NULL,fileId);

EOS



EA

RT

H P

RO

BE

S

DA

TA

INF

OR

MA

TIO

N S

YS

TE

M

LK-20

Example (cont.)

/* write to default non-HDF file */

ret = PGS_MET_Write(mdHandles[ODL_IN_MEMORY],

NULL, NULL);

SDend(sdid);

/* Remove MCF from memory */

ret = PGS_MET_Remove();

}

Technology

Metadata/Time-Date Tools (Toolkit_MTD)