21
05/23/2006 SM23A-06 1 A Virtual Observatory A Virtual Observatory for the Ionosphere- for the Ionosphere- Mesosphere- Mesosphere- Thermosphere Community Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. Yee, and E. Talaat Yee, and E. Talaat (Johns Hopkins University, Applied Physics Laboratory 11100 (Johns Hopkins University, Applied Physics Laboratory 11100 Johns Hopkins Rd. Laurel MD, 20723: 240-228-4172) Johns Hopkins Rd. Laurel MD, 20723: 240-228-4172) J. Russell J. Russell (Hampton University) (Hampton University) R. Heelis R. Heelis (Univ. Texas at Dallas) (Univ. Texas at Dallas) J. Kozyra J. Kozyra (Univ. of Michigan) (Univ. of Michigan) D. Bilitza, R. McGuire, and R. Candey D. Bilitza, R. McGuire, and R. Candey (NASA Goddard Spaceflight Center) (NASA Goddard Spaceflight Center) P. Fox P. Fox (HAO/NCAR) (HAO/NCAR)

05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

Embed Size (px)

DESCRIPTION

05/23/2006 3SM23A-06 What is a VxO? An observatory is a location used for observing terrestrial and/or celestial events. An observatory is a location used for observing terrestrial and/or celestial events. A virtual observatory is a system that allows an observer to work as if they were at the observatory. A virtual observatory is a system that allows an observer to work as if they were at the observatory. The VO does this typically by allowing the observer to access multiple data files (“observations”) from multiple sources of the object of interest. The VO does this typically by allowing the observer to access multiple data files (“observations”) from multiple sources of the object of interest. The VO allows access to “different instruments” to study a given system or phenomena. The VO allows access to “different instruments” to study a given system or phenomena. A VxO is a virtual observatory that focuses on a particular domain such as solar, magnetospheric, or ITM. A VxO is a virtual observatory that focuses on a particular domain such as solar, magnetospheric, or ITM.

Citation preview

Page 1: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 SM23A-06 1

A Virtual Observatory A Virtual Observatory for the Ionosphere-for the Ionosphere-

Mesosphere-Mesosphere-Thermosphere Thermosphere

CommunityCommunityD. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. Yee, and E. D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. Yee, and E. TalaatTalaat

(Johns Hopkins University, Applied Physics Laboratory 11100 Johns (Johns Hopkins University, Applied Physics Laboratory 11100 Johns Hopkins Rd. Laurel MD, 20723: 240-228-4172) Hopkins Rd. Laurel MD, 20723: 240-228-4172)

J. RussellJ. Russell(Hampton University)(Hampton University)

R. HeelisR. Heelis(Univ. Texas at Dallas)(Univ. Texas at Dallas)

J. KozyraJ. Kozyra(Univ. of Michigan)(Univ. of Michigan)

D. Bilitza, R. McGuire, and R. CandeyD. Bilitza, R. McGuire, and R. Candey(NASA Goddard Spaceflight Center)(NASA Goddard Spaceflight Center)

P. FoxP. Fox(HAO/NCAR)(HAO/NCAR)

Page 2: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 2SM23A-06

Virtual ITM ObservatoryVirtual ITM Observatory The Virtual ITM Observatory (VITMO) The Virtual ITM Observatory (VITMO)

is one of the recently awarded is one of the recently awarded domain specific NASA virtual domain specific NASA virtual observatories (VxO).observatories (VxO).

VITMO will focus on data covering VITMO will focus on data covering the ITM region as well as providing the ITM region as well as providing connections to the energy drivers for connections to the energy drivers for this region (principally solar and this region (principally solar and magnetospheric). magnetospheric).

Page 3: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 3SM23A-06

What is a VxO?What is a VxO? An An observatoryobservatory is a location used for observing is a location used for observing

terrestrial and/or celestial events. terrestrial and/or celestial events. A virtual observatory is a system that allows an A virtual observatory is a system that allows an

observer to work as if they were at the observatory.observer to work as if they were at the observatory. The VO does this typically by allowing the observer The VO does this typically by allowing the observer

to access multiple data files (“observations”) from to access multiple data files (“observations”) from multiple sources of the object of interest.multiple sources of the object of interest.

The VO allows access to “different instruments” to The VO allows access to “different instruments” to study a given system or phenomena.study a given system or phenomena.

A VxO is a virtual observatory that focuses on a A VxO is a virtual observatory that focuses on a particular domain such as solar, magnetospheric, or particular domain such as solar, magnetospheric, or ITM.ITM.

Page 4: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 4SM23A-06

Virtual Observatory Concepts Discovery of Data

“one-stop shopping” for the ITM community Browsing - more than data query, enhanced by use of

meta-data Different views as data use differs between users

Delivery of / access to readers, and other tools ITM is a heterogeneous data community with diverse

file formats and granularity of observables Distributed Access

Data providers have an integral role to provide data access, description, and meta-data

Adaptable framework responsive to community needs and requests

Comprehensive and historical

Page 5: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 5SM23A-06

ITM is Diverse in Observables

An ITM VO must facilitate correlative study between multiple parameters from multiple sources and cross-discipline studies.

Page 6: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 6SM23A-06

ITM Domain DifferencesITM Domain Differences Data covering the ITM region has a number of differences Data covering the ITM region has a number of differences

from data in other domains. from data in other domains. The ITM region is observed by ground based remote sensing The ITM region is observed by ground based remote sensing

instruments, satellite based remote sensing instruments, and instruments, satellite based remote sensing instruments, and in-situ satellite instruments. in-situ satellite instruments.

There are external drivers in solar radiation and the solar There are external drivers in solar radiation and the solar wind and magnetospheric particle inputs. wind and magnetospheric particle inputs.

A Virtual Observatory that covers the ITM region needs to A Virtual Observatory that covers the ITM region needs to deal with the large diversity of data types in the study of this deal with the large diversity of data types in the study of this region. region.

A heterogeneous data format environment also exists here. A heterogeneous data format environment also exists here. Solar imagery is typically in FITS files, ground based data are Solar imagery is typically in FITS files, ground based data are

typically ASCII or NetCDF, in-situ satellite data are typically ASCII or NetCDF, in-situ satellite data are commonly in CDF format, remote sensing satellite data are commonly in CDF format, remote sensing satellite data are typically HDF, NetCDF, or CDF. typically HDF, NetCDF, or CDF.

Non-NASA data sources (i.e. NSF, NOAA, DoD) are important Non-NASA data sources (i.e. NSF, NOAA, DoD) are important contributors to the ITM domain.contributors to the ITM domain.

Page 7: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 7SM23A-06

The Ability to Find the The Ability to Find the Appropriate Data Will Be a Key Appropriate Data Will Be a Key

Requirement of a VITMORequirement of a VITMO

Increasingly satellites in the ITM community will rely on remote sensing instruments – TIMED, DMSP, NPOESS

Page 8: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 8SM23A-06

How the VITMO Will Be How the VITMO Will Be DifferentDifferent

For a VxO to be truly useful it must go beyond traditional mission data centers. For a VxO to be truly useful it must go beyond traditional mission data centers. It should be able to It should be able to interrogate multiple data centers, finding products that interrogate multiple data centers, finding products that

overlap in time and/or location. overlap in time and/or location. It must allow a It must allow a higher level of searchhigher level of search to be performed than previously available. to be performed than previously available.

An example of such a complex search would be something like the following: An example of such a complex search would be something like the following: “what near Earth orbiters are observing the auroral region when Bz is negative “what near Earth orbiters are observing the auroral region when Bz is negative and the solar wind speed is greater than 400 km / sec.?” and the solar wind speed is greater than 400 km / sec.?” Today, this would Today, this would require a manual process where the researcher would first find plots of the require a manual process where the researcher would first find plots of the Earth’s magnetic field component (Bz) and plots of solar wind speed from Earth’s magnetic field component (Bz) and plots of solar wind speed from NASA’s CDAWeb (http://cdaweb.nasa.gsfc.gov). After they manually looked for NASA’s CDAWeb (http://cdaweb.nasa.gsfc.gov). After they manually looked for overlap conditions they would utilize these time periods to search known Earth overlap conditions they would utilize these time periods to search known Earth orbiters to find out if they were observing during those time periods. They would orbiters to find out if they were observing during those time periods. They would then have to manually review summary products to determine if those orbiters then have to manually review summary products to determine if those orbiters were viewing the auroral region during those intervals. were viewing the auroral region during those intervals. VITMO will handle these VITMO will handle these types of queries automatically.types of queries automatically.

VITMO will also be able to organize toolsVITMO will also be able to organize tools, whether plotting, subsetting, or , whether plotting, subsetting, or analysis tools by the type of data they are to be applied to as well as the types of analysis tools by the type of data they are to be applied to as well as the types of operations that are to be performed. If the user requested time series data then operations that are to be performed. If the user requested time series data then tools appropriate for operating on the time series should be presented to them. If tools appropriate for operating on the time series should be presented to them. If they chose images then tools appropriate for images should be made available. If they chose images then tools appropriate for images should be made available. If the data were in CDF format, then FITS format data tools need not be presented. the data were in CDF format, then FITS format data tools need not be presented. Additionally, VITMO will understand that model output can be treated as high Additionally, VITMO will understand that model output can be treated as high level data products and that models should be available, just as tools are, to the level data products and that models should be available, just as tools are, to the end user.end user.

Page 9: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 9SM23A-06

Page 10: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 10SM23A-06

VITMO Architecture VITMO Architecture OverviewOverview

high-levelhigh-level conceptual model of the scientific disciplines conceptual model of the scientific disciplines and domain data in the region from the Sun to the Earthand domain data in the region from the Sun to the Earth

Regions – Solar, Near-Earth – F-region, E-regionRegions – Solar, Near-Earth – F-region, E-region Features – exist for long periods of timeFeatures – exist for long periods of time Events – Occur at particular points in timeEvents – Occur at particular points in time

description of resource metadata in the Knowledge Base description of resource metadata in the Knowledge Base supporting any accessible scientific resourcesupporting any accessible scientific resource

descriptive metadatadescriptive metadata includes program, satellite and instrument includes program, satellite and instrument information, as well as data details like observed phenomena, information, as well as data details like observed phenomena, units of measure, time and spatial coverage, processing level units of measure, time and spatial coverage, processing level

structural metadatastructural metadata defines the details for data access and defines the details for data access and retrieval if selected by the use retrieval if selected by the use

expand the notion of a data resource in our metadata definition expand the notion of a data resource in our metadata definition to also include databases and computer programs (such as to also include databases and computer programs (such as format conversion tools or data models) that generate requested format conversion tools or data models) that generate requested data on demand. Thus, file access, database access, and program data on demand. Thus, file access, database access, and program invocation details are all included in the resource metadata invocation details are all included in the resource metadata model. model.

Page 11: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 11SM23A-06

Coupling in VITMO Coupling in VITMO ArchitectureArchitecture

a a loose couplingloose coupling between the conceptual model between the conceptual model and the resource metadata.and the resource metadata. A query in the conceptual model identifies the desired A query in the conceptual model identifies the desired

parameters, which are then mapped to the related data parameters, which are then mapped to the related data resources. Data discovery is simplified by performing resources. Data discovery is simplified by performing requests at the conceptual level, permitting users without requests at the conceptual level, permitting users without knowledge of specific mission, or instrument to readily knowledge of specific mission, or instrument to readily find data of interest. find data of interest.

conceptual model can be readily expanded if new classes conceptual model can be readily expanded if new classes of scientific concepts or new relationships between of scientific concepts or new relationships between scientific concept classes would improve the model's scientific concept classes would improve the model's representation of the Sun to Earth domain. representation of the Sun to Earth domain.

a data provider can add resources (of any format, access a data provider can add resources (of any format, access method, or processing level) simply by providing method, or processing level) simply by providing information that describes the data and its access details.information that describes the data and its access details.

model eliminates the need for rigid metadata standards model eliminates the need for rigid metadata standards or unique, resource-specific integration software or unique, resource-specific integration software

Page 12: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 12SM23A-06

VITMO Based on VITMO Based on ServicesServices

Page 13: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 13SM23A-06

Enhanced Search Enhanced Search ServicesServices

Today’s researcher needs to find datasets from Today’s researcher needs to find datasets from different missions that overlap in time and space. different missions that overlap in time and space.

Many of today’s satellite missions include remote Many of today’s satellite missions include remote sensing instruments. sensing instruments. Finding cross comparisons Finding cross comparisons between instruments is more difficult because between instruments is more difficult because we must be concerned not just with spacecraft we must be concerned not just with spacecraft location, but also with multiple look-points for location, but also with multiple look-points for the instruments.the instruments.

The prime NASA mission for study of the MLT The prime NASA mission for study of the MLT region is the TIMED satellite. All of the region is the TIMED satellite. All of the instruments on this satellite are remote sensing instruments on this satellite are remote sensing instruments. Each remote sensing instrument instruments. Each remote sensing instrument has a different look geometry, some looking as has a different look geometry, some looking as far as 2000 km away from the satellite. far as 2000 km away from the satellite.

Page 14: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 14SM23A-06

Enhanced Search Services Enhanced Search Services ContinuedContinued

AIM will have a fly’s eye imager.AIM will have a fly’s eye imager. The fields-of-view (regard) of these The fields-of-view (regard) of these

instruments define the extent of the instruments define the extent of the data products.data products.

This location information is This location information is notnot contained in the existing metadata in contained in the existing metadata in today’s catalogs.today’s catalogs.

Page 15: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 15SM23A-06

Virtual Metadata GeneratorsVirtual Metadata Generators VITMO will employ special coincidence calculators that VITMO will employ special coincidence calculators that

know the viewing geometries of the various know the viewing geometries of the various instruments.instruments.

The TIMED coincidence calculator is an example of a The TIMED coincidence calculator is an example of a tool which when converted into a web service can be tool which when converted into a web service can be used to augment existing data querying.used to augment existing data querying.

The TIMED coincidence calculator has recently been The TIMED coincidence calculator has recently been imbedded into the TIMED query system.imbedded into the TIMED query system.

These calculators will operate as web services inside These calculators will operate as web services inside the VITMO query system (or by outside query systems).the VITMO query system (or by outside query systems).

These calculators generate metadata for the These calculators generate metadata for the coincidence of interest which is used by the catalog coincidence of interest which is used by the catalog system to answer your query.system to answer your query.

The metadata generated by this service is discardedThe metadata generated by this service is discarded after it is after it is used – hence “virtual metadata”.used – hence “virtual metadata”.

Page 16: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 16SM23A-06

Enhanced Search ResultsEnhanced Search Results Addition of virtual Addition of virtual

metadata metadata capabilities allows capabilities allows pre-selection of pre-selection of data for analysis.data for analysis.

VITMO search VITMO search will deliver results will deliver results from searches from searches such as “What such as “What GUVI data that GUVI data that contains 23-01 contains 23-01 MLT during active MLT during active geomagnetic geomagnetic conditions?”conditions?”

Page 17: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 17SM23A-06

Prototype DisplayPrototype DisplayThe left hand side of the web page allows the user to search or browse for resources (be they data, models, tools, etc) by the particular region of space, time, satellite coincidences or other criteria.

After resources are searched for the results are displayed and possible display plots may be shown.

If the user selects the “Related Tools and Models” button on the top of the page a display of appropriate tools and models for the data or resources chosen will be presented.

This list of tools and models is generated dynamically based on the metadata in the VITMO catalog describing the data, tools, and models available through the VITMO.

Page 18: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 18SM23A-06

Uniqueness of the Proposed Uniqueness of the Proposed Approach Approach

Unique search capabilities allowing complex overlapping Unique search capabilities allowing complex overlapping searches to be performed, eliminating much of the tedious searches to be performed, eliminating much of the tedious work of finding the appropriate data sets work of finding the appropriate data sets

architecture that builds on the latest technologies like web architecture that builds on the latest technologies like web services, but flexible enough that it can accommodate new services, but flexible enough that it can accommodate new technologies, not yet available technologies, not yet available

simplified metadata description that allows new data sets simplified metadata description that allows new data sets and data servers to be easily added and data servers to be easily added

a design that incorporates more than data, that includes a design that incorporates more than data, that includes models and tools as well models and tools as well

a design that a design that builds on existing data centers rather than builds on existing data centers rather than duplicating their functionsduplicating their functions

an approach that was designed for low long-term an approach that was designed for low long-term maintenance costs in the future maintenance costs in the future

a team that represents the major future ITM satellite a team that represents the major future ITM satellite missions and the major ITM-related data centers and missions and the major ITM-related data centers and services services

Page 19: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 19SM23A-06

Halloween 2003 Superstorm as Halloween 2003 Superstorm as an Examplean Example

Increasingly satellites in the ITM community will rely on remote sensing instruments – TIMED, DMSP, NPOESS

Page 20: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 20SM23A-06

What will VITMO mean What will VITMO mean to users?to users? By taking on a more global view of data, VITMO can help the researcher By taking on a more global view of data, VITMO can help the researcher

understand the connectedness between the different data sets. understand the connectedness between the different data sets. Example - storm studies - taking the Example - storm studies - taking the Halloween 2003 superstormHalloween 2003 superstorm as a case study. as a case study. During this event the following things happened: (1) 3 active regions (ARs) During this event the following things happened: (1) 3 active regions (ARs)

produced a total of at least 124 soft X-ray flares and > 60 coronal mass ejections produced a total of at least 124 soft X-ray flares and > 60 coronal mass ejections (CMEs) between Oct. 18 and Nov. 5, 2003. (2) 2 CMEs impacted Earth head-on (CMEs) between Oct. 18 and Nov. 5, 2003. (2) 2 CMEs impacted Earth head-on producing huge geomagnetic storms.producing huge geomagnetic storms.

Today:Today: A researcher may start with: LASCO images and ACE solar wind data over the broad A researcher may start with: LASCO images and ACE solar wind data over the broad

time range in October and November 2003. If the researcher was interested in auroral time range in October and November 2003. If the researcher was interested in auroral impacts from the storm they may have looked for time periods when Kp was between 3 impacts from the storm they may have looked for time periods when Kp was between 3 and 9 and Bz was negative. and 9 and Bz was negative.

They might have looked for Polar and GUVI imagery or auroral conductivity maps They might have looked for Polar and GUVI imagery or auroral conductivity maps during those time periods. during those time periods.

All of this work would have involved finding and navigating numerous web sites that All of this work would have involved finding and navigating numerous web sites that provide data, indices, and other information for the specific time period. provide data, indices, and other information for the specific time period.

Tomorrow – with VITMO:Tomorrow – with VITMO: The researcher will be able to select the needed LASCO images and ACE solar wind The researcher will be able to select the needed LASCO images and ACE solar wind

data. data. They will also be able to select auroral data during the time periods when Kp is They will also be able to select auroral data during the time periods when Kp is

between 3 and 9 and Bz was negative. between 3 and 9 and Bz was negative. The VITMO, knowing that the researcher was interested in auroral data, would have The VITMO, knowing that the researcher was interested in auroral data, would have

automatically presented GUVI and Polar instrument products as appropriate. automatically presented GUVI and Polar instrument products as appropriate. It would then suggest data readers, appropriate for the data format, and tools that the It would then suggest data readers, appropriate for the data format, and tools that the

researcher could use to work with or analyze that data. researcher could use to work with or analyze that data.

Page 21: 05/23/2006 SM23A-06 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/23/2006 21SM23A-06

Resources SupportedResources Supported TIMED, AIM, SNOE, C/NOFS, UARS, TIMED, AIM, SNOE, C/NOFS, UARS,

SuperDARN, DMSPSuperDARN, DMSP CDAWeb, SSCWeb, ModelWeb, CDAWeb, SSCWeb, ModelWeb,

VSTOVSTO Other VxO’sOther VxO’s NGDCNGDC