05/22/2006 VxO Workshop 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,

05/22/2006 VxO Workshop 1

A Virtual Observatory A Virtual Observatory for the Ionosphere-for the Ionosphere-

Mesosphere-Mesosphere-Thermosphere Thermosphere

CommunityCommunityD. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. Yee, and E. D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund, J.-H. Yee, and E. TalaatTalaat

(Johns Hopkins University, Applied Physics Laboratory 11100 Johns (Johns Hopkins University, Applied Physics Laboratory 11100 Johns Hopkins Rd. Laurel MD, 20723: 240-228-4172) Hopkins Rd. Laurel MD, 20723: 240-228-4172)

J. RussellJ. Russell(Hampton University)(Hampton University)

R. HeelisR. Heelis(Univ. Texas at Dallas)(Univ. Texas at Dallas)

J. KozyraJ. Kozyra(Univ. of Michigan)(Univ. of Michigan)

D. Bilitza, R. McGuire, and R. CandeyD. Bilitza, R. McGuire, and R. Candey(NASA Goddard Spaceflight Center)(NASA Goddard Spaceflight Center)

P. FoxP. Fox(HAO/NCAR)(HAO/NCAR)

05/22/2006 2VxO Workshop

ITM is Diverse in Observables

An ITM VO must facilitate correlative study between multiple parameters from multiple sources and cross-discipline studies.


VITMO ConstituencyVITMO Constituency

VITMO is focused on the VITMO is focused on the “interdisciplinary” ITM user rather “interdisciplinary” ITM user rather than the “instrument team” user.than the “instrument team” user.

Focus is on facilitating multi-Focus is on facilitating multi-instrument and multi-mission instrument and multi-mission investigations.investigations.

Provide tools and capabilities to tie Provide tools and capabilities to tie together disparate measurements of together disparate measurements of the same or related phenomena.the same or related phenomena.


People, Institutions, Data People, Institutions, Data SetsSets D. Morrison – Principal Investigator D. Morrison – Principal Investigator

M. Weiss, R. Daley, L. Immer, - SRAS PrincipalsM. Weiss, R. Daley, L. Immer, - SRAS Principals Science Advisory PanelScience Advisory Panel

J.-H. Yee and E. TalaatJ.-H. Yee and E. Talaat - APL - APL J. Russell J. Russell - Hampton University- Hampton University R. Heelis R. Heelis - Univ. Texas at Dallas- Univ. Texas at Dallas J. Kozyra - J. Kozyra - Univ. of MichiganUniv. of Michigan D. Bilitza, R. McGuire, and R. Candey D. Bilitza, R. McGuire, and R. Candey - NASA Goddard Spaceflight - NASA Goddard Spaceflight

Center / RaytheonCenter / Raytheon P. Fox P. Fox - HAO/NCAR- HAO/NCAR

Data / System ProvidersData / System Providers D. MorrisonD. Morrison – APL – GUVI/TIMED, SSUSI/DMSP, AURORA/NPOESS – APL – GUVI/TIMED, SSUSI/DMSP, AURORA/NPOESS S. Nylund S. Nylund – TIMED MDC Integration – APL– TIMED MDC Integration – APL J. Russell J. Russell - Hampton University – SABER, AIM, UARS- Hampton University – SABER, AIM, UARS R. Heelis R. Heelis - Univ. Texas at Dallas – C/NOFS, DMSP SSIES- Univ. Texas at Dallas – C/NOFS, DMSP SSIES D. Bilitza, R. McGuire, and R. Candey D. Bilitza, R. McGuire, and R. Candey - NASA Goddard Spaceflight - NASA Goddard Spaceflight

Center / Raytheon – SPDF, CDAWeb, ModelWebCenter / Raytheon – SPDF, CDAWeb, ModelWeb P. Fox P. Fox - HAO/NCAR – CEDAR, VSTO- HAO/NCAR – CEDAR, VSTO Scott Bailey Scott Bailey – Univ. Alaska – SNOE– Univ. Alaska – SNOE Jan Merka - Jan Merka - NASA Goddard Spaceflight Center - VMONASA Goddard Spaceflight Center - VMO


The Ability to Find the The Ability to Find the Appropriate Data Will Be a Key Appropriate Data Will Be a Key

Requirement of a VITMORequirement of a VITMO

Increasingly satellites in the ITM community will rely on remote sensing instruments – TIMED, DMSP, NPOESS


How the VITMO Will Be How the VITMO Will Be DifferentDifferent

For a VxO to be truly useful it must go beyond traditional mission data centers. For a VxO to be truly useful it must go beyond traditional mission data centers. It should be able to It should be able to interrogate multiple data centers, finding products that interrogate multiple data centers, finding products that

overlap in time and/or location. overlap in time and/or location. It must allow a It must allow a higher level of searchhigher level of search to be performed than previously available. to be performed than previously available.

An example of such a complex search would be something like the following: An example of such a complex search would be something like the following: “what near Earth orbiters are observing the auroral region when Bz is negative “what near Earth orbiters are observing the auroral region when Bz is negative and the solar wind speed is greater than 400 km / sec.?” and the solar wind speed is greater than 400 km / sec.?” Today, this would Today, this would require a manual process where the researcher would first find plots of the require a manual process where the researcher would first find plots of the Earth’s magnetic field component (Bz) and plots of solar wind speed from Earth’s magnetic field component (Bz) and plots of solar wind speed from NASA’s CDAWeb (http://cdaweb.nasa.gsfc.gov). After they manually looked for NASA’s CDAWeb (http://cdaweb.nasa.gsfc.gov). After they manually looked for overlap conditions they would utilize these time periods to search known Earth overlap conditions they would utilize these time periods to search known Earth orbiters to find out if they were observing during those time periods. They would orbiters to find out if they were observing during those time periods. They would then have to manually review summary products to determine if those orbiters then have to manually review summary products to determine if those orbiters were viewing the auroral region during those intervals. were viewing the auroral region during those intervals. VITMO will handle VITMO will handle these types of queries automatically.these types of queries automatically.

VITMO will also be able to organize toolsVITMO will also be able to organize tools, whether plotting, subsetting, or , whether plotting, subsetting, or analysis tools by the type of data they are to be applied to as well as the types of analysis tools by the type of data they are to be applied to as well as the types of operations that are to be performed. If the user requested time series data then operations that are to be performed. If the user requested time series data then tools appropriate for operating on the time series should be presented to them. tools appropriate for operating on the time series should be presented to them. If they chose images then tools appropriate for images should be made If they chose images then tools appropriate for images should be made available. If the data were in CDF format, then FITS format data tools need not available. If the data were in CDF format, then FITS format data tools need not be presented. Additionally, VITMO will understand that model output can be be presented. Additionally, VITMO will understand that model output can be treated as high level data products and that models should be available, just as treated as high level data products and that models should be available, just as tools are, to the end user.tools are, to the end user.


VITMO is Java based.

The Knowledge Base is currently an Oracle database.


VITMO Architecture VITMO Architecture OverviewOverview

high-levelhigh-level conceptual model of the scientific disciplines conceptual model of the scientific disciplines and domain data in the region from the Sun to the Earthand domain data in the region from the Sun to the Earth

Regions – Solar, Near-Earth – F-region, E-regionRegions – Solar, Near-Earth – F-region, E-region Features – exist for long periods of timeFeatures – exist for long periods of time Events – Occur at particular points in timeEvents – Occur at particular points in time

description of resource metadata in the Knowledge description of resource metadata in the Knowledge Base supporting any accessible scientific resourceBase supporting any accessible scientific resource

descriptive metadatadescriptive metadata includes program, satellite and instrument includes program, satellite and instrument information, as well as data details like observed phenomena, information, as well as data details like observed phenomena, units of measure, time and spatial coverage, processing level units of measure, time and spatial coverage, processing level

structural metadatastructural metadata defines the details for data access and defines the details for data access and retrieval if selected by the use retrieval if selected by the use

expand the notion of a data resource in our metadata definition expand the notion of a data resource in our metadata definition to also include databases and computer programs (such as to also include databases and computer programs (such as format conversion tools or data models) that generate requested format conversion tools or data models) that generate requested data on demand. Thus, file access, database access, and data on demand. Thus, file access, database access, and program invocation details are all included in the resource program invocation details are all included in the resource metadata model. metadata model.


Coupling in VITMO Coupling in VITMO ArchitectureArchitecture

a a loose couplingloose coupling between the conceptual model between the conceptual model and the resource metadata.and the resource metadata. A query in the conceptual model identifies the desired A query in the conceptual model identifies the desired

parameters, which are then mapped to the related data parameters, which are then mapped to the related data resources. Data discovery is simplified by performing resources. Data discovery is simplified by performing requests at the conceptual level, permitting users requests at the conceptual level, permitting users without knowledge of specific mission, or instrument to without knowledge of specific mission, or instrument to readily find data of interest. readily find data of interest.

conceptual model can be readily expanded if new classes conceptual model can be readily expanded if new classes of scientific concepts or new relationships between of scientific concepts or new relationships between scientific concept classes would improve the model's scientific concept classes would improve the model's representation of the Sun to Earth domain. representation of the Sun to Earth domain.

a data provider can add resources (of any format, access a data provider can add resources (of any format, access method, or processing level) simply by providing method, or processing level) simply by providing information that describes the data and its access details.information that describes the data and its access details.

model eliminates the need for rigid metadata standards model eliminates the need for rigid metadata standards or unique, resource-specific integration software or unique, resource-specific integration software


Scientific Conceptual Model Scientific Conceptual Model in the VITMO Knowledge in the VITMO Knowledge

Base Base


Descriptive metadata Descriptive metadata portion of the VITMO portion of the VITMO

Knowledge Base Knowledge Base


Structural metadata Structural metadata portion of the VITMO portion of the VITMO

Knowledge Base Knowledge Base


VITMO Based on VITMO Based on ServicesServices


Enhanced Search Enhanced Search ServicesServices

Today’s researcher needs to find datasets from Today’s researcher needs to find datasets from different missions that overlap in time and space. different missions that overlap in time and space.

Many of today’s satellite missions include remote Many of today’s satellite missions include remote sensing instruments. sensing instruments. Finding cross comparisons Finding cross comparisons between instruments is more difficult because we between instruments is more difficult because we must be concerned not just with spacecraft must be concerned not just with spacecraft location, but also with multiple look-points for location, but also with multiple look-points for the instruments.the instruments.

The prime NASA mission for study of the MLT The prime NASA mission for study of the MLT region is the TIMED satellite. All of the region is the TIMED satellite. All of the instruments on this satellite are remote sensing instruments on this satellite are remote sensing instruments. Each remote sensing instrument instruments. Each remote sensing instrument has a different look geometry, some looking as has a different look geometry, some looking as far as 2000 km away from the satellite. far as 2000 km away from the satellite.


Enhanced Search Services Enhanced Search Services ContinuedContinued

AIM will have a fly’s eye imager.AIM will have a fly’s eye imager. The fields-of-view (regard) of these The fields-of-view (regard) of these

instruments define the extent of the instruments define the extent of the data products.data products.

This location information is This location information is notnot contained in the existing metadata in contained in the existing metadata in today’s catalogs.today’s catalogs.


Virtual Metadata GeneratorsVirtual Metadata Generators VITMO will employ special coincidence calculators that VITMO will employ special coincidence calculators that

know the viewing geometries of the various know the viewing geometries of the various instruments.instruments.

The TIMED coincidence calculator is an example of a The TIMED coincidence calculator is an example of a tool which when converted into a web service can be tool which when converted into a web service can be used to augment existing data querying.used to augment existing data querying.

The TIMED coincidence calculator has recently been The TIMED coincidence calculator has recently been imbedded into the TIMED query system.imbedded into the TIMED query system.

These calculators will operate as web services inside These calculators will operate as web services inside the VITMO query system (or by outside query systems).the VITMO query system (or by outside query systems).

These calculators generate metadata for the These calculators generate metadata for the coincidence of interest which is used by the catalog coincidence of interest which is used by the catalog system to answer your query.system to answer your query.

The metadata generated by this service is discardedThe metadata generated by this service is discarded after it is after it is used – hence “virtual metadata”.used – hence “virtual metadata”.


Enhanced Search ResultsEnhanced Search Results Addition of virtual Addition of virtual

metadata metadata capabilities allows capabilities allows pre-selection of pre-selection of data for analysis.data for analysis.

VITMO search will VITMO search will deliver results from deliver results from searches such as searches such as “What GUVI data “What GUVI data that contains 23-01 that contains 23-01 MLT during active MLT during active geomagnetic geomagnetic conditions?”conditions?”


Integrating with VITMOIntegrating with VITMO

VITMO can integrate with data providers VITMO can integrate with data providers directly – understanding provider data and directly – understanding provider data and file structure.file structure.

VITMO can integrate with high level VITMO can integrate with high level providers (other VxOs) through services (i.e. providers (other VxOs) through services (i.e. the way we will integrate CDAWeb).the way we will integrate CDAWeb).

VITMO can integrate with high level query VITMO can integrate with high level query services and data providers (and older style services and data providers (and older style providers) through Front Door Integration – providers) through Front Door Integration – typically how the VITMO could interface to typically how the VITMO could interface to NGDC and SPIDR.NGDC and SPIDR.


Front Door IntegrationFront Door Integration We have prototyped and developed a new

approach that can allow many of the capabilities of an existing data center to be easily incorporated into the VITMO by interfacing to their system by a specialized web service.

Consider, for example, a data center that provides data query and retrieval services. A significant amount of time and effort will certainly have been spent developing the user interface to such a system.

We have developed services that encapsulate that entire user interface, allowing user input, enabling a combined system to take full advantage of the rich capabilities that already exist.

We demonstrated the feasibility of this approach by incorporating the Space Physics Data Markup Language search engine (SPDML) into the SRAS system (VITMO predecessor system).

Advantages Allows reuse of user interfaces Concept mapping is high level (user level) rather

than at the database or metadata level – much simpler.

Allows integration of extensive search services.

The following is an example of mapping VITMO parameters to another system, i.e. SPDML.


Prototype DisplayPrototype DisplayThe left hand side of the web page allows the user to search or browse for resources (be they data, models, tools, etc) by the particular region of space, time, satellite coincidences or other criteria.

After resources are searched for the results are displayed and possible display plots may be shown.

If the user selects the “Related Tools and Models” button on the top of the page a display of appropriate tools and models for the data or resources chosen will be presented.

This list of tools and models is generated dynamically based on the metadata in the VITMO catalog describing the data, tools, and models available through the VITMO.


ScheduleSchedule Year 1Year 1 Perform a preliminary task analysis for ITM data query. This will help define the Perform a preliminary task analysis for ITM data query. This will help define the

number of the web services that need to be created and establish the parameters that number of the web services that need to be created and establish the parameters that need to be passed. We will define the lists of rules that will be used to determine what need to be passed. We will define the lists of rules that will be used to determine what web services are called and the order that they are called to satisfy the data query.web services are called and the order that they are called to satisfy the data query.

Work on defining the WSDL that will be used to pass messages between the different Work on defining the WSDL that will be used to pass messages between the different web services.web services.

Define integration approach for web service capabilities provided by CDAWeb and Define integration approach for web service capabilities provided by CDAWeb and SSCWeb. Define any changes to existing web services they provide. Define additional SSCWeb. Define any changes to existing web services they provide. Define additional web services desired.web services desired.

Define model and tool metadata. Evaluate MODELWeb models as services.Define model and tool metadata. Evaluate MODELWeb models as services. Develop general-purpose mechanisms to incorporate web service virtual metadata Develop general-purpose mechanisms to incorporate web service virtual metadata

generators into the data discovery request process.generators into the data discovery request process. Survey all data products, tools, and models available for inclusion in first build of Survey all data products, tools, and models available for inclusion in first build of

VITMO.VITMO. Present results at an AGU meeting and/or an NSF CEDAR meeting.Present results at an AGU meeting and/or an NSF CEDAR meeting. Year 2Year 2 Continue implementing the rule-based middleware used to determine the number and Continue implementing the rule-based middleware used to determine the number and

order of query operations.order of query operations. Develop web interface based on feedback from .previous workshop.Develop web interface based on feedback from .previous workshop. Implement metadata describing tools and models into VITMO.Implement metadata describing tools and models into VITMO. Implement the TIMED solar geophysical database as a web service.Implement the TIMED solar geophysical database as a web service. Provide a workshop to solicit community input and demonstrate proposed approaches Provide a workshop to solicit community input and demonstrate proposed approaches

and solutions to integration problems and search capabilities.and solutions to integration problems and search capabilities. Implement MODELWeb models as web services.Implement MODELWeb models as web services. Present results at an AGU meeting and/or an NSF CEDAR meeting.Present results at an AGU meeting and/or an NSF CEDAR meeting.


Schedule – Year 3Schedule – Year 3 The third year will focus on tying all of the pieces together into a complete The third year will focus on tying all of the pieces together into a complete

demonstration system.demonstration system. We will have TIMED, UARS, SNOE, C/NOFS, AIM, Polar, IMAGE and ACE We will have TIMED, UARS, SNOE, C/NOFS, AIM, Polar, IMAGE and ACE

data sets (available at SPDF [data sets (available at SPDF [BilitzaBilitza, 2003]), DMSP from F15, F16, and F17 , 2003]), DMSP from F15, F16, and F17 on. In addition, we will have a selection of ground sites through VSTO, as on. In addition, we will have a selection of ground sites through VSTO, as well as the SuperDARN radar network, added in.well as the SuperDARN radar network, added in.

We will tie our web services architecture into our existing VITMO system We will tie our web services architecture into our existing VITMO system which will already have an interface for making the type of queries we which will already have an interface for making the type of queries we require.require.

Integrate with other VxOs (VHO, VSO, and VMO) to allow searches between Integrate with other VxOs (VHO, VSO, and VMO) to allow searches between systems and sharing of tools, models, etc. systems and sharing of tools, models, etc.

Provide a final workshop to solicit community input and demonstrate Provide a final workshop to solicit community input and demonstrate proposed approaches and solutions to integration problems and search proposed approaches and solutions to integration problems and search capabilities. Propose remaining problems in the development of a complete capabilities. Propose remaining problems in the development of a complete operational system (security, performance, etc). operational system (security, performance, etc).

Present results from workshop at an AGU meeting and/or NSF CEDAR Present results from workshop at an AGU meeting and/or NSF CEDAR meeting.meeting.

Develop a guidebook describing how future missions can become part of the Develop a guidebook describing how future missions can become part of the VITMO.VITMO.

Publish software developed and web services protocols used. This will allow Publish software developed and web services protocols used. This will allow outside organizations such as NSSDC, NGDC, and future virtual outside organizations such as NSSDC, NGDC, and future virtual observatories to tie into our system to enhance their existing capabilities.observatories to tie into our system to enhance their existing capabilities.


Community InputCommunity Input

Routine input from Science Advisory Routine input from Science Advisory PanelPanel

Annual workshops demonstrating Annual workshops demonstrating functionality, refining requirements, etc.functionality, refining requirements, etc.

AGU – demonstrationsAGU – demonstrations First year input (including Science First year input (including Science

Advisory Panel) may be postponed to Advisory Panel) may be postponed to second year to provide more funding for second year to provide more funding for software development (strong review software development (strong review panel recommendation).panel recommendation).

Documents

05/22/2006 VxO Workshop 1 A Virtual Observatory for the Ionosphere-Mesosphere- Thermosphere Community D. Morrison, M. Weiss, R. Daley, L. Immer, S. Nylund,