ESDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham

eSDO Algorithms, Visualization and Data Access

Elizabeth Auden

21 September 2006

AHM 2006

Nottingham

21 September 2006 UK e-Science AHM 2006 Elizabeth Auden

What is eSDO?eSDO is an e-science project funded by PPARC to make SDO data and

algorithms available to the UK solar community using the virtual observatory.

Algorithms

• Code: feature recognition, local helioseismology, global helioseismology

• Grid integration: AstroGrid and JSOC

Date centres

• AstroGrid interface with US JSOC data centre

• UK data centre

Visualization tools

• Catalogues

• SDO Streaming tool

• On-the-fly image galleries and movies


eSDO Schedule

RESEARCH - Phase A: 1 October 2004 – 30 September 2005

IMPLEMENTATION - Phase B: 1 October 2005 – 30 September 2007

Overview of Phase A activities: Phase A Summary Report• available on eSDO project wiki: http://

www.mssl.ucl.ac.uk/twiki/bin/pdf/SDO/PhaseASummaryReport

Detailed report including 6 deliverable documents: Phase A Report• available on eSDO project wiki:

http://www.mssl.ucl.ac.uk/twiki/bin/pdf/SDO/PhaseReport


eSDO Project Team

• MSSL– Elizabeth Auden, project manager– Len Culhane, PI– Lidia van Driel-Gesztelyi– Mike Smith

• RAL– Richard Harrison– Andrzej Fludra– Vito Graffagnino

• Sheffield– Mike Thompson– Sergei Zharkov

• Birmingham– Yvonne Elsworth– Bill Chaplin– Thierry Toutain

US Links:

JSOC, Joint Operations Centre

Stanford University

Lockheed Martin


AlgorithmsCode

Feature Recognition / Event Analysis

• Coronal loop detection, MSSL

• Non-linear magnetic field extrapolation, MSSL

• Helicity computation, MSSL

• Small event detection, RAL

• CME dimming region recognition, RAL

Local Helioseismology

• Perturbation map generation, Sheffield

• Subsurface flow analysis, Sheffield

• Local helioseismology inversions, Sheffield

Global Helioseismology

• Mode parameters analysis, Birmingham

• Mode asymmetry analysis, Birmingham


AlgorithmsGrid Integration

JSOC• SDO pipeline at Stanford University / Lockheed Martin• eSDO algorithms deployed as pipeline modules (automated or user-run)• Data will be obtained locally from JSOC archive

AstroGrid CEA• UK hosted algorithms hosted as web services• Accessible through AstroGrid workflows• Data transferred from MySpace, remote data centre or user’s local storage

SolarSoft• Algorithm C modules wrapped in IDL• Distribution through MSSL SolarSoft gateway• Data transferred to algorithm from user’s local storage


AlgorithmsAstroGrid Deployment

9 algorithms have been deployed as CEA applications


Data CentresUS Data Centre

Primary US Data Store

• Files stored in SUMS (Storage Unit Management System) at Stanford University with metadata in distributed DRMS (Data Resource Management System)

• DSA / DRMS integration for grid data access investigations

• VSO (Virtual Solar Observatory) primary US access

AstroGrid / VSO Interface

• VSO queries in AstroGrid workflows through STAP (Simple Time Access Protocol) web service

US / UK Network Latency Tests

• Test expediency of file transfer mechanisms

• Test between Stanford, MSSL, UCL, RAL


Data CentresPossible UK Data Centre

Storage

• 30 TB disc cache

• 60 day rolling cache HMI/AIA

• True cache for HMI / AIA

• DRMS / SUMS

Transfer

• Export formats from JSOC

• Data search – DSA

• Data transfer - CEA

• Test installation MSSL

Investigation of DRMS / SUMS


Data CentresPossible European Data Centre

• HELAS: European Helio and AstroSeismology Network

• EU Framework 6 support for European helioseismology data and

algorithms

• Possible full mirror of SDO data in Germany

• Discussion of possible collaboration between eSDO and HELAS in late

September 2006 at HELAS workshop in Nice

• Use UK-held DSA / DRMS interface for AstroGrid access to SDO data in

Germany instead of UK?


Note: AstroGrid components (JES, DSA, MySpace, Workbenchl) may be located anywhere.

Data Centres eSDO / Solar-B Prototype: system perspective

UKSSDCADS file system


Data Centres User access through AstroGrid

HelioScope: solar data results from Simple Time Access Protocol services.

AstroGrid workbench: STAP service access through HelioScope, DSA access through the Task Launcher.

Task Launcher: locate DataSet Access components, submit ADQL queries

STAP service will point to URLs in UK, US or Germany.

DSA will integrated with DRMS (Data Resource Management System) in UK or Germany.


Visualization ToolsCatalogues

Small Event / CME Dimming catalogue

• Generated from continuously running of small event detection / CME dimming region recognition algorithms

• Tables of event / CME statistical information

Helioseismology Mode Parameters catalogue

• GONG month of data generated from mode parameters analysis algorithm

• Generated in US

Thumbnail catalogue

• Basic metadata for generated thumbnail images

Catalogues can be accessed through ADQL query to DSA.


Visualization ToolsSDO streaming tool

User Facilities

• Pan in space • Zoom in space • Pan in time

• Zoom in time

Data Products

• LoS Magnetogram

• HMI continuum map

• AIA images

Future Work

• FFT / wavelet analysis for coronal wave investigations


Visualization ToolsOn-the-fly Image Galleries and MoviesThumbnail Extraction• Extract thumbnails in US

from HMI and AIA science products

Image Gallery Generation• User specifies start time,

stop time, cadence, data product

• Image gallery generated on the fly in web browser

Movie Generation• User specifies start time,

stop time, cadence, data product

• MPEG movie generated on the fly

Find data with AstroGrid HelioScope.

Use PLASTIC to load data into visualization tools.


Future Work

Phase B implementation development now in progress:• Summer / Autumn 2006:

– Unit and science testing for algorithms– Add STAP service client to visualization tools– Add movie functionality to streaming tool– Investigate DRMS integration with DSA

• 2007: – completed algorithms– algorithm integration with AstroGrid, JSOC and SolarSoft– Coronal wave analysis in SDO streaming tool– Work with HELAS to explore European SDO data centre– Catalogues

Documents

ESDO Algorithms, Visualization and Data Access Elizabeth Auden 21 September 2006 AHM 2006 Nottingham