Hubble Legacy ArchiveLee Quick - TIPS meeting July 19, 2012

• Goals• Data• History• Current Work• Demo

Goals of the HLA Project

– Provide intuitive searching, browsing and download of Hubble data– Create online, enhanced data products for all Hubble data– Develop enhanced user interface capabilities– hla.stsci.edu

Hierarchy of HLA Image Products • Level 1 data are the individual exposures, projected

("drizzled") onto a common frame for each visit.

• Level 2 data are the combined images (exposures with the same filter, same camera, and within the same visit).

• Level 3 images are deep combined images or mosaics creating by combining data taken in different visits to the target.

• Level 4 data are false color images (a combination of two or three colors, depending on what is available).

• Level 5 data are the community-contributed high-level science products (HLSP).

• Level 0 (calibrated, unprocessed) are not available for direct searches but may often by found using the More... link.

HLA - Data Holdings

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA (a brief) History

HLA Early Data Release - August 2007

• Data online for immediate access

• Footprint service to make it easier to browse and download

• Interactive image display • HLA combined images for

~25% of ACS data• Prototype ACS source lists • NICMOS grism extractions

for ~10% of G141 data (produced at ST-ECF)

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 1 – February 2008

• NEW DATA PRODUCTS• HLA combined images for ~90% of ACS data• NICMOS grism extractions for ~80% of available data

(produced at ST-ECF)• DAOphot source lists for ~50% of ACS images• Prototype SExtractor source lists for ~10% of ACS images• A few prototype combined WFPC2 images (produced at CADC• USER INTERFACE ENHANCEMENTS• Footprints for ACS, WFPC2, STIS, NICMOS, FOS, and GHRS• STIS images accessible in HLA interactive display• Non-HLA data products included in inventory and images,

accessible through DADS• Ability to filter inventory table to select observations of

interest• Readout of mouse RA/Dec position in interactive display• Plot cuts of rows or columns through images (or 2-D spectra)

in interactive display• Introductory training movies on how to use the HLA

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 2- September 2008

• NEW DATA PRODUCTS• Combined HLA images for ~95% of WFPC2

data (produced at CADC)• Combined HLA images for ~95% of ACS

observations• Both DAOphot and SExtractor source lists for

70% of ACS observations• USER INTERFACE ENHANCEMENTS• (Shopping) Cart to allow users to download

multiple files at once• Enhanced plotting capabilities• Additional catalog overlays in interactive

display (SDSS, 2MASS, GSC2, FIRST)• User defined search lists• Much faster footprint interface• Additional advanced search options (Proposal

ID, Moving Targets, Filter)

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 3 – May 2009

• NEW DATA PRODUCTS• HLA drizzled combined images for ~30% of NICMOS data• Sample of 1235 ACS extracted grism spectra (provided by ST-

ECF) • Preliminary DAOphot and SExtractor source lists for ~80% of

WFPC2 images• Sample prototypes of ACS multi-visit mosaics• USER INTERFACE ENHANCEMENTS• User search list positions are accepted in a variety of input

formats• Selected datasets (e.g., from clicking on footprints) can be

added to cart• Selection of datasets can be used for sorting and filtering (e.g.,

to show selected data at the top of the table)• Cart can be used to retrieve both DADS and HLA data products• Cart includes estimated download data volume• GALEX catalog is available for overlay in interactive display• Advanced contrast controls in interactive display can be used

to change color balance

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 4 – March 2010

• NEW DATA PRODUCTS• Fully reprocessed NICMOS data• Fully reprocessed WFPC2 data (produced at CADC) with new source lists • Access to WFC3 Early Science release data• New high-level science products• Additional ACS multi-visit mosaics• Updated ACS combined images• USER INTERFACE ENHANCEMENTS• Plotting tool allows for more dynamic user interaction, including zooming and panning• Spectra from FOS and GHRS are displayed in the plotting tool using the "Interactive Display"

links in the inventory and images views.• COS and WFC3 observations are included in the search results and footprints• Proprietary data are included in the search results plus footprints and can be retrieved from

DADS (by privileged users) via the HLA cart• Search by IPPPSOOT exposures and HST visit names (propid_visit)• Inventory table updates• Consolidated Help Center holds HLA resources such as FAQ, Help, Movies etc., and includes

a new HLA documentation search tool.• Source lists are dynamically generated from our databases, which improves consistency

between the various catalog sources and makes it possible to download them from the inventory

• Search results get automatically gzip-compressed when the browser permits it (which is usually the case).

• The interactive display shows the original image pixel value in the coordinate box when the mouse cursor stops moving.

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 5 – March 2011

• USER INTERFACE ENHANCEMENTS• A completely new footprint interface, featuring:

– Zooming in/out capability– Panning– Fully integrated table, so that selections in one

screen remain valid in the others– Selection across instruments possible– Option to search through proprietary and/or non-

proprietary data• Improved download flow from shopping cart

– Single or sequential download– Partial downloads can be restarted

• Sources clickable in interactive display• When filtering RA or Dec columns, the values may

be entered either in decimal degrees or in "sexagesimal" format (hh:mm:ss, dd:mm:ss)

• Multiband source list (ACS and WFPC2) • NEW DATA PRODUCTS• Completed WFPC2 processing (over 300 new visits

with source lists) • New High Level Science Products• ECF-generated ACS and NICMOS extracted GRISM

data hosted at STScI

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

HLA Data Release 6 – January 2012

• NEW DATA PRODUCTS• WFC3 image data: includes ~62% of data from

observations that have a release date up to 11-30-2011

• New High Level Science Products• Additional ACS (updated to release date 2-16-

2011), WPFC2 and NICMOS data• USER INTERFACE ENHANCEMENTS• Interactive Display enhancements.• The inventory and images interfaces now include

a "Select All" button that can be used to select all the datasets in the current list.

• The HLA Simple Image Access Protocol server is now fully compliant with the Virtual Observatory standards. The HLA SIAP server will be added to the VO registry for easy access to HLA data from scripts and other VO tools.

• Searches using lists of positions now show the number of lines in the list.

EDR 8-2007

DR12-2008

DR29-2008

DR35-2009

DR43-2010

DR53-2011

DR61-2012

Current work – WFC3 Source lists

• Based on existing HLA pipelines (ACS, NICMOS) with structural changes• Some algorithmic changes

– New alignment module performs very well on crowded fields– Sky equalization included– More rigorous handling of header information

• Source lists are undergoing quality review, will be released later

Current work – Spectral Data

• Strong desire to better integrate spectral data in the HLA• General purpose spectral pipeline not yet available - focus on High

Level Science Products• First release will include collections of processed spectra (e.g.,

STARCAT)• Interface modifications to handle spectral products • Preview code replaces image previews and cutouts• New Spectral Plotting tool with better functionality to view and

interact with spectra• Difficulties:

• Spectral data products inhomogeneous in properties, combination process

• Naming conventions in development• Large number of data points in some spectra can challenge

some browsers• User preferences not yet established - may need to iterate on

specifics• Expect ~200 products at first release