National Center for Supercomputing Applications

Observational Astronomy

NCSA projects radio astronomy: CARMA & SKA optical astronomy: DES & LSST

access: NVO

• The big questions:

– The formation and evolution of the universe from the earliest cosmic times to the present

– The nature of the “dark energy” driving the current acceleration of the universe

• These fundamental questions require a new era of ultra-sensitive, high-resolution imaging survey telescopes at multiple wavelengths, requiring:

• Surveys over large solid angle and redshift cosmic volumes

• Fine synoptic time-sampling and/or high astronomical completeness.

• These survey telescopes will have:

• High receptor count and data acquisition rates

• Software/hardware boundary far closer to receptors than at present

• Efficient, high-throughput survey operations modes

Key questions in current astrophysics

• Processing requirements:– High sensitivity, wide-field

imaging– Demanding time and

frequency non-imaging analysis and transient detection

• Data management and processing implications:– Large O(109) survey

catalogs– High associated data rates

(TBps)– Compute processing rates

(PF)– Petabyte and Exabyte

archives with sophisticated community access mechanisms

The new era of great survey telescopes

LSST

SKA

Data rates in observational Astronomy

(Szalay & Gray)

• Data rates are driven by:• Contemporary

astrophysics questions require surveys of large cosmic volumes

• Moore’s Law advances in detector counts and data output

• Increasingly sophisticated data processing needed.

• Data rates are exponential and require fundamentally new approaches to data management and processing.

20042006

20082010

2012

0.00

100.00

200.00

300.00

400.00

500.00

600.00

700.00

800.00

900.00T

era

by

tes

Year

VLBA

ALMA

CARMA

VLA

Optical data rates

Telescope area

CCD pixel count

Radio data rates

• Deep processing and data product production at Archive Center at NCSA• Archive Capacities: 12 PB – 83 PB over 10 year lifetime• Compute Requirements: 80 TF – 270 TF over 10 year lifetime

• A new optical survey telescope to be located in Chile• Map entire visible sky every 3-4 days• Rapid remapping for detecting moving

objects in solar system and more distant transient phenomena

• Increasing sensitivity with each remapping will result in deep images of distant universe

• A tool for constraining dark energy• A Data Challenge across two continents

• 3 Gpixel camera will produce 13 TB raw data per night

• Real-time processing at Base Site to detect and report new transients

Just a few LSST challenges

• To meet demands, part of data center must be optimized for high I/O bandwidth• Drives up “spindle count” (the number of drives) over their capacity• Current models suggest that meeting performance requires 10x excess DB

storage• Investigating new hardware and database technologies to address issues

• Allowing Community to leverage LSST processing cyberinfrastructure for custom processing and analysis• All sky analysis not possible on desktop• Scientists need to insert new processing into existing pipelines• VM-based clusters and cloud technologies can provide better access for user

code

• Sharing a massive database with the community• Single DB releases will grow to over 10

TB with trillions of rows• Community access will be a mix of

simple and complex queries• Simple queries need to return

quickly (50 simultaneous queries in < 2 sec)

• A few complex queries may require full table scans

New-era radio interferometer in the 2015-2025 decade

One square kilometer of collecting area

Science:

1) cradle of life

2) probing the dark ages

3) cosmic magnetism

4) strong field tests of gravity

5) galaxy evolution, cosmology, & dark energy

A peta- or exa-scale processing and data management challenge for this decade.

National Center for Supercomputing Applications

Observational astronomy faces a coming tsunami of data promising enormous scientific advances but threatening to overwhelm computational and data management resources offering great ⎯opportunities for NCSA