TDP activities in calibration and processing

Athol Kemball for TDP Calibration & Processing Group (CPG)

Department of Astronomy, University of Illinois at Urbana-Champaignakemball@illinois.edu

Current CPG membership

• Athol Kemball (Illinois) (Chair)• Sanjay Bhatnagar (NRAO)• Geoff Bower (UCB)• Shami Chatterjee (Cornell)• Jim Cordes (Cornell; TDP PI)• Shep Doeleman (Haystack/MIT)• Joe Lazio (NRL)• Colin Lonsdale (Haystack/MIT)• Lynn Matthews (Haystack/MIT)• Steve Myers (NRAO)• Jeroen Stil (Calgary)• Greg Taylor (UNM)• David Whysong (UCB)• Mark Yashar (UIUC)

Calgary.... .

. .Cornell

NRLUIUC MIT

NRAOUCB UNM

Primary CPG goal

Contribute to a construction-ready Phase 1 SKA design and proposal.

Provide a quantitative cost and feasibility model for calibration and processing elements of the US TDP LNSD SKA design.

Current CPG activities • Continued progress on Project

Execution Plan (PEP) tasks.• Key recent and continuing focus is

coordination with needs of DVA-1 project within TDP:– Processing Verification Program (PVP).

• Liaison with PrepSKA project and pathfinders.

• Graduate students research involvement and engagement.

• Outreach at meetings and in community.

(L. Matthews)

Feasibility: tracing science requirements

SKA science requires unprecedented sensitivity and

dynamic range. Need too meet thermal noise

limit:

(Lazio et al. (2009); Chakraborty & Kemball (2009))

We are considering these requirements on calibration and processing per science use

case.

( 1)th

ant ant

SEFD

N N t

Processing verification strategy• Trace calibration and processing

requirements to the SKA science cases.

• Assess practical community experience in high-dynamic range imaging with current interferometers.

• Parallel/simultaneous strategies:– Analytic or semi-analytic approaches.– Numerical simulations.– Pathfinder tests.

(3C286; ATAWhysong & Bower)

PVP: projecting DVA-1 performance• Antenna and feed

design parameters:– Mount type– Reflector:

• Size, shape, and manufacturing method

– Optics– Feed and illumination– Polarization purity– Net bandwidth– Etc.

• Overall system performance verification:– Cost per unit achieved

sensitivity as a function of:• Angular distance from

center of main lobe: ρ• Polarization: {I,Q,U,V}• Frequency: ω

– Feasibility:• Limiting sensitivity in

{I,Q,U,V} (ρ ,ω) due to uncorrected systematic errors.

Feasibility: practical community experience in high-DR imaging

• Community survey of current DR limitations (EVLA, WSRT, GMRT, ATCA) (Chakraborty & Kemball 2009):– Hardware is destiny

• Sky-mount vs rotating antennas• Closure errors

– Direction-dependent errors:• Pointing• Beam pattern errors

– RFI– Computational and software limits

• Longitudinal study on 3C273, 1974-2010 (Chakraborty & Kemball, 2010).– DR is tracking thermal sensitivity

evolution, but at cost of calibration complexity.

(Chakraborty & Kemball 2010)

Mean DR evolution over time

Numerical simulations• Current focus of

numerical simulations:– Calibrating semi-analytic

dynamic-range scaling laws derived earlier.

– Placing constraints on radiation patterns (e.g. sidelobe levels and stability, pointing) as part of DVA-1 support.

1.4 GHz leakages for antennas (numbered) in real,imaginary space as a function of frequency.

Typical values up to 10%.

Smooth changes with frequency.  Rate of ~3%/10 MHz at 1.4 GHz.

Spiral and loop patterns seen in real-imag space.

Period of loops similar to that of log-periodic feed (e.g., 3/4 of a turn at 1.4 GHz).

Leakages are contiguous between adjacent bands.  This suggests leakages originate in the feed.

Leakages for two adjacent correlator tunings.  The leakages change smoothly between bands.

Feasibility: wide-band polarization calibrationPathfinder ATA studies (Bower et al)

Leakages are measured in offset pointings from calibrator.

Similar leakages throughout the primary beam.

Random contribution increases off-axis and toward higher frequencies.

Random component of order 3% within half the HP point at 1.4 GHz.

Leakages for ant 11x at center and half power points.  Color shows frequency dependence.  Axes show leakage scale.

Same leakages, but showing diff with center.

Feasibility: direction-dependent polarization calibration

Pathfinder ATA studies (Bower et al)

Feasibility: primary beam radiation pattern stability

(ATA: Harp et al. 2010)

Correlator FOV Shaping: Smart (t, ν) averaging

• Multiply the sky by a weighting (window) function W convolve the u-v plane by Fourier transform of the window function, effectively tailoring the FOV

• Can be used to mitigate sidelobes (shape FOV) in high-resolution imaging

W Jinc/top hat function

• Applying single weighting function in (u, v) plane will impose same FOV on all baselines

• Do this for each visibility, during correlation, pre-calibration

Using both real (MERLIN) and simulated (MAPS) data, the MIT Haystackgroup has been exploring:

- Attenuation of distant sources due to realistic primary beams (from Cortes) and array synthesis beams (based on Bolton & Millenaar)

- Expected noise contributions from distant sources before and after convolution

- How well various choices of the correlator FOV function reduce the noise contributions from sources outside the field of interest; impact on achievable dynamic range

- Requirements for implementation in future correlators

No convolution Gaussian Top Hat Top Hat Top Hat

3C343

3C343.1

from Lonsdale et al., in prep.

Costing: emerging trends in exaflop power consumption

• Exaflop architectures will have extreme parallelism and heterogeneous processing elements.– Energy costs will be a key architectural

driver.• Future energy costs lie predominantly with:

– Memory– Data transfer/movement costs within

system– Both costs will likely dominate processor

power.– Many technical innovations under

consideration• Proprietary information and variable

technological risk• This introduces uncertainty into

energy extrapolation.

Yashar & Kemball (2009)

Current power projections• Considering system power only:

– Exascale general-purpose system built out using today’s technology: 150 MW++

– Exascale general-purpose system assuming timely but evolutionary energy efficiency advances: 60-100 MW+

• Very aggressive technological evolution assumed: 30MW+

– Exascale system with limited computational modes (i.e. tuned for modest number of SKA operation types):

• Low tens of MW (say 30 MW). • A hardware-software co-design (e.g. a

vendor partnership) – or an “off-the-shelf” configurable supercomputer.

– A single-mode exaflop system:• < 20 MW, perhaps << 20 MW.

(Gara 2008)

(Donofrio et al. 2009)

Power utilization effectiveness• Cooling (chilling) and circulation sub-systems (+20-35%,

but extensive room for improvement).– Optimized flow of coolant or air (at the board and chip

level).– Extensive environmental monitoring of system and data

center.– Passive environmental cooling.– Minimizing number of voltage down-conversions from HT

to chip board voltage.• E.g. variable voltage computer boards.

– Processor and board power optimization (slowdown) modes.– A strong industry driver:

• You can’t sell an exaflop system that needs 150 MW of power!• Regulatory interventions loom for large data centers.

• Example: National Petascale Compute Facility:– LEED Gold certification for green buildings– Power utilization effectiveness (PUE) ~ 1.1 to 1.2– Opening June 2010

(Cray Ecophlex system document)

Liaison with PrepSKA and international SKA project

• Participation in key PrepSKA and SKA meetings:– PrepSKA WP2 PY#2 meeting

(Manchester UK, Nov 09).– SKA 2010.– PrepSKA WP 2.6 group participation.

• TDP is lead institution on:– Calibration and Imaging Techniques for

WBSPF (PrepSKA WBS 2.6.3.1)– Exascale Computing & Hardware

(PrepSKA WBS 2.6.7)

TDP and PrepSKA have close synergies in goals during

design and development phase

Close engagement with PrepSKA in refining and

coordinating cal. & processing work in WP2; TDP funding facilitates US engagement.