Upload
beck
View
58
Download
1
Tags:
Embed Size (px)
DESCRIPTION
National Radio Astronomy Observatory. Jared Crossley Loránt Sjouwerman Edward Fomalont Nicole Radziwill. 2008-06-25 SPIE: Observatory Operations: Strategies, Processes, and Systems. NRAO VLA Archive Survey. (NVAS) - PowerPoint PPT Presentation
Citation preview
NRAO VLA Archive SurveyNRAO VLA Archive SurveyNational Radio Astronomy ObservatoryNational Radio Astronomy Observatory
2008-06-25 SPIE: Observatory Operations: Strategies, Processes, and Systems2008-06-25 SPIE: Observatory Operations: Strategies, Processes, and Systems
Jared CrossleyLoránt SjouwermanEdward FomalontNicole Radziwill
Jared CrossleyLoránt SjouwermanEdward FomalontNicole Radziwill
NRAO VLA Archive SurveyNRAO VLA Archive Survey(NVAS)
•Survey of the VLA Archive generated by an automated data pipeline.
•Under development since 2006.
•Data products include: images, calibrated data, diagnostics, logs, …
•Project ongoing, but data is already available on the Web.
http://archive.nrao.edu/
(NVAS)
•Survey of the VLA Archive generated by an automated data pipeline.
•Under development since 2006.
•Data products include: images, calibrated data, diagnostics, logs, …
•Project ongoing, but data is already available on the Web.
http://archive.nrao.edu/
Very Large ArrayVery Large Array
• Maximum baseline: 0.3 to 35 km• Frequencies: 0.07 to 50 GHz• Produces over 150 papers per
year• Outputs “visibility” measurements
• Maximum baseline: 0.3 to 35 km• Frequencies: 0.07 to 50 GHz• Produces over 150 papers per
year• Outputs “visibility” measurements
• Synthesis array of 27 antennas
• Located in New Mexico, USA• Operational for 28+ years
• Synthesis array of 27 antennas
• Located in New Mexico, USA• Operational for 28+ years
“Typical” VLA Data Reduction
• Project scientists are responsible for all parts of the data reduction process.
• Download VLA raw data from the VLA Archive.
• Making images requires:– Careful editing and calibration– Fast Fourier transform processing– Deconvolution methods
• To make data more easily accessible we have created the VLA Archive Survey
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Present State of Survey
Number of Images 72839
Unique Sky Positions
15590
Dates 1991 to 2003
Frequencies 1 to 50 GHz
Observing Modes Continuum
Array Configurations
D,C,B, some A
Sky Coverage
NVAS Pipeline System
ProcessProcess2.
AcquireAcquire1.
ArchiveArchive3.
ValidateValidate4.
• Makes use of preexisting components:– Raw data archive and interface– AIPS synthesis imaging software
package
• Perl and shell scripts connect components to form the pipeline.
• NVAS data is incorporated in the raw data archive. One interface One interface provides access to raw and provides access to raw and processed data.processed data.
Acquire Data
• Perl script automates communication with Archive server by HTTP requests.
• Input start and stop dates for processing1. Script queries the server2. Algorithm selects one or more data files3. Download to local disk4. Data is loaded into AIPS, split by frequency,
and prepared for processing
NVAS Pipeline Stages
AcquireAcquire1.
Data Processing
• AIPS “runfile” (script) performs standard synthesis imaging steps(VLARUN)
• Output data products:– Images– Calibrated data– Diagnostic plots– Log files– Metadata for Web interface
NVAS Pipeline Stages
AcquireAcquire
ProcessProcess
1.
2.
Example image and diagnostic plots:
Crab Nebula at 5 GHz
Archiving
• Shell and Perl scripts • Move data products to
archive server• Generate HTML for survey
Web interface
NVAS Pipeline Stages
AcquireAcquire
ProcessProcess
ArchiveArchive
1.
2.
3.
Validation
• Human-interactive process• A private Web interface is
used to remove low quality data.
• All NVAS data is reviewed by a human.
• Less than 5% of images have been removed during data validation.
NVAS Pipeline Stages
AcquireAcquire
ProcessProcess
ArchiveArchive
ValidateValidate
1.
2.
3.
4.
Pipeline System PerformancePipeline System Performance
• Only data validation requires human interaction.
• All other pipeline stages are limited by computer and network data rates.– Data processing requires the most time;
high-resolution imaging may take many hours
• Survey data is 1.4 TB and growing
Access the survey from http://archive.nrao.edu/
Data Vault:
7019-47
Future Development
1. Finish imaging of VLA continuum dataAt the same time, continue algorithm
upgrades:• Self-calibration• Better data editing• Mosaic imaging• Composite imaging of long-timescale
projects
2. Apply new algorithms to all survey data3. Process spectral line data
Expect to finish these stages end of 2010.
SummarySummary
• Automated survey of the full VLA archive
• Makes VLA data more accessible to all astronomers
• Provides reference images, calibrated data, diagnostics, logs, …
• Ongoing survey, expected completion in 2010
• Currently available at:
http://archive.nrao.edu/
• Automated survey of the full VLA archive
• Makes VLA data more accessible to all astronomers
• Provides reference images, calibrated data, diagnostics, logs, …
• Ongoing survey, expected completion in 2010
• Currently available at:
http://archive.nrao.edu/
NRAO VLA Archive Survey is…NRAO VLA Archive Survey is…