Upload
penelope-mason
View
228
Download
0
Tags:
Embed Size (px)
Citation preview
OmegaCAM: The 16k x 16k Survey Camera for the VST
OmegaCAM: The 16k x 16k Survey Camera for the VST
Calibration, Data Analysis Strategy and Software
Calibration, Data Analysis Strategy and Software
Erik R. DeulKonrad KuijkenEdwin A. Valentijn
Erik R. DeulKonrad KuijkenEdwin A. Valentijn
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
2
People involvedPeople involved
• The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K.
KuijkenSterrewacht Leiden: R. Rengelink, E.R. Deul
• Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp,
H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. SagliaUniversitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif
• ItalyAstronomical Observatory of Capodimonte - Napoli: E. CasconeOsservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held,
H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi• ESO
D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi
• The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K.
KuijkenSterrewacht Leiden: R. Rengelink, E.R. Deul
• Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp,
H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. SagliaUniversitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif
• ItalyAstronomical Observatory of Capodimonte - Napoli: E. CasconeOsservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held,
H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi• ESO
D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
3
DetectorsDetectors
• Science array 1 x 1 degree, 32 CCDs– 15 m pixels – 0.21 arcsec/pixel– Marconi (former EEV) 2k x 4k– 16k x 16k pixels
• Auxiliary CCD’s – 4 CCDs– For guiding– Image analysis
• Science array 1 x 1 degree, 32 CCDs– 15 m pixels – 0.21 arcsec/pixel– Marconi (former EEV) 2k x 4k– 16k x 16k pixels
• Auxiliary CCD’s – 4 CCDs– For guiding– Image analysis
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
4
FiltersFilters
• Primary set– Sloan u’, g’, r’, i’, z’– Johnson B, V– Narrow-band e.g. H up to 8000 km/s– Composite u’,B,V,i’ in four quadrants
• User filter
• Primary set– Sloan u’, g’, r’, i’, z’– Johnson B, V– Narrow-band e.g. H up to 8000 km/s– Composite u’,B,V,i’ in four quadrants
• User filter
More details see Harald Nicklas [4836-34]
More details see Harald Nicklas [4836-34]
28/08/2002 5
VST constructionsee [4836-09]
Mancini
Details instrument control see [4848-10] Baruffolo
VST constructionsee [4836-09]
Mancini
Details instrument control see [4848-10] Baruffolo
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
6
Wide Field Imaging ScienceWide Field Imaging Science
• Provide targets for VLT• 2/3 of time through ESO’s OPC• Individual programs
– Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs
• Sky Surveys• Long term archival research (10 yr mission)
• Science Cases– Finding exceptional single, rare objects– Statistics on large samples of objects
• Provide targets for VLT• 2/3 of time through ESO’s OPC• Individual programs
– Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs
• Sky Surveys• Long term archival research (10 yr mission)
• Science Cases– Finding exceptional single, rare objects– Statistics on large samples of objects
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
7
Large Data VolumeLarge Data Volume
• Wide-field imaging instruments, vast amounts of data– E.g.: VST = Southern sky (30 min exp, 300 nights/y) in
3 years. Large amount of data! 100 Tbyte
• Wide-field imaging instruments, vast amounts of data– E.g.: VST = Southern sky (30 min exp, 300 nights/y) in
3 years. Large amount of data! 100 Tbyte
• Science can only be archive-based• Science can only be archive-based
• Handling of the data is non-trivial– Pipeline data reduction– Calibration and re-calibration– Image comparisons and combinations– Working with source lists– Visualization
• Handling of the data is non-trivial– Pipeline data reduction– Calibration and re-calibration– Image comparisons and combinations– Working with source lists– Visualization
ESOcompliantESOcompliant}}
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
8
Concepts for solutionConcepts for solution
• Environment that provides systematic and controlled– Access to all raw and calibration data– Execution and modification reduction/calibration pipelines– Execution of source extraction algorithms– Archiving reduced data and source lists, or regenerates these
dynamically– Can be federated to link different data centers
• Environment that provides systematic and controlled– Access to all raw and calibration data– Execution and modification reduction/calibration pipelines– Execution of source extraction algorithms– Archiving reduced data and source lists, or regenerates these
dynamically– Can be federated to link different data centers
• Dynamical archive continuously grows, can be used for – small or large science projects– generating and checking calibration data– exchanging methods, scripts and configuration
• Dynamical archive continuously grows, can be used for – small or large science projects– generating and checking calibration data– exchanging methods, scripts and configuration
• Key functionality– Link back from source data to the original raw pixel data and
calibration files
• Key functionality– Link back from source data to the original raw pixel data and
calibration files
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
9
How to use thisHow to use this
• Deep multi-color fields– No need to take all data in one campaign– Combine data of particular quality, assess results– Select sources, visualize interesting ones, …
• 1-in-1,000,000 events spurious or not?
• Deep multi-color fields– No need to take all data in one campaign– Combine data of particular quality, assess results– Select sources, visualize interesting ones, …
• 1-in-1,000,000 events spurious or not?
• Large homogeneous surveys– E.g. weak lensing maps, cluster searches, star counts
• Large homogeneous surveys– E.g. weak lensing maps, cluster searches, star counts
• Variability (source list - or pixel based) – Proper motions (asteroids, nearby stars)– Flux variations
• Variability (source list - or pixel based) – Proper motions (asteroids, nearby stars)– Flux variations
• Monitor instrument (calibration files)• Monitor instrument (calibration files)
• Planning observations– View quality of existing data– Build on what already exists, add more filters, more
exposure time, better seeing, …
• Planning observations– View quality of existing data– Build on what already exists, add more filters, more
exposure time, better seeing, …
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
10
SolutionSolution
• Procedurizing– Data taking at telescope for both science and
calibration data– Full integration with data reduction– Design – Data model (classes) defined for data reduction and
calibration– View pipeline as an administrative problem
• Procedurizing– Data taking at telescope for both science and
calibration data– Full integration with data reduction– Design – Data model (classes) defined for data reduction and
calibration– View pipeline as an administrative problem
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
11
Observing ModesObserving Modes
• Dither matching max. gap between arrays ~400 pixels– N pointings (N=5 is standard) – nearly cover all gaps in focal plane and maximizes sky coverage– the context map will be very complex – couple the photometry among individual CCDs.
• Dither matching max. gap between arrays ~400 pixels– N pointings (N=5 is standard) – nearly cover all gaps in focal plane and maximizes sky coverage– the context map will be very complex – couple the photometry among individual CCDs.
• Jitter matching the smallest gaps in CCDs ~5 pixels– optimizes for maximum homogeneity of the context map – observations for which the wide CCD gaps are not critical– all data from single sky pixel originates from single chip
• Jitter matching the smallest gaps in CCDs ~5 pixels– optimizes for maximum homogeneity of the context map – observations for which the wide CCD gaps are not critical– all data from single sky pixel originates from single chip
• Stare reobserving fixed pointing positions multiple times– main workhorse monitoring instrument and optical
transients.
• Stare reobserving fixed pointing positions multiple times– main workhorse monitoring instrument and optical
transients.
• SSO observing Solar System objects– non-siderial tracking and the auto guiding switched off.
• SSO observing Solar System objects– non-siderial tracking and the auto guiding switched off.
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
12
Observing StrategiesObserving Strategies
• Standard– Single observations (one observing block)
• Deep– Long, multiple integrations– Selected atmospheric conditions– Several nights
• Frequent– Monitors same field– Timescales from minutes to months (overriding)
• Mosaïc– Maps areas of sky > 1o
• Standard– Single observations (one observing block)
• Deep– Long, multiple integrations– Selected atmospheric conditions– Several nights
• Frequent– Monitors same field– Timescales from minutes to months (overriding)
• Mosaïc– Maps areas of sky > 1o
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
13
Calibration proceduresCalibration procedures
Sanity checksSanity checks
Quality controlQuality controlCalibration proceduresCalibration procedures
Image pipelineImage pipeline
Source pipelineSource pipeline
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
14
Science ObservationsScience Observations
Photometric pipelinePhotometric pipeline
Bias pipeline
Flatfield pipeline
Image pipeline
Source pipeline
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
15
Monitoring Photometric CalibrationMonitoring Photometric Calibration
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
16
Share the loadShare the load
• Processing– Hardware
• Beowulf processors – 32 (most cases)• Multi Terabyte disks (10 – 100)
– Data reduction• Derive calibration• Run image pipeline (1 Mpx/s)
• Processing– Hardware
• Beowulf processors – 32 (most cases)• Multi Terabyte disks (10 – 100)
– Data reduction• Derive calibration• Run image pipeline (1 Mpx/s)
• Archiving– Storage
• Images (100’s Tbyte), Calibration files (10 Tbyte)• Source parameters (1-10 Tbyte)
– Federate (network speed)• 5 Mb/s (24 hours/day) full replication • 200 Mb/s no replication, on-the-fly retrieval
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
17
Contents of federationContents of federation
• Raw data– Observed images– Ancillary information
• Calibration results– Calibration files time stamped
• Reduced images– Single observation– Coadded images
• Software– Methods (pipelines) for processing calibration– Configuration files
• Source lists – catalogues– Extracted source information– Associated among different data objects
• Raw data– Observed images– Ancillary information
• Calibration results– Calibration files time stamped
• Reduced images– Single observation– Coadded images
• Software– Methods (pipelines) for processing calibration– Configuration files
• Source lists – catalogues– Extracted source information– Associated among different data objects
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
18
Concepts of federationConcepts of federation
• Federation maintained by a single database• Full history tracking
– of all input that went into result – providing on-the fly reprocessing
• Dynamical archive - Context as object attributes– Project: Calibration, Science, Survey, Personal– Owner: Pipeline, Developer, User– Strategy: Standard, Deep, Freq (monitoring), Mosaïc– Mode: Stare, Jitter, Dither, SSO– Time: Time stamping
• Software standards– Classes/data model/procedures– 00 – inheritance/ persistency– Python scripts/ c-libraries
• Federation maintained by a single database• Full history tracking
– of all input that went into result – providing on-the fly reprocessing
• Dynamical archive - Context as object attributes– Project: Calibration, Science, Survey, Personal– Owner: Pipeline, Developer, User– Strategy: Standard, Deep, Freq (monitoring), Mosaïc– Mode: Stare, Jitter, Dither, SSO– Time: Time stamping
• Software standards– Classes/data model/procedures– 00 – inheritance/ persistency– Python scripts/ c-libraries
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
19
ScheduleSchedule
• Hardware– Dome/Telescope erected at location– Camera on telescope Q1 2004– First run: Jan 2004– Second run: Mar 2004
• Software– Design – review Q2 2002- Done– Basic operations – Q4 2003– Evaluate and prepare for mass production 2004– Qualify and populate 2005– Deliver survey system – satellites
• Hardware– Dome/Telescope erected at location– Camera on telescope Q1 2004– First run: Jan 2004– Second run: Mar 2004
• Software– Design – review Q2 2002- Done– Basic operations – Q4 2003– Evaluate and prepare for mass production 2004– Qualify and populate 2005– Deliver survey system – satellites