The ALMA Software The ALMA Software SystemSystem
Joseph Schwarz (ESO)
ACS Course, 15 January 2007 The ALMA Software System
ALMA is…
• A radio aperture-synthesis telescope covering ν50-1000 GHz (λ ~ 0.3 - 6 mm), and consisting of...
• 54 12m antennas and 12 7m antennas that will make detailed images of astronomical objects.
• They can be positioned as needed with baselines from 0.15 to 14 kilometers so as to give the array a zoom-lens capability, with angular resolution reaching 10 milliarcseconds.
• A leap of over two orders of magnitude in both spatial resolution and sensitivity over existing instruments
• ALMA's great sensitivity and resolution make it ideal for medium scale deep investigations of the structure of the submillimeter sky.
• A joint project of the North American, European and Japanese astronomical communities.
Courtesy of Al Wootten, ALMA/US Project Scientist
Location: the Llano de Chajnantor, Chile, at an altitude of about 5000m.
ACS Course, 15 January 2007 The ALMA Software System
Complete Frequency Access
ACS Course, 15 January 2007 The ALMA Software System
Where can such transparent skies be found?
Living Earth
ALMA
...or here
Here...
ACS Course, 15 January 2007 The ALMA Software System
Performance requirements
• Baseline correlator H/W produces ~ 1 Gbyte/s• Must reduce to average/peak data rates of 6/60
Mbyte/s (baseline)• Raw (uv) data ~ 95%, image data ~ 5% of the total• Implies ~ 180 Tbyte/y to archive• Archive access rates could be ~5 higher (cf. HST)
• Feedback from calibration to operations• ~ 0.5 s from observation to result (pointing, focus, phase
noise)
• Science data processing must keep pace (on average) with data acquisition
ACS Course, 15 January 2007 The ALMA Software System
Software Scope
• From the cradle… • Proposal Preparation• Proposal Review• Program Preparation• Dynamic Scheduling of Programs• Observation• Calibration & Imaging• Data Delivery & Archiving
• Afterlife: • Archival Research & VO Compliance
ACS Course, 15 January 2007 The ALMA Software System
And it has to look easy…
• From the Scientific Software Requirements:• “The ALMA software shall offer an easy to use interface to
any user and should not assume detailed knowledge of millimeter astronomy and of the ALMA hardware.
• “The general user shall be offered fully supported, standard observing modes to achieve the project goals, expressed in terms of science parameters rather than technical quantities...”
• The expert must still be able to exercise full control• But what is simple for the user will therefore be
complex for the software developer.• Architecture should relieve developer of unnecessary
complexity• Separation of functional from technical concerns
ACS Course, 15 January 2007 The ALMA Software System
System data flow
6-60 Mbyte/s
~0.5 s feedback time
ACS Course, 15 January 2007 The ALMA Software System
Distributed development
ALMAOSF & AOS
DRAO
Penticton
ALMAATF
Arcetri
Observatory
Jodrell Bank
Brera
ObservatoryIRAM
Grenoble
ATC
Edinburgh
NAOJ
ESO
NRAO
Obs de
Paris
Univ.Calgary
c
c
SantiagoSCO
MPI Bonn
DAMIR/IEMMadrid
10
ACS Course, 15 January 2007 The ALMA Software System
Distributed operations
~30 km
~15
km
This facility must operate 24 hours/day, 365 days/year
~30 km
~15
km
11
ACS Course, 15 January 2007 The ALMA Software System
12
ACS Course, 15 January 2007 The ALMA Software System
Distributed runtime
• Devices @ the AOS (5000m)• 66 Antennas
• Every antenna in ALMA has its own computer (ABM) and its own container on that computer.
• 40 devices (LRUs) per antenna• Each hardware device on the antenna is represented as a component in that
antenna’s container.
• Correlator
• 1000 LRUs in the ARTM
• Operations, Archive, Data reduction/analysis• At the OSF (2900m, 30 km from the AOS)
• ACS will have to manage > 3500 components
ACS Course, 15 January 2007 The ALMA Software System
"What the deuce is it to me?" You say that we go round the sun.If we went round the moon it wouldnot make a pennyworth of difference
to me or to my work."
Separation of concerns
• Functional: Physics, algorithms, hardware…• PIs can concentrate on their research specialties
• Software encapsulates aperture synthesis expertise
• Technical: Communications, databases, etc.• Subsystem teams should concentrate on function
• Technical architecture should provide simple and standard ways to:
• Access remote resources
• Store and retrieve data
• Manage security needs
• Communicate asynchronously between subsystems, components
ACS Course, 15 January 2007 The ALMA Software System
ALMA Common Software (ACS)
• Main vehicle for handling technical concerns• Framework for distributed object architecture
• Used all the way down to device level in control system• Built on CORBA, but hides its complexity
• Wraps selected CORBA services• Multi-language support: Java, C++, Python• Vendor-independent• High-quality open-source ORBs available (e.g., TAO)
• System evolving to meet developers’ needs• Initial developer resistance• Developers now asking for more
• Dedicated team of systems-oriented developers
ACS Course, 15 January 2007 The ALMA Software System
Data modeling & code generation
• Define data structure and content abstractly• UML: Project (APDM) & Science (ASDM) data models
• Spreadsheets: Hardware device characteristics
• Open-source code generation framework• Wrapped & supplied by ACS
• Generate automatically:• XML schemas, Java, C++, SQL, docs from UML
• Binding classes from schemas
• Type-safe native language access to data
• Automatic validation possible
• Exchange of XML documents between subsystems• In a language-independent way
• Via the Archive or via the network
ACS Course, 15 January 2007 The ALMA Software System
Avoiding nasty surprises
• Iterative development• Releases every 6-months (4 major, 3 minor to date)
• 6 major ACS releases to date
• Function-based teams develop & deliver integrated s/w on shorter time scales (typically, 3-4 FBTs per release cycle)
• Periodic “CDR”s: Focus on plans for next FBTs, next release
• Emphasize integrated system • Simulate interferometry while waiting for hardware
• We’ll be ready for first fringes at the ATF in Q2
• Two integrated user tests at the Antenna Test Facility (ATF) so far
• Optical pointing: feedback led to...
• Holography: greatly improved robustness, user satisfaction
20
ACS Course, 15 January 2007 The ALMA Software System
ATF: Where the action is
VLA site, Plains of San Augustin, New Mexico, USA
ACS Course, 15 January 2007 The ALMA Software System
ALMA Schedule
• 2007 Q2: • First fringes on prototype antennas at ATF (VLA site)
• First production antenna on site at OSF in Chile
• 2008 Q1: Second production antenna at OSF
• 2008 Q3: Two-antenna interferometer at OSF
• 2008 Q4: “Array software” complete
• 2009 Q1: • Three-antenna interferometer at AOS (@ 5000m)
• Start of commissioning
• 2010 Q4: Early science operations
• 2012 Q3: Hardware & “Observatory s/w” complete• Full science operations (full antenna complement)