Click here to load reader

Changes at ECMWF

  • View

  • Download

Embed Size (px)


Changes at ECMWF. Walter Zwieflhofer Head of Computer Division. Operational system (October 2001). Data assimilation 4D-Var with a 12-hour data window Horizontal resolution: 120 km for inner and 40 km for outer loops Global atmospheric forecasts - PowerPoint PPT Presentation

Text of Changes at ECMWF

PowerPoint Presentation4D-Var with a 12-hour data window
Horizontal resolution: 120 km for inner and 40 km for outer loops
Global atmospheric forecasts
Forecasts to 10 days from 12 UTC at 40 km resolution
Ensemble forecasts to 10 days with 51 members at 80 km resolution
Four forecasts per day to 3 days to provide boundary conditions for some of the ECMWF Member States (40 km, 3D-Var)
Ocean wave forecasts
European waters forecasts to 5 days at 15 km resolution
Seasonal forecasts
21r1 Assimilation of raw TOVS/ATOVS radiances
21r4 New error statistics for 4D-Var Jb, SSMI wind data,
corrected use of humidity observations, increased
PBL resolution, improved cloud, convection and
additional ATOVS data, improved land-surface,
sea-ice and LW radiation
23r3 T511/159 resolution, second ATOVS satellite,
increased angular resolution in wave model
N Hem Mean reduction in day-one error measured in pre-operational trials S Hem
Planned improvements
2002 Increase of the inner loop resolution of 4D-Var from 120 km to 80 km
2003-2004 Assimilation of cloud and rain in 4D-Var
Higher vertical resolution in free troposphere and lower stratosphere
Assimilation of AIRS data
2005-2006 Assimilation of IASI data
Uses the operational configuration of the forecasting system and allows researcher to introduce modifications to:
input data (including the selection of observations)
the model code
the resolution to be used and many other parameters
Written Java, with a client part that can be installed on remote machines
Allows registered external users to configure and submit experiments via the Internet
Desktop systems (procurement completed in 2000)
Decision was taken to replace all proprietary UNIX workstations with PCs running LINUX over a period of three years
Data Handling System – DHS (procurement in progress)
A scalable system that will be able to support the expected growth over 6+ years is required
Invitation to Tender for a new system was issued earlier this year
Supercomputer – HPC (procurement in progress)
Fujitsu systems are rented and the contract ends on 31 Dec 2002
Invitation to Tender for a replacement to cover the period 2003-2006 was issued in March this year
DHS procurement
Archive holds currently some 400 TB of data (primary data) and has been growing by some 60% per annum
Two services need to be supported at ECMWF:
ECFS, a large-scale UNIX-like filesystem for general use
MARS, a customised meteorological archival and retrieval system
Vendors were asked to offer a scalable storage system and were given the option to offer a replacement for the ECFS and/or MARS software layer
DHS and HPC procurements were separated because:
Lifetime of the data handling systems tends to be considerably longer than that of the HPC systems
Allows a simpler HPC procurement not affected by the complexities associated with the replacement of the archive system
HPC procurement
ECMWF asked for a phased introduction of resources covering the period 2003-2006, with a parallel run during the second half of 2002
The specifications allowed for some flexibility, enabling vendors to adapt their response to their product development plans. This included the option to propose a 3-month shift of the service period
The funds available were published
Vendors were required to commit to performance increases, expressed as factors, over the performance of the Fujitsu systems installed at ECMWF
Requirements for disk space, I/O and network bandwidth were expressed as function of these factors
IFS tests (for the performance commitments)
4D-VAR (T511/255 L60; 2, 4 or 6 copies depending on PHASE)
DETERMINISTIC FORECAST (T799 L90; 2 or 4 copies)
EPS (T399 L60; 50 or 100 copies)
Burton Smith taxonomy:
Suitable for both T-type and C-type systems
IFS model runs well on all architectures
4D-Var is very sensitive to communication bandwidth and has sections on which scalar machines do well
Resolutions representative for the workload planned for 2003-2006 were used
Vendors were allowed to optimize the codes
However, changes that would affect the readability or portability were not permitted
Some vendors used a hybrid MPI / OpenMP approach
Some vendors found that such a hybrid approach did not improve the results obtained by using MPI only
Percentages of peak:
4D-Var scaled not as well the deterministic forecast and the EPS
Scalar machines: 3% to 12%, depending on test/product
Vector machines: 14% to 38%, depending on test/product
Rule of thumb for ECMWF’s codes was confirmed:
~10% for scalar architectures and ~30% for vector architectures
CAS 2001, Annecy Slide *
Status of both procurements
The Tender Evaluation Boards have completed their work and have made recommendations to the ECMWF Director
The Director has awarded both contracts, subject to approval by ECMWF Council
Documents detailing the respective procurement process and the proposed solutions have been sent to the technical and financial committees of the ECMWF Council
These committees will meet in November to consider ECMWF’s proposals and to make recommendations to Council
Council will meet in December and (hopefully) authorize the Director to sign the appropriate contracts
Linux PCs

Search related