© Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 0

© Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation24 March 20091

© Fujitsu Laboratories of Europe 2009

Chaste WorkshopUniversity of Oxford

24 March 2009

HPC and Chaste:

Towards Real-Time Simulation James Southern

Fujitsu Laboratories of Europe


The petascale challenge

• Petascale computing has been achieved in the last 12 months.

• 1015 (one million billion) floating point operations per second

• There will be many challenges in building and using the next-generation computer systems.

• One of the biggest is developing applications, algorithms and numerical libraries that will make efficient use of the computer.

• Fujitsu aims to contribute through collaboration with world-leading modelling and simulation groups.


First petascale system: IBM’s Road Runner

June 2008


“World’s most powerful supercomputer for science”

November 2008

Cray XT5 (Opteron)

Oak Ridge

The world’s second petascale computer


Petascale computing and the heart

• The MEXT Next-Generation Supercomputer Project will provide a ‘leadership’ 10 petaflop/s resource for Japan by 2012.

• 10 times faster than the world’s current fastest computer.• A budget of £500 million.• Fujitsu to be one of the main contractors

• One of the main application fields to be targeted is the life sciences.• Real-time simulation of a beating heart is a life science application that requires

petascale computing.


Parallel solution of the bidomain equations

• Current simulation methods are >100,000 times slower than real-time for a realistic mesh on a single processor.

• Large scale parallel computing will be needed if we are to realize real-time heart simulations in the near future.

• This leads to a series of additional factors that must be considered when solving the bidomain equations.

• How do we minimize communication between processors?

• How do we ensure each processor does the same amount of work?

• It is important that we balance these factors with the scientific computing issues that also have an impact.


Making the code run faster

• Improve performance of existing methods.• Profile and benchmark the code to identify parts of the

code that could be improved.• Which parts of the code take most of the time?• Are some processors spending a lot of time waiting for others

to finish executing code?

• Develop new, improved methods.• More efficient solution of the bidomain equations.

• Better linear solvers, preconditioners.

• Improved ODE solvers.

• Getting the same level of accuracy for less computation.• Adaptivity.

8


Benchmarking and profiling

• Benchmarking tells us how fast Chaste is in comparison to similar codes.

• Difficult to get a precise measure as we often can’t run exactly the same simulations.

• Meshes used by others not available to us, not always possible to get hold of other applications to run comparison.

• Indications are that Chaste is very competitive.

• Profiling showed that partitioning was causing a load imbalance when we used large meshes.

• Some processors doing a lot more work than others.

• Solution: re-distribute mesh nodes over the processors by optimizing with METIS library.

9


Improved load balancing


Improved load balancing


Parallel scaling

• Good load balancing is key to good parallel performance.• Allows us to minimize the time each processor spends

waiting for synchronization with others.

• Introducing METIS has significantly improved load balancing in Chaste.

• Also important to limit the amount of time processors have to spend communicating with one another.

• This time could be better used doing calculations.

• How good is Chaste’s parallel performance currently?• For a perfectly load balanced code with no communication

overhead we’d expect linear speedup.• i.e. Twice as many processors Half the execution time

12


Parallel scaling


Parallel efficiency


Summary

• Chaste has been designed to work with HPC.• Should not impact performance on smaller systems.

• We regularly benchmark and profile the code both sequentially and in parallel.

• Performance improvements can be rapidly implemented to address any issues raised by profiling.

• Chaste’s parallel performance is very good (up to at least 32 processors).

• We will be working to ensure that this scaling is carried forward onto even larger systems.

15



• A flagship eHealth project in FP7.

• Computational frameworks and ICT-based tools for multiscale models of the human anatomy, physiology and pathology.

• Libraries of data and toolbox for simulation and visualisation.

• A key facilitator for:• Personalized (patient-specific) healthcare

solutions.

• Early diagnosis and predictive medicine.

• Assessment of safety/efficacy of drugs using patient-specific computational models.

• Innovative personalized drugs.

Virtual Physiological Human


preDiCT

• Collaborative project to create a framework for in silico testing of drugs, using Chaste as the base software.

• Systems-based approach, using multi-scale and multi-physics techniques to integrate components from different levels of biological organization.

• Partners from high-performance computing, academic institutions and the pharmaceutical industry.

• Ultimate goal is to facilitate real time in silico testing of the cardiac toxicity of drugs.

• FLE leads the Computational Tools and Methodologies work package of the project.


preDiCT partners

Documents

© Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 0