As the world's largest oil producer, Saudi Aramco is under constant pressure to work more efficiently and more safely in the most complex of conditions. It demands rapid technology development that taxes computational resources.

With these requirements in mind, Saudi Aramco collaborated with simulation software vendor ANSYS and Cray customer KAUST Supercomputing Core Lab (KSL) to see what results they could get on a typical oilfield application running ANSYS® Fluent® on KSL's Cray® XC™ supercomputer. Could they reduce design development time? Better predict equipment performance? Scale the problem up?

Not only did they answer "yes" to all, they broke a supercomputing scaling record by 5x and reduced the run time from weeks to a single day.

Challenge For their test, the team modeled a multiphase separator vessel. Used for separating well fluids (gas, oil, water), these large cylindrical vessels are the first process the well stream meets when it comes to the surface.

To enhance the efficiency of these vessels, oil producers retrofit them with efficient internals Saudi Aramco uses computational fluid dynamics (CFD) modeling and simulation to select or design the best internals as well as predict the vessels' performance under varying conditions.

Multiphase problems are complex and require multiple global synchronizations. These factors make them more difficult to scale than a single-phase laminar or turbulent flow simulation. Add in unstructured mesh and complex geometry and the problem becomes even more difficult.

CASE STUDY

www.cray.com/solutions/energy

“Not all codes can scale to full scale for real-life engineering problems. This full scale Fluent run is unprecedented.”

- Rooh Khurrman, Staff Scientist, KAUSTSupercomputing Core Lab

Saudi Aramco, KAUST & ANSYS Collaboration Breaks Record, Makes Safety and Efficiency Gains

Thus, it was critical that the application be representative of an actual design problem. "Our scalability tests are not just designed for the sake of obtaining scalability at scale," says KSL director Jysoo Lee. "This was a typical Aramco separation vessel with typical operation conditions."

SolutionScalability reveals the efficiency of an application by using increasing numbers of parallel processing elements. Cray XC supercomputers like KSL's "Shaheen II" exploit parallelism, making them ideal for exactly this type of problem.

Shaheen II is composed of 6,174 nodes representing a total of 197,568 processor cores tightly integrated with a richly layered memory hierarchy and interconnect network.

Still, achieving linear scalability at full scale depends on a number of factors lining up, says KSL staff scientist Rooh Khurram: "The code needs to be scalable, the hardware needs to be performing at its best, the optimized libraries need to be linked, the right environment and compiler flags need to be selected, and perfect load balance and minimization of communication needs to be ensured."

In preparation for the run, the team conducted compute and interconnect tests to ensure nodes falling below the threshold value were disabled and broken or sub-optimal links were addressed.

ResultsUsing ANSYS Fluent on Shaheen II, the problem scaled to almost 200,000 cores — the system's full core count. This number represents a 5x increase over a record 36,000 cores reached three years ago. And the problem ran in a day rather than weeks.

"Not all codes can scale to full scale for real-life engineering problems," says Dr. Khurram. "This full scale Fluent run is unprecedented. It is a good reflection on the health of the machine and the parallel implementation of the code."

These results also "confirm the value of high-performance computing in reducing turnover time," says Ehab Elsaadawy, computational modeling specialist and oil treatment team leader at Saudi Aramco. "This helps in building the business case … for new HPC hardware."

Wim Slagter, director of HPC and cloud alliances at ANSYS, takes an even broader look at the project's significance: "To continue breaking records and redefining the possible in the real world, we must continually push the envelope of computing power."

901 Fifth Avenue, Suite 1000 | Seattle, WA 98164 | Tel: 206.701.2000 Fax: 206.701.2500 | www.cray.com©2017 Cray Inc. All rights reserved. Cray, the Cray logo and Cray XC are registered trademarks of Cray Inc. All other trademarks mentioned herein are the properties of their respective owners. 20171017KL

SYSTEM DETAILS

Cray® XC™ series supercomputer

7.2 PF peak performance

36 cabinets

6,174 nodes

197,568 processor cores

17 PB storage

CASE STUDY

www.cray.com/solutions/energy