Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice

Wolfgang Gentzsch & Burak Yenier

ISC Frankfurt, July 16, 2015

Docker Workshop

Containers for Scientific & Engineering Applications

2012: UberCloud started

its free voluntary Cloud Experiments

HPC as a Service, on demand, in a team experiment

to explore the end-to-end process

using cloud computing resources

as a service, on demand, at your fingertips

and learning how to resolve the roadblocks.

TODAY: 172 experiments, 2500 companies, 72 countries, 50 case studies

Supported by our sponsor INTEL

Major Cloud Roadblocks

Supply Cloud providers ISVs Consultants Trainers

Demand Engineers Scientists Data analysts Experts

.

.

.

.

.

Complexity

Data Transfer

Security Licensing

Uncertain Cost

Roadblocks

2015: The UberCloud Marketplace

Supply Cloud providers ISVs Consultants Trainers …

Demand Engineers Scientists Data analysts Experts

Marketplace

And HPC Docker

Marketplace

UberCloud application containers based on Docker

UberCloud application containers

Ready-to-execute packages of software, designed to deliver the tools that an engineer needs

Ready to execute, in an instant. No need to install software, deal with complex OS commands, or configure.

The ISV or Open Source tools are pre-installed, configured, and tested, and are running on bare metal, without loss of performance.

Based on open Linux container technology

Enhanced for engineering & scientific applications

ANSYS, CD-adapco, OpenFOAM, Gromacs, Scilab, and more

Virtual Machines vs Docker

VM: Each virtualized application includes not only the application and the necessary binaries and libraries, but also an entire guest operating system.

Docker Engine container comprises application and its dependencies. Isolated process in user space on host operating system, sharing kernel with other containers. Enjoys resource isolation and allocation benefits of VMs but more portable and efficient.

9

Builder

Launcher

Controller ISV Data Tools

Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.

Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.

Collect granular usage data, logs. Monitor, alert, report.

Any Workstation Any Cluster Any Cloud

Run Time Run Time Run Time

Containers: Build once, run anywhere

Builder

Launcher

Controller ISV Data Tools

Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.

Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.

Collect granular usage data, logs. Monitor, alert, report.

Any Workstation Any Cluster Any Cloud

Run Time Run Time Run Time

Containers: Build once, run anywhere

Builder

Launcher

Controller ISV Data Tools

Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.

Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.

Collect granular usage data, logs. Monitor, alert, report.

Any Workstation Any Cluster Any Cloud

Run Time Run Time Run Time

Containers: Build once, run anywhere

Builder

Launcher

Controller ISV Data Tools

Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data.

Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed.

Collect granular usage data, logs. Monitor, alert, report.

Any Workstation Any Cluster Any Cloud

Run Time Run Time Run Time

Containers: Build once, run anywhere

Containers remove Portability & Standard related challenges. By integrating 3rd party tools we tackle the following:

UberCloud multi-container environment

Docker Runtime Docker Runtime

Docker Runtime

. . . .

NFS

Host Host Host

Container Container Container

MPI

Ve

rtic

al s

cali

ng

Horizontal scaling

Dramatically simplifying Packaging, Porting, Access, Use, Scaling, with Low Overhead

OpenFOAM Test: Bare Metal vs Docker

Performance Test Results demonstrating comparison between the same OpenFOAM run (with 1.4 million cells) repeated on bare metal and in a Docker container.

Dockerized Technical Computing Ecosystem

Experts CFDsupport

CAEtech Dacolt

Any Inhouse &

Open Source

ISV ANSYS

CD-adapco OpenFOAM

Resource AWS

Azure LLNL

UberCloud multi-container environment

Intel MPI libraries pre-setup for multi-node high performance

Intel compilers achieve architecture specific high performance

Networking between containers

Setting up shared file system access (NFS)

Secure communication (ssh) between containers

Benefits from single-node transferred to multi-node containers:

No setup of the environment

Portability and packaging

Easy access and use

Low overhead

+

Hot Spot Question for Jérôme Petazzoni

Docker recently announced that it acquired SocketPlane to help with container networks. I am very interested in some more background and details and how this could support HPC multi-container approach.

There are multiple container implementations, Docker, CoreOS and now a collective effort, runC. Can you talk about what this all means for the community.

Virtualization has been around for a long time, why re-invent the wheel?

In HPC clusters shared file systems are quite commonly used. Can you comment on how to secure a Docker installation in conjunction with a shared file system. Wouldn't the shared file system be vulnerable when Docker is in the mix?

ANSYS Container Demo

ANSYS multi-physics (5 s/w) running inside Docker container

Pre- and post-processing in the cloud works the same way that it would work

on the engineer’s desktop

User experience in the cloud matches that on the desktop

GPU support for high-resolution remote visualization

Automated monitoring keeps you connected, under control

4-min demo: https://goo.gl/YIrTr4

6-min demo: http://goo.gl/pDXiTq

Containers as basic technology for the UberCloud SaaS Factory

UberCloud SaaS Factory

Software SaaS

Commercial ISV Open Source In-house

Public Cloud Private Cloud On-Premise

$$ $

We turn high-TCO, complex software into user-friendly pay-per-use services:

Development Services (Docker) --------------------

Deployment & Management Services

------------------ Marketplace

Providers Consumers

$

UberCloud SaaS Factory

Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice

Thank You !

Please register at

http://www.TheUberCloud.com

It’s free