SingularityContainer Workflows for Compute.
Gregory M. KurtzerCEO, Sylabs Inc.
@gmkurtzer
http://www.sylabs.io@SylabsIO@SingularityApp
Gregory M. Kurtzer
CEO and Founder, Sylabs Inc.
Previously spent ~20 years at LBNL/DOE as the HPC Systems Architect.
I’m also known for founding various open source projects like Warewulf, CentOS Linux, and most recently Singularity!
INTRODUCTIONS…
Host 2Host 1
APPLICATION CONTAINERIZATION 101
CPU Memory Devices
Kernel
Applications, Libraries, Services
CPU Memory Devices
Kernel
Apps, libs, servicesContainer
SCP, HTTP, FTP, Archive
An environment can be built on one host, encapsulated, and packaged up into a container image.
The container image can be copied to another host, and applications can be executed directly as if they are running native.
You can additionally isolate or integrate the container environment on the host as the need necessitates.
Singularity is differentiated by two primary categories:
• Container Format: Sylabs created an image format to encapsulate OCI and Docker based containers, which is single file based, cryptographically signed, trusted, and immutable.
• Runtime Engine: Standardizing on existing POSIX security practices, Singularity improves performance, integration, ease of use and reduces attack surfaces while enabling HPC and the growing need for compute based orchestration.
DESIGNED FOR SECURITY, MOBILITY, AND PERFORMANCE
RuntimeEngine
Environment Format
SINGULARITY IMAGE FORMAT (SIF)
”Building a container can be done in only 52 lines of code!” – Liz Rice, Container Camp 2016
SIF is a unique, single file, container encapsulation
format!
SIF is to containers what RPM and DEB is to source code!
Immutable RuntimeContainer Image
Global Header
Recipe Definition
Labels
Environment
Writable Overlay
Signature Block
CR
YP
TOG
RA
PH
ICA
LLY
SIG
NE
D
Descriptors
SIF encapsulates OCI and Docker containers into a single file adding benefits such as:
• Guaranteed immutable and reproducible
• Easy to move, share, archive, etc..
• POSIX compatible
• Encapsulates the entire application and environment stack
• Cryptographic signatures and validation
• No layers or dependencies
• No tarballs, SIF is the runtime format
• Encryption (with in-kernel description) coming soon
A NEW DELIVERY PARADIGM FOR SOFTWARE
Singularity ContainerTRUST
sha2
56:9
4ed
0..
sha2
56:9
4061
..
sha2
56:a
a74a
...
sha256:becac…
…
Host Operating System
Presentation Layer
Root Owned Container Daemon
Network Registry
SIF PERFORMANCE
Objectives:
1.Measure scaling of python startup
and import speed with increasing
numbers of concurrent python
interpreters
2.Compare scaling of a standard
python installation with an identical
containerized installation
Note: Underlying file system is NFS, max
jobs was 5120 over 320 nodes, graph is
logarithmic on both axis.
DR. WOLFGANG RESCH
HTTPS://GITHUB.COM/WRESCH/PYTHON_IMPORT_PROBLEM
Invocation performance over shared storage
Singularity provides absolute trust and accountability
Execution of containers can be limited to only valid keys, and/or key fingerprints
If a malicious user is found, keys are revoked from the SylabsKeyStore, limiting exposure
ABSOLUTE TRUST OF ALL WORKLOADS
$ singularity pull container.sif library://user/container$ singularity verify container.sifData integrity checked, authentic and signed by:
Gregory Kurtzer [email protected], KeyID F4EIAL82E…
$ singularity sign container.sif$ singularity push container.sif library://user/container
EXTREME MOBILITY OF COMPUTE – BYOE
Absolute mobility from laptop, to
HPC, cloud all and the way out to
the edge.
• Changing the packaging and mobility paradigm for
application and data
• Disrupts the barriers of portability and bridges the
gaps between all available resources
• From private resources, to public clouds and all the
way out to edge and IoTLocal Compute
IoT Edge
NVIDIA DGX
Designed for the complicated integration needs of compute
• Container Runtime:• Works on all supported Linux Distributions (runtimes and kernels)
• Designed for massive efficiency and performance
• Additional support for alignment between user and kernel space
• Container Image:• Designed for absolute mobility, user freedom, and reproducibility
• Highly performant on shared and parallel file system deployments
• Can be easily shared, archived, and controls compliant; containers are just data
• Environment:• Optimized for application workflows like MPI and schedulers
• Allows direct access to GPUs, InfiniBand, FPGAs, file systems, data, etc.
COMPATIBLE AND INTEGRATION AWARE
Data is shared between container and host as fluently as if contained applications were running natively on the host.
NATIVE HOST INTEGRATION
$ singularity exec ubuntu.sif pwd
$ singularity exec ubuntu.sif python ./python_script_in_pwd.py
$ cat python_script_in_pwd.py | singularity exec docker://python:latest python
Singularity integrates with all batch resource managers, with zero modifications, by calling the Singularity command directly within the
batch script
BATCH SUPPORT
#!/bin/sh
#SBATCH --N 32
mpirun singularity exec ~/ubuntu.sif mpi_program.exe
With a PMIx supporting launcher, you can run a fully contained MPI process directly from a compatible resource manager
MPI AND SLURM
$ srun -n 32 singularity exec ubuntu.sif mpi_program.exe
When a container includes a GPU enabled application and libraries, Singularity (with the “--nv” option) can properly inject the required Nvidia
GPU driver libraries into the container, to match the host’s kernel
GPU / CUDA SUPPORT
$ singularity exec --nv ubuntu.sif gpu_program.exe
$ singularity run --nv docker://tensorflow/tensorflow:gpu_latest
MVAPICH2 APPLICATION PERFORMANCE
Benchmarks published by MVAPICH team at Ohio State Universityhttp://mvapich.cse.ohio-state.edu/performance/singularity-application/
IMB NETWORK PERFORMANCE
Benchmarks published by SDSC at UCSDhttps://dl.acm.org/citation.cfm?doid=3093338.3106737
IMB SendRecv Run using Singularity and Non-Singularity IMB PingPong Run using Singularity and Non-Singularity
Content published here with explicit permission from the authors
OSU NETWORK LATENCY
Benchmarks published by SDSC at UCSDhttps://dl.acm.org/citation.cfm?doid=3093338.3106737
Content published here with explicit permission from the authors
LS-DYNA PERFORMANCE
Benchmarks published by the Dell EMC HPC Innovation Lab http://en.community.dell.com/techcenter/high-performance-computing/b/general_hpc/archive/2018/02/19/performance-of-ls-dyna-on-singularity-containers
“The performance difference while running LS-DYNA within Singularity containers remains within 2%, which is within the run-to-run variability of the application itself..”
Designed for the security needs of compute
• Container Engine:• Singularity has no root owned daemon processes
• Implements privilege separation over an API to a secure thread
• DoD: Singularity is the only allowed container system
• Audited and certified by EU lab for use on the European Compute Grid
• NSF grant for 3rd party security assessment (in progress, going well!)
• Container:• Singularity containers are immutable
• Cryptographically signed and verifiable
• Public keys can be managed over standard HKP protocol (or Sylabs key services)
• Environment Requirements:• Containers are run as the calling user
• Blocks all privilege escalation from within the container
SECURITY FOCUSED
You are always yourself within a Singularity context, and Singularity will block escalation attempts within the container
Even if you know the root password, even if you have sudo installed, even if you implement a SUID hack, Singularity will prevent privilege
escalation
SECURITY BLOCKS
$ singularity exec centos.sif whoami
$ singularity exec centos.sif sudo su -
$ singularity exec centos.sif /proc/$$/root/bin/su
• System administrators, always in 100% control
• Supports User Namespace (when kernel supports it)
• Linux Capabilities (per user or group ACLs)
• Directly integrates with host’s:• SELinux
• AppArmor
• Seccomp
• Container execution can be limited by:• Container owner or group
• Location on file system (trusted paths)
• Whitelist/blacklist by signed container finger prints
ADDITIONAL SECURITY FEATURES
• Backend code updated to GO
• Fully OCI compatible (3.1: `singularity oci …`)
• Integration with enterprise standards:• OCI: Image support with all container registries
• CNI: Support for all container networking options (port forwarding, NAT, etc..)
• CGroups: Resource limitations
• SIF updates• Encapsulation of OCI and Docker formats
• Immutable and 100% guaranteed reproducible
• Cryptographically signed and verifiable
• No tarballs or archives: SIF is the runtime container format
• Multi-stage builds, and “disposal” development overlay• Nvidia HPC-CM container builder
• Build tool integration: Spack, EasyBuild,… Docker, Img, Buildah, etc…
• Native support for MacOS and Windows (coming soon)
• Kubernetes Support (native CRI)
WHAT ELSE IS NEW AND COMING SOON
NATIVE SINGULARITY SUPPORT ON MACOS
Singularity Desktopcoming soon (Q1 2019)
BRIDGING THE GAP BETWEEN COMPUTE AND SERVICES
Native integration between Singularity with OCI, Kubernetes and Nomad to be completed in Q1 2019.
AI workflows typically have a “train” and “execute” workflow, where the training is the most
computationally intensive
Singularity enables this workflow and enables large scale distribution and
provides the needed assurance, security and accountability for scale and
production
Train
Distribute
Build
Inference
ARTIFICIAL INTELLIGENCE MULTISTAGE WORKFLOWS
• Parallel training
• Distribution of trained models
• Real time AI / compute
• Data streaming
• Complete validation and trust
• Supporting all tools
• “HPC as a Service”
Singularity is the unifying substrate for all compute needs
EXPANDING THE WORKFLOW SUPPORT OF THE ECOSYSTEM
Data Stream(s)
KubernetesKafka - Stream Splitter and Balancer
Compute Based Service
Compute Based Service
Compute Based Service
Compute Based Service
Real time collectors, Visualization, Storage, analytics, etc.
TENSORFLOW GPU PERFORMANCE
HPC and AI Solutions Engineering group at Dell EMC https://www.nextplatform.com/2018/03/19/singularity-containers-for-hpc-deep-learning/
“The performance comparison between a bare metal versus a containerized version of the framework at 32 Tesla V100 is still under 2%, showing negligible performance delta between the two.”
NVIDIA SUPPORTS SINGULARITY
Three years counting, HPC Wire awards for Singularity
Singularity, the container runtime of choice for HPC, EPC/AI, and
enterprise workloads
As of Singularity 3.0:
• Multi-millions of container runs per day
• Approx 250,000 downloads (not counting redistributors)
• Installed on over 5 million x86 cores, 250k ARM
The same reasons that make Singularity fantastic for HPC,is what makes Singularity fantastic for all enterprise compute needs!
MASSIVE ADOPTION AND GROWTH
SYLABS IN THE NEWS
Sylabs Among “The 10 Hottest Container Startups Of 2018”
• SingularityPRO:• Fully supported versions of Singularity
• Code curated, trusted builds, RPM/DEB, simple deployment
• Feature identical to open source
• Stable with long term life
• Per node or site licensed
• Sylabs Cloud Services (SCS):• KeyStore: Public key service for signed containers
• Container Library: A place to host, develop, sell, reference, and share containers and AI trained models
• Remote Builder: safely and securely build containers without root, with a web based development interface or use the native Singularity CLI
• Pipelines: CI/CD configurable pipelines for DevOps workflows (coming soon)
• Professional services, support, training, development, etc.
SYLABS OFFERINGS
THE SYLABS TEAM
Singularity User’s Group!
March 12th-13th
San Diego Supercomputing Center
CFP closing tomorrow!
THE INAUGURAL SINGULARITY USER GROUP MEETING
Come see me after if you want Singularity swag!
T Shirts, Stickers, Pens, etc…
SingularityContainer Workflows for Compute.
Gregory M. KurtzerCEO, Sylabs Inc.
@gmkurtzer
http://www.sylabs.io@SylabsIO@SingularityApp