Embed Size (px)
Citation preview
A scalable and flexible platform to run various types of resource intensive applications on clouds
ISWG2015 3-5 June 2015
Budapest, Hungary
Tamas Kiss, Hannu Visti, Gabor Terstyanszky, Gregoire Gesmier
[email protected]@westminster.ac.uk
• Funded by the European Commission FP7 programme, FoF: Factories of the Future• July 2013 – December 2015• EUR 4.5 million overall funding• Coordinated by the University of Westminster• 29 project partners from 8 European countries• 24 companies (SMEs) and 5 academic/research institutions
An example for intensive industry collaboration
CloudSME - EU funded industry oriented research project
*The CloudSME project develops a cloud-based, one-stop-shop solution providing a scalable platform for small or larger scale simulations, and enable the wider take-up of simulation technologies in manufacturing and engineering SME’s.
The CloudSME projectCloud-based Simulation platform for
Manufacturing and Engineering
* Defines generic and concrete business models for SMEs in the manufacturing/engineering sector to facilitate the take-up of cloud-based simulation solutions
* CloudSME builds a simulation platform that allows seamless access to multiple heterogeneous cloud resources and provides a high level of abstraction to users when accessing these resources for simulations in a one-stop-shop solution.
* Provides a Platform as a Service (PaaS) solution to build customised cloud applications
* Enables simulation software providers to offer Software as a Service (SaaS) simulation solutions
* Enables SMEs in the manufacturing and engineering domain to access simulation services
* Provides seamless access to HPC resources in order to speed up the simulations on-demand
The CloudSME Simulation Platform
Targeted application areas
*High Performance Computing (HPC)*relatively small number of end-users with high computation
demand
*many used simulation before
*enable software vendors to extend their product with cloud support
*High Throughput Computing – parameter sweep (HTC)*use of multiplier entities (e.g. consultant companies) and
technologies (e.g. templates)
*although smaller in scale, still computationally intensive, typically parameter sweep
*Scalable Web Applications (SWA)
*Use CloudSME Simulation Platform (CSSP) to deploy and manage web services on public Internet.
High Performance Computing
*Example Case: Fluid dynamics simulation
*Benefits*Provides access to various clouds, grids and clusters –
allowing execution of code where it is optimal either from performance or cost perspective
*Challenges*Scales well within a single node by adding more cores.
Communication between instances is a problem if more than one node is needed. CloudSigma are working on a solution in CloudSME project, and there is also supercomputer access from the platform.
High Throughput Computing
*Example Case: Repast *Repast: Recursive Porous Agent Simulation Toolkit
*Requires a model (Java code) and a parameter file
*Benefits
*CloudSME infrastructure provides parameter sweep workflow functionality and possibility to orchestrate jobs to different clouds.
*Challenges:*Each model is different. Some are computationally heavy and a
single run takes a long time to complete, while others complete in fractions of a second.
*Currently each run runs exactly one simulation with a certain model and parameter file. If thousands or tens of thousands of runs are required, this needs to be optimised.
Scalable Web Applications
*Example Case: Outlandish and Tidybooks*Three-tier architecture (proxy server with public IP address,
application server and database server)
*Can run LAMP or MEAN stack – not limited to any particular architecture
*Can be managed from Cloudbroker GUI or Cloudbroker API
*Benefits*CloudSME infrastructure allows access to different clouds. A
Cloudbroker job is a representation of application functionality and data, which fits to Web application architecture neatly.
*Setting up the architecture is basically a workflow
HTTP proxy1 HTTP proxy 2 HTTP proxy N
App server 1 App server 2 App server 3 App server N
DB server 1 DB server 2Replication
InternetPublic IP
Private IP
Adminserver
Scalable Web Applications
Scalable Web Applications
*Challenges:*Needs public IP address. Academic clouds generally do not
provide this.
*Cost and performance optimisation + need to add mechanisms of measuring.
* If the web service traffic volume varies a lot, it would need to be scaled up or down on the fly. Managing this, especially between clouds, would require a portable representation of the application, its configuration and also data. Means of transport exist already in Cloudbroker input-output parameters, but there are problems (potentially massive volumes of data – financial optimisation would need a way of querying this)
*Currently not tested with WS-PGRADE portal – the setup workflow is implemented as a Python script. There is not currently an easy way to run Cloudbroker API commands from workflows. This could be solved by a front-end application deployment but doing this would not be trivial.
