Upload
eubrasilcloudforum-
View
68
Download
0
Embed Size (px)
Citation preview
1º WCN - Workshop on Cloud Networks
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPC Cloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. NavauxGPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento Paralelo e Distribuído
1º WCN - Workshop on Cloud Networks July 06, 2016 2
High performance applicationsHigh performance applications
Applications that require a lot of computing Fluid dynamics
Computational biology
Seismic models
Climatological models BRAMS and OLAM
HPC is traditionally obtained with the use of supercomputers, clusters and grids
3
Fluids DinamicFluids Dinamic
Sterling e Brandt 2012
1º WCN - Workshop on Cloud Networks July 06, 2016 4
1997 2007DoE
Combustion EngineCombustion Engine
5
Molecular DinamicMolecular Dinamic
Sterling e Brandt 2012
6
HuricanesHuricanesDisasters PreventionDisasters Prevention
Click to edit the outline text formatSecond Outline Level
Third Outline LevelFourth Outline Level
Fifth Outline LevelSixth Outline Level
Seventh Outline LevelAzure for Research Workshop
Click to edit the outline text format Second Outline Level
Third Outline LevelFourth Outline Level
Fifth Outline LevelSixth Outline Level
Seventh Outline LevelMay 25-26, 2015
“Numerous reports have documented the technical challenges and nonviability of simply scaling Numerous reports have documented the technical challenges and nonviability of simply scaling existing computer designs to reach exascale,” said Dongarra. “Drawing from these reports and existing computer designs to reach exascale,” said Dongarra. “Drawing from these reports and experience, our subcommittee has identified the top 10 computing technology advancements that experience, our subcommittee has identified the top 10 computing technology advancements that are critical to making a productive, economically viable exascale system.” (Dongarra 2014)are critical to making a productive, economically viable exascale system.” (Dongarra 2014)
Top 10 avancemments:Top 10 avancemments:• Energy efficiency,Energy efficiency,• Interconnect technology,Interconnect technology,• Memory Technology,Memory Technology,• Scalable System Software,Scalable System Software,• Programming systemsProgramming systems
Data managementData management
Exascale Algorithms,Exascale Algorithms,
Algorithms for discovery, design, Algorithms for discovery, design, and decision,and decision,
Resilience and correctness,Resilience and correctness,
Scientic productivity.Scientic productivity.
1º WCN - Workshop on Cloud Networks July 06, 2016 8
Supercomputers TitanSupercomputers Titan
1º WCN - Workshop on Cloud Networks July 06, 2016 9
Questions ?Questions ?
Can we process applications that need high performance processing on cloud?
E Science
Did the performance suitable?
1º WCN - Workshop on Cloud Networks July 06, 2016 10
Cluster vs CloudCluster vs Cloud
Traditional hardware Cluster Hardware cost
Hardware maintenance
People for maintenance
Cloud Computing Without acquisition and maintenance cost
Minimum staff
Pay per use
1º WCN - Workshop on Cloud Networks July 06, 2016 11
Challenges in applications migration to cloudChallenges in applications migration to cloud
Portability: Lack of application and code documentation, slow down migration. Scientific applications have a very specific problem domains. Migration requires a high level knowlegde of application . Applications developed without version control or comments creates a bottleneck in migration (legacy code)
Costs Porting application code to PaaS. Number of processing instances Acceptable time to solution
1º WCN - Workshop on Cloud Networks July 06, 2016 12
Challenges in applications migration to cloudChallenges in applications migration to cloud
Resources Network
Scaling, instance migration, resilience.
Storage Local and distributed file systems.
Processing Number of instances vs number of processors.
Applications CPU-bound, IO-bound, Network-bound.
1º WCN - Workshop on Cloud Networks July 06, 2016 13
Case: Weather ForecastingCase: Weather Forecasting
Numerical Weather Forecast
Used to accurately predict atmosphericbehavior for a future time period.Represents the state of the atmosphereat a xed time (temperature,wind components, etc) over a discretizeddomain.The results are denominated weatheror climate forecasts, a few days forweather, months for climate.
1º WCN - Workshop on Cloud Networks July 06, 2016 14
1º WCN - Workshop on Cloud Networks July 06, 2016 15
BRAMS Climate ModelBRAMS Climate Model
1º WCN - Workshop on Cloud Networks July 06, 2016 16
BRAMS ModelBRAMS Model
Brazilian Regional Atmospheric Modeling System
Mesoscale model based on
RAMS, developed by the Atmospheric
Science Department at the CSU,
adapted to Brasil Climate.
The main objective of BRAMS
is to provide a single model
to Brazilian Regional Weather
Centers. Each center execute
mesoscale forecasts in their geographical area.
1º WCN - Workshop on Cloud Networks July 06, 2016 17
BRAMS is a free BRAMS is a free softwaresoftware
1º WCN - Workshop on Cloud Networks July 06, 2016 18
BRAMS Ensemble ForecastBRAMS Ensemble Forecast
Ensemble Forecast
The impact of initial condition error
propagation in the forecast integration
time can be alleviated by
the ensemble forecast technique.
A set of independent integrations
are done, with small variations
in initial conditions among each
other, resulting in distinct forecasts.
1º WCN - Workshop on Cloud Networks July 06, 2016 19
BRAMS - ECMWF MethodBRAMS - ECMWF Method 12 executions per year (for 10 years)
1º WCN - Workshop on Cloud Networks July 06, 2016 20
BRAMS Result example: Rain PrecipitationBRAMS Result example: Rain Precipitation
1º WCN - Workshop on Cloud Networks July 06, 2016 21
BRAMS in Azure
Affinity groups.Wrapped in a Cloud ServiceProcessing VMs Created on demandFrontend is the only fixed instanceBilled by forecast runService is fully automatedData stored in Azure file service
1º WCN - Workshop on Cloud Networks July 06, 2016 22
Approach:Using Azure File Srvice:
Pros:•Legacy Compatibility•Using SMB•Highly scalable, replication•Easy to setup
Cons:•No file permissions•Throughput limited•(Preview)•Metadata is lost
1º WCN - Workshop on Cloud Networks July 06, 2016 23
Shared Usage
Multiples instances of BRAMS in Cloud
72 hours
4 regions of South America
Sharing input data
1º WCN - Workshop on Cloud Networks July 06, 2016 25
Conclusions of HPC in Cloud: Climate caseConclusions of HPC in Cloud: Climate case
The main objective of the work was to migrate the Brazilian Regional
Atmospheric Modeling System (BRAMS) to a new platform that
uses the advantages provided by a cloud computing infrastructure
and evaluate the challenges of this procedure.
The goals of the work was:
Migrate Brams to Azure.
Measure the performance of the migrated version.
Analyse the cost of execution to achieve useful results.
Study the viability of usage of the implemented model by other weather centers.
1º WCN - Workshop on Cloud Networks July 06, 2016 26
Cloud and SupercomputingCloud and Supercomputing
Cloud Companies are installing Supercomputing
facilities to improve their processing power in
executing in cloud.
1º WCN - Workshop on Cloud Networks July 06, 2016 27
Conclusion of HPC in Cloud and with Supercomputing Conclusion of HPC in Cloud and with Supercomputing facilitiesfacilities
- we can use HPC applications in cloud
-.the performance is not the same
- but cloud are changing including supercomputers features
- cloud hadd VM facilities
BUT
- interconnection is one of the challenges
- I/O is another challenge
1º WCN - Workshop on Cloud Networks
Thanks!Thanks!
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPC Cloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. NavauxGPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento Paralelo e Distribuído