Upload
emmeline-payne
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
Sponsored by the National Science Foundation
GENI and Cloud Computing
Niky RIga
GENI Project Office
Sponsored by the National Science Foundation 2IC2E‘14 – 10 March 2014 www.geni.net
Cloud, HPC, and Distributed Computing
• Why Cloud / Distributed Computing– Performance– Locality– Dynamic resource allocation– Reliability
Slide by Paul Ruth
Image by: http://www.iec.ch/etech/2012/etech_0512/pic_tech/tech-2_network_lrg.jpg
Sponsored by the National Science Foundation
GENI Ricks and SDN
SDN and Clouds
GENI and other Cloud Providers
International Collaborations
GENI and Cloud Research
Sponsored by the National Science Foundation 4IC2E‘14 – 10 March 2014 www.geni.net
Cloud, HPC, and Distributed Computing
• Combine multiple resources– Compute, Network, Storage
• Need to distribute tasks– PBS/Torque, MPI, HTCondor, Hadoop, Slirm – Tightly or loosely coupled
• Cloud (GENI)– Allocate compute, network, and storage– GENI can allocate WAN resources
Slide by Paul Ruth
Sponsored by the National Science Foundation 5IC2E‘14 – 10 March 2014 www.geni.net
Cloud Providers
Observatory
Wind tunnel
Workflow
Slide by Paul Ruth
Cloud, HPC, and Distributed Computing
Sponsored by the National Science Foundation 6IC2E‘14 – 10 March 2014 www.geni.net
GENI Racks as Clouds
• A GENI Rack– Raw PCs– 100s of VMs– Separates control and data
planes– Flexibility on host OSes– Hosts isolated experiments
Racks can act as cloud providers, or to deploy clouds
Sponsored by the National Science Foundation 7IC2E‘14 – 10 March 2014 www.geni.net
GENI Racks
InstaGENI ExoGENI *
Nodes per rack 5 10workers + 1head
Cores per rack 60 120
Network interfaces 4x 1Gbit 2x 10Gbit with SR-IOV
Storage 1 TB local 150GB+500 GB local + 6 TB SAN
Switches HP ProCurve 5406 (Vlan-based OpenFlow)
IBM G8264R (Port-based OpenFlow)
* Listed are the specs for ExoGENI IBM-based racks.
InstaGENI: Less powerful more placesExoGENI: More powerful fewer placesEach rack has two raw PCs
Sponsored by the National Science Foundation 8IC2E‘14 – 10 March 2014 www.geni.net
High Throughput Computational Genomics
MotifNetwork– Jeffery L. Tilson (RENCI)
– Identification of functional
domains and the identification of
conserved functional
relationships across large
numbers of genomes.
– Workflow ensemble
(Pegasus/HTCondor)
– Task data sets on the order of
50-100 GB.
– Implicit iteration
– Scales to 1000s of tasksSlide by Paul Ruth
Sponsored by the National Science Foundation 9IC2E‘14 – 10 March 2014 www.geni.net
Urgent Computing
ADCIRC– Brian Blanton (RENCI)
– Storm surge and tide
model
– Finite element model
– MPI tightly coupled
– Approved by FEMA for
computing storm surge
flood hazard simulations
– Used for Digital Flood
Insurance Rate Maps
(DFIRMs)
– Scales to 10000+ MPI
processesSlide by Paul Ruth
Sponsored by the National Science Foundation
GENI and Cloud Research
SDN and Clouds
GENI and other Cloud Providers
International Collaborations
SDN and Clouds
Sponsored by the National Science Foundation 11IC2E‘14 – 10 March 2014 www.geni.net
Cloud Research and Networking
Cloud deployments rise new network challenges– Within one Datacenter– In distributed Datacenters
Google’s SDN WAN
SDN enables easy, customizable network innovation
Sponsored by the National Science Foundation 12IC2E‘14 – 10 March 2014 www.geni.net
GENI OpenFlow Deployment
OpenFlow-enabled hardware switch at:– Each GENI Rack– Backbone and regional networks (between racks)
Sponsored by the National Science Foundation 13IC2E‘14 – 10 March 2014 www.geni.net
GENI Experiment: Virtual Desktop Clouds
Sponsored by the National Science Foundation 14IC2E‘14 – 10 March 2014 www.geni.net
Prasad Calyam,U. of Missouri
Program realtime load-balancing functionality
deep into the network to improve QoE
GENI Experiment: Virtual Desktop Clouds
Sponsored by the National Science Foundation
GENI and Cloud Research
SDN and Clouds
GENI and other Testbeds
International Collaborations
GENI and other Cloud Providers
Sponsored by the National Science Foundation 16IC2E‘14 – 10 March 2014 www.geni.net
Interconnecting
grid
internet
Sponsored by the National Science Foundation 17IC2E‘14 – 10 March 2014 www.geni.net
Interoperate
internet
e.g. LabWiki
GENI AM API
GENI Tools
Sponsored by the National Science Foundation 18IC2E‘14 – 10 March 2014 www.geni.net
NowCast System
Slide by Mike Zink, UMass Amherst
Short-term weather prediction (1-15 mins)
Forecasts as we know them:• Data from many
sensors: Radar, satellite, balloons,
• Usually for large • regions• Takes super
computers to calculate
Sponsored by the National Science Foundation 19IC2E‘14 – 10 March 2014 www.geni.net
Multi-radar NetCDF Data
Nowcast Processing
1. Spin up system in Amazon commercial EC2 and S3 services on demand
“raw” live data
Generate “raw” live dataViSE/CASA radar nodes
http://stb.ece.uprm.edu/current.jsp
ViSE views steerable radars as shared, virtualized resourceshttp://geni.cs.umass.edu/vise
Nowcast images for display
Weather NowCastingUniversity of Massachusetts
David Irwin et al
Create and run realtime “weather service on demand”as storms turn life-threatening
Sponsored by the National Science Foundation
GENI and Cloud Research
SDN and Clouds
GENI and other Cloud Providers
International Collaborations International Collaborations
Sponsored by the National Science Foundation 21IC2E‘14 – 10 March 2014 www.geni.net
International Collaboration
Modified slide from: http://groups.geni.net/geni/attachment/wiki/GEC18Agenda/MonPlenary/
GEC18_brecht_vermeulen_International_Federation.pdf
Efforts to provide L2, highspeed, SDN network between International testbeds
Sponsored by the National Science Foundation 22IC2E‘14 – 10 March 2014 www.geni.net
GENI Experiments: TransGEO
The Intercloud will be about the seamless movement of computation
Design and development of protocols, security procedures, architectures,
economic models that permit computation to move to data, rather than the other way around
Sponsored by the National Science Foundation 23IC2E‘14 – 10 March 2014 www.geni.net
GENI Experiments: TransGEO
Compute how “Green” a City using satellite images
Transcontinental Federation of Cloud Systems with Private 10Gb/s transcontinental network linking sites
Perform distributed query on TransCloud
Sponsored by the National Science Foundation 24IC2E‘14 – 10 March 2014 www.geni.net
QUESTIONS?