Basics of Cloud Computing - Kursused ?· Basics of Cloud Computing ... •Introduction to distributed…

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Basics of Cloud Computing

    MTAT.08.027 Basics of Cloud Computing (3 ECTS) MTAT.08.027 Basics of Cloud Computing (3 ECTS)

    MTAT.08.011 Basics of Grid and Cloud Computing

    Satish Sriramasatish.srirama@ut.ee

  • Course Purpose

    Introduce cloud computing concepts

    Introduce cloud providers

    Introduction to distributed computing algorithms like MapReducealgorithms like MapReduce

    Glance of research at Mobile Cloud Lab in cloud computing domain

    http://courses.cs.ut.ee/2013/cloud/

    4/3/2013 Satish Srirama 2/33

  • Questions

    Is everyone comfortable with data structures?

    How comfortable you are with algorithms?

    How comfortable you are with programming? Java ? Java ?

    External APIs?

    Python I assume you are

    Web programming

    Were you able to submit all exercises in Grid part?

    4/3/2013 Satish Srirama 3/33

  • Outline

    Cloud computing

    Cloud providers

    MapReduce

    MapReduce in different domains MapReduce in different domains

    Large scale data processing on cloud

    4/3/2013 Satish Srirama 4/33

  • Grading

    Written exam 50%

    Labs 45% 6 lab exercises

    Active participation in the lectures (Max 5%) Active participation in the lectures (Max 5%)

    To pass the course You need to score at least 50% in each of the

    subsections

    You need to score at least 50% in the total

    4/3/2013 Satish Srirama 5/33

  • Course schedule

    03.04 Basics of Cloud computing

    10.04 Cloud Providers

    17.04 MapReduce

    24.04 MapReduce algorithms 24.04 MapReduce algorithms

    01.05 No lecture May day

    08.05 MapReduce in Information Retrieval

    15.05 Cloud scale distributed data storage

    22.05 No lecture I will be on a project meeting in Paris

    4/3/2013 Satish Srirama 6/33

  • Course schedule - continued

    Labs

    03-09.04 Starting with a cloud

    10-16.04 Working with SciCloud

    17-23.04 MapReduce - Basics17-23.04 MapReduce - Basics

    24-30.04 Data analysis with MapReduce

    08-14.05 MapReduce in information retrieval

    15-21.05 NoSQL

    4/3/2013 Satish Srirama 7/33

  • Course schedule - continued

    28.05 - Examination 1

    29.05 - Examination 2

    Students who are defending their theses should pass the exam in the first attemptshould pass the exam in the first attempt

    Examination for second attempt students 12th

    June

    4/3/2013 Satish Srirama 8/33

  • CLOUD COMPUTING

    Lecture 1

    4/3/2013 Satish Srirama 9

  • WHAT IS CLOUD COMPUTING?

    Its nothing new...weve redefined Cloud Computing to include everything that we already do... I dont understand what we would do differently ... other than change the wording of some of our ads.

    Its a trapIts worse than stupidity: its marketing hype. Somebody is saying this is inevitableand whenever you hear that, its very likely to be a set of businesses campaigning to make it true.

    WHAT IS CLOUD COMPUTING?

    No consistent answer!

    Everyone thinks it is something else

    the wording of some of our ads.

    Larry Ellison, CEO, Oracle (Wall

    Street Journal, Sept. 26, 2008)

    campaigning to make it true.

    Richard Stallman, Founder, Free Software Foundation (The Guardian, Sept. 29, 2008)

    Slide taken from Professor Anthony D. Josephs lecture at RWTH Aachen4/3/2013 Satish Srirama 10

  • What is Cloud Computing?

    Computing as a utility Utility services e.g. water, electricity, gas etc Consumers pay based on their usage

    Cloud Computing characteristics Illusion of infinite resources Illusion of infinite resources No up-front cost Fine-grained billing (e.g. hourly)

    Gartner: Cloud computing is a style of computing where massively scalable IT-related capabilities are provided as a service across the Internet to multiple external customers

    Satish Srirama4/3/2013 11/33

  • Timeline

    4/3/2013 Satish Srirama 12/33

  • How Cloud & Grid are related

    Share a lot commonality Intention, architecture and technology

    Differences Programming model, business model, compute

    model, applications, and Virtualization.model, applications, and Virtualization.

    The problems are mostly the same Manage large facilities; Define methods by which consumers discover, request

    and use resources provided by the central facilities;

    Implement the often highly parallel computations that execute on those resources.

    4/3/2013 Satish Srirama 13/33

  • Virtualization

    Virtualization techniques are the basis of the cloud computing

    Virtualization technologies partition hardware and thus provide flexible and scalable computing platforms

    Virtual machine techniques VMware and Xen OS

    App App App

    OS OS VMware and Xen OpenNebula Amazon EC2

    Grid do not rely on virtualization as much as Clouds do, each

    individual organization maintain full control of their resources

    For cloud, virtualization is almost an indispensable ingredient

    Hardware

    OS

    Hypervisor

    OS OS

    Virtualized Stack

    4/3/2013 Satish Srirama 14/33

  • 4/3/2013 Satish Srirama

    Enabling Grids for E-sciencE - EGEE

    15/33

  • Clouds - Why Now (not then)?

    Commoditization of HW x86 as universal ISA, plus fast virtualization Bet: Can statistically multiplex multiple instances onto a single box

    without interference between instances

    Web 2.0 Standard software stack, largely open source (LAMP) Standard software stack, largely open source (LAMP) Asynchronous JavaScript and XML (AJAX)

    Novel economic model: fine grain billing Earlier examples: Sun, Intel Computing Serviceslonger commitment,

    more $$$/hour

    Infrastructure software: e.g. Google FileSystem, HDFS Operational expertise: failover, DDoS, firewalls... More pervasive broadband Internet

    4/3/2013 Satish Srirama

    ISA - Instruction Set Architecture

    LAMP Linux, Apache Http Server, MySQL, PHP, perl or python

    16/33

  • Cloud Computing - Services Software as a Service SaaS

    A way to access applications hosted on the web through your web browser

    Platform as a Service PaaS A pay-as-you-go model for IT

    resources accessed over the

    SaaS

    Facebook, Flikr, Myspace.com,

    Google maps API, Gmail

    Level of

    Abstraction

    resources accessed over the Internet

    Infrastructure as a Service IaaS Use of commodity computers,

    distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data

    VirtualizationSatish Srirama

    PaaS

    Google App Engine, Force.com, Hadoop, Azure,

    Amazon S3, etc

    IaaSAmazon EC2, SciCloud, Joyent Accelerators, Nirvanix Storage Delivery Network, etc.

    4/3/2013 17/33

  • Cloud Computing - Themes

    Massively scalable On-demand & dynamic Only use what you need - Elastic

    No upfront commitments, use on short term basis

    Accessible via Internet, location independent Accessible via Internet, location independent Transparent

    Complexity concealed from users, virtualized, abstracted

    Service oriented Easy to use SLAs

    Satish Srirama

    SLA Service Level Agreement

    4/3/2013 18/33

  • Cloud Models

    Internal (private) cloud Cloud with in an organization

    Community cloud Cloud infrastructure jointly

    owned by several organizations

    Public cloud Public cloud Cloud infrastructure owned by

    an organization, provided to general public as service

    Hybrid cloud Composition of two or more

    cloud models

    Satish Srirama4/3/2013 19/33

  • Short Term Implications of Clouds

    Startups and prototyping

    Minimize infrastructure risk

    Lower cost of entry

    Batch jobs Batch jobs

    One-off tasks

    Washington post, NY Times

    Cost associatively for scientific applications

    Research at scale

    4/3/2013 Satish Srirama 20/33

  • Cloud Application Demand

    Many cloud applications have cyclical demand curves Daily, weekly, monthly,

    Demand

    Re

    so

    urc

    es

    Workload spikes are more frequent and significant When some event happens like a pop star has expired:

    More # tweets, Wikipedia traffic increases 22% of tweets, 20% of Wikipedia traffic when Michael Jackson expired in 2009

    Google thought they are under attack

    Demand

    Time

    Re

    so

    urc

    es

    4/3/2013 Satish Srirama 21/33

  • Economics of Cloud Users

    Pay by use instead of provisioning for peak

    Capacity

    Re

    so

    urc

    es

    Re

    so

    urc

    es

    Unused resources

    Static data center Data center in the cloud

    Demand

    Time

    Re

    so

    urc

    es

    Demand

    Capacity

    TimeR

    eso

    urc

    es

    4/3/2013 Satish Srirama 22/33

  • Unused resources

    Economics of Cloud Users - continued

    Risk of over-provisioning: underutilization

    Huge sunk cost in infrastructureCapacity

    Static data center

    Demand

    Time

    Re

    so

    urc

    es

    4/3/2013 Satish Srirama 23/33

  • Economics of Cloud Users - continued

    Heavy penalty for under-provisioning

    Re

    so

    urc

    es

    Demand

    Capacity

    Re

    so

    urc

    es

    Lost revenue

    Lost users

    Re

    so

    urc

    es

    Demand

    Capacity

    Time (days)1 2 3

    Demand

    Time (days)1 2 3