Evolution of Inter-Cloud Computing to Handle Big ?· Evolution of Inter-Cloud Computing to Handle Big…

  • Published on
    03-Sep-2018

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Evolution of Inter-Cloud Computing to Handle Big Data June 7, 2013 Tomonori Aoyama

    Chair of GICTF Keio University

    5th China Cloud Computing Conference 2013

  • Inter-cloud computing allows on-demand reassignment of cloud resources including compute, storage and network, and transfer of workload through interworking of multi-national and multi-operator cloud systems.

    What is Inter-cloud ?

  • Private Cloud

    General UserSmall Company

    Public Cloud

    Provider

    Single Cloud2010 Hybrid Cloud 2011 Inter Cloud 2015

    Big Company

    Government Organization

    Private Cloud

    General UserSmall Company

    Public Cloud

    Provider

    Big Company

    Government Organization

    Public Cloud

    Public Cloud

    Private Cloud

    Private Cloud

    Network

    Evolution of Cloud Computing

    From Single Cloud to Hybrid Cloud, which is a connection

    between Public and Private Cloud, and then to global

    Inter-Cloud Computing

    T. AOYAMA 3

  • More than Zetta Byte Data are being produced. Enterprise Data, Video Data, Sensor Data,

    Open Data Policy Personal Number Law (Japan) was approved.

    Medical Welfare Cloud for EHR & PHR My hospital anywhere is targeted. (Japan)

    I N TER CLOUD

    Why ? Three Data require Inter-Cloud

  • Annual global IP traffic will pass the zettabyte threshold by the end of 2016,

    Internet Video Will Drive Most Consumer Internet Traffic Through 2016

    Sources: http://www.wired.com/magazine/2010/08/ff_webrip/ Cisco estimates based on CAIDA publications, Andrew Odlyzko

    email

    ftp P2P

    Web

    Video

    2000 199

    0

    2010

    54%

    Internet video

    (86%, the sum of all

    forms of video (TV,

    VoD, Internet, and

    P2P) )

    Sources:http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/VNI_Hyperconnectivity_WP.html

    T. Aoyama 5

    http://www.wired.com/magazine/2010/08/ff_webrip/

  • Provider Y: Community cloud

    Internet

    Provider X

    Avoid additional delay caused by the

    increased load by means of

    alternating data for access point to

    other cloud provider

    overload

    Lease the required

    quantity of

    resources from the

    community cloud

    Service consumer

    Network

    Distribute the load on the own

    servers to outside the own cloud

    system

    Numerou

    s access

    attempts Exchange data such as ID

    information of consumer

    Return the processing to the

    original cloud system when load

    distribution becomes

    unnecessary

    Service 1

    Service 1

    Service 2

    Abrupt

    increase in

    the

    number of

    access

    attempts

    Use case 1: Guaranteeing performance against an abrupt increase of the load

  • DB

    Cloud A

    Cloud D

    Cloud C Service 1

    Disaster

    Service 2

    Service user

    Continue to provide services by recovering each service of Cloud A by using a cloud system other than that of Cloud A

    Select a cloud system to be used for service recovery through negotiation with other cloud systems

    DBs are copied and synchronized in advance

    Remotely activate the application for Service 2 (some quality requirements are satisfied on a best-effort basis)

    Remotely activate the application for Service 1 (to guarantee the required quality)

    DB

    AP

    MW

    Service 1

    Service 1

    Service 2

    Service 1/2

    Service Y

    Service X

    Service 2

    Cloud B

    AP

    MW

    Use case 2 : Disaster Recovery

    7 GICTF

  • 8

    GICTF: A forum for the Inter-cloud Technology

    Applications

    Network

    Servers

    Storage devices

    Cloud system A

    Standard inter-cloud interface

    Cloud system B

    Applications

    Internet, NGN, SDN.

    Application

    Promote R&D on Inter-cloud technologies and the global standardization

    through collaboration between academia, government and industry

    Established on July 17th 2009

    Member: 86 organizations ( NTT, KDDI, NEC, Hitachi, Fujitsu,

    Toshiba, Microsoft, IBM, Oracle,

    Cisco, VMware, IIJ, BIGLOBE, NICT,

    NII, NRI, etc. ), 38 members

    from Univ.

  • Medical

    Network

    Server

    Storage

    Application

    Provisioning

    Monito

    r and C

    ontro

    l

    Cloud System A

    (A) Inter-cloud interworking architecture and standard interfaces

    CDN

    CDN e-Gov

    Cloud System B

    Provisioning M

    onito

    r and C

    ontro

    l

    Legacy System

    (C) Provisioning, monitoring and control across multiple clouds

    Finance

    Medical E-Gov

    (B) SLA assurance for totality of server, storage and NW

    Key Issues for Inter-Cloud Service Federation

    Our target is an environment in which there are more than 10 distributed clouds, each

    consisting of several hundreds of applications and several thousands of virtual servers.

    Finance

  • 10

    Functional Structure of Inter-cloud systems

    Intranet

    Servers and storage

    Monitoring and control

    Network Intranet

    Servers and storage

    Monitoring

    and control

    Provisioning Service provision requirements

    Monitoring information

    Resource plan

    Search for available resources Policy negotiation

    Network configuration / optimization

    Resource reservation request Flexible resource allocation,

    reconfiguration

    Provisioning

    Inter-cloud monitoring and negotiation control

    Dynamic reconfiguration of the server environment

    Dynamic reconfiguration of the network environment

    GICTF

  • The Worlds First Public International Standards Setting on Cloud Network

    7 frameworks of public international standards are

    almost completed

    I. Cloud Computing Definition Vocabulary definition, etc.

    II. Requirements Service requirements Use cases, etc.

    III. Cloud Network Network structures Network requirements, etc.

    IV. Cloud Computing Infrastructure Structures of infrastructure Infrastructure requirements, etc.

    V. IP Resources Management Ability of resources management, etc.

    VI. Inter-Cloud Method of inter-cloud Security management, etc.

    VII. Cloud Computing DaaS Requirements and architecture of

    Desktop as a Service

    Major Deliverables

    I. Use case and functional requirements

    for Inter-Cloud Computing E/Aug 2010

    II. Inter-Cloud interface specification on

    protocols J/Dec 2011, E/Apr 2012

    III. Inter-Cloud interface specification on

    resources data model for network

    control J/Dec 2011, E/Apr 2012

    IV. Network and technical requirements in

    support of Inter-Cloud J/Dec 2011

    E/Apr 2012

    Proposal based on

    the results

    http://www.gictf.jp/index_e.html 11

  • Framework of Inter-cloud Interface Specification

    The interface between two cloud systems administered by different operators

    Three layer modeling Lower layer protocols assumed as some XML message exchange,

    e.g., REST or SOAP

    Inter-cloud protocol: Information flows, message semantics with associated parameters specified

    Data models for network resources specified

    Data models for computing and storage referenced to other SDOs specifications

    Inter-Cloud Protocol

    Lower Layer Protocols (e.g., REST or SOAP)

    Data Center Resource Data Model

    (Computing and Storage)

    Network Resource Data Model (Network)

    Inte

    r-C

    lou

    d IF

    Cloud System

    Data Center Resource

    Network Resource

    Cloud System

    Data Center Resource

    Network Resource

    Inter-Cloud Protocol

  • PaaS API

    SaaS API

    IaaS API

    Implementation

    BSS/ OSS

    Security

    Open Cloud Manifesto

    CSA

    ENISA

    DMTF

    OASIS

    ETSI

    Inter-Cloud

    Other Cloud

    TM forum

    ISO/IEC JTC1

    ITU-T IEEE

    DMTF

    ITU-T

    OCC

    Taxonomy, Use cases, SLA, Requirements

    NIST

    NIST

    Architecture ITU-T NIST

    SDO coordination OMG IEEE

    ITU-T

    OCC

    SIENA OGC CBA

    KCSA CCF

    GICTF

    Japan Asia EU USA Global

    GICTF

    Cloud

    OGF

    SNIA

    DMTF

    ATIS CSF

    OpenStack

    Users/

    Operators

    Cloud Mng

    API

    ISO/IEC

    JTC1

    Cloud Standardization Status

  • SLG Special Liaison

    Group member

    OGF OMG Cloud Interoperability

    Roadmaps Session

    ITU-T FG Cloud

    CC Standard Study Group

    15

    NIST CC forum

    OGF-

    Europe

    DMTF SNIA

    OpenStack

    Open Source

    Community

    DE-facto Standard

    US Government

    De-jure Standard

    Cloud Business Google, Salesforce, Amazon, etc.

    ISO/IEC JTC SC38

    15

    Collaboration among GICTF and various SDOs

    T. AOYAMA

  • Inter-Cloud System R&D* (2009-2012)

    Cloud Resource

    Federation and

    Reconfiguration

    Cloud Resource

    Provisioning

    Real-time Sensor

    Node

    Dynamically

    Reconfigurable

    Network

    *Funded by Ministry of Internal affairs and Communications of Japan

    Physical World

    WAN edge

    Network

    Cloud System A

    Cloud System B

    Server

    Storage

    AP AP AP Application

    Wide Area Network

  • JGN-X Tohoku-1

    JGN-X Tokyo-1

    Base(1) Tokyo

    Base(2) Tohoku

    JGN-X Kyushu-4

    Exclusive Line100M

    JGN-X Kyushu-1

    Inter-cloud technology evaluation will start in 2012 on the national-scale test bed where three cloud data centers in Tokyo , Tohoku, and Kyushu will be connected with each other via JGN-X.

    Inter-cloud test bed planned in 2012

    Base(3) Kyushu

    Cloud C

    Cloud A

    Cloud B

  • Cloud Computing Testbed

    New Generation Network Testbed JGN-X

    R&D Testbed domestic companies and universities

    Wireless Networking

    Technologies Optical Networking

    Technologies

    Inter-Cloud

    Technology

    Advanced Application

    Technology

    companies &

    universities

    International

    cooperation

    IEEE Cloud Testbed

    Testbed for Cloud Networking will be an essential part of Inter-Cloud technology developments

    Coordinated developments through open test environments will be mutually profitable for both cloud computing and networking like New Generation Network

    International cooperation with various organizations in the industry for the efficient development of Inter-Cloud standards

    Promotion of cloud networking technologies and gaining the momentum

    18

    Virtualized Network

    Necessity of Inter-Cloud Testbed

    Cloud Reserch Institution

    Cloud Reserch Institution

  • Emerging Areas of Study in Future Network Research

    Network Virtualization (NV) Software Defined Network (SDN) Network Functions Virtualization (NFV) And more?

    19

    Introducing programmability into networking to flexibly and dynamically resolve constantly arising contemporary issues.

    The University of Tokyo Confidential

  • Publication at SDN and OpenFlow World Congress in October 22-24, 2012

    SDN (Software Defined Network) and NFV ( Network Functions Virtualization)

  • SDN Standardization

  • 22

    DPN: Deeply Programmable Network

    Application Programmability

    Control-Plane Programmability Interfaces Functions

    Route Control Access Control Network Management

    Data-Plane Programmability Interfaces Functions

    Packet Data Processing Network Appliances (DPI, BRAS, EPC) In-Network processing (Cache, Transcode) Wide-Area generic processing

    Handling New Protocols IPvN (N>6), New Layer2, CCN

    The University of Tokyo Confidential

    Control-Plane Elements

    Network Applications

    Interfaces

    Data-Plane Elements

    e.g, OpenFlow Controller

    e.g, OpenFlow Switches

    e.g, Southbound API

    Interfaces

    e.g, Northbound API

    Applications

    Control Plane

    Data Plane

    ONF OpenFlow

    SDN

    OpenDaylight

    NFV

    DPN

    Deeply Programmable

    Network (DPN)

  • Internet

    Architecture of VNode Prototype

    DC

    R

    P

    R

    P AGW

    AGW Users

    Users

    Internet Programmer

    R

    P

    Redirector (R) Programmer (P)

    Slice Traffic

    Slow Path

    Fast Path

    VNode Manager

    SP

    SP

    SP

    SP

    SP

    FP

    FP

    FP

    FP

    FP

    Forwarding Engines

    Data Plane

    Tunnel Redirection

    Redirector Manager Programmer Manager

    Redirector

    Access Gateway (AGW)

    VNode

    Domain Controller

    23

    Production Router (10Gbps)

    VNode Architecture

    User Opt-In

    User Opt-In

    Form he Skelton of Virtualized Networks

    Programmability in Virtualized Networks

    T. Aoyama

  • JGN-X Network Figure (from 2011/4)

    USA

    Thailand

    Singapore

    Hong Kong

    40G 10G

    DF

    VN Virtual Node

    OF Openflow

    DCN DCN

    Ex

    Realizing multiple New Generation Network Planes on a virtual JGN-X network. Also made available is to interconnect with other Testbeds.

    VLAN Testbed Network

    VN

    VN

    VN

    VN

    OF

    OF OF OF

    OF OF

    DCN

    DCN DCN

    V-node Plane

    Openflow Plane

    DCN Plane

    Optical Testbed Wireless Testbed

    1G Virtual Storage

    StarBED

    To Korea

    Optical Testbed

    Koganei-Otemachi OtemachiOtemachi

    Bandwidth International

    Japan- USA 10G Japan - Korea 10G Japan Hong Kong 10G Hong Kong - Singapore 2.4G Singapore - Thailand 622Mbps

    Wireless Testbed

    Fukuoka

    Hiroshima

    Okayama

    Osaka

    Nagoya

    Kanazawa

    Sendai

    Tokyo NICT Koganei

    40Gx2

    40G 40G

    10G 10G

    10G

    10G

    VN

    VN

    VN

    OF

    OF

    OF

    OF

    OF

    OF DCN

    DCN

    DCN

    Sapporo

    10G

    10Gx2 10Gx2

    10G

    1G

    OF

    JGN-X Planned POP

    Kashima Hakusan Yokosuka Kobe Hokuriku

    Keihannna Tsukuba Kyusyu Univ. Kyusyu Inst. Tech Tohoku Univ.

    Tokyo Univ. Osaka Univ.

    DCN

    LA, USA

    T. Aoyama 24

  • Social Cloud Infrastructure w/ SDN

    Security and Data Portability in Virtualization & SDN env.

  • Integration of Inter Cloud and

    Network Virtualization

    Versatile Information Services

    Total Handling and Control Mechanism

    T. Aoyama 26

    Virtual Network Resource Virtual Server Resource Virtual Storage Resource

  • Technical Issues on Network Virtualization for Intercloud Cooperation

    Resource Abstraction Hiding details of resources Name resolving of resources

    Resource Isolation

    Isolation among private virtual networks QoS control to guarantee bandwidth or latency

    Programmability

    Enabling creation of network functions based on new ideas (cache/encryption)

    Elasticity Providing network resources on-demand Cloud networking among clouds

    Acknowledgement & Credits: The white paper of Network virtualization study group in 2010.

  • Summary

    Inter-Cloud is a sequent trend toward future ICT systems

    GICTF objectives

    Promoting the open interface between cloud systems

    Contributing the Inter-Cloud standardization based on technical feasibility studies

    GICTF deliverables and contributions to SDOs

    Use case and functional requirements for Inter-Cloud Computing

    Technical Requirements for Supporting the Inter-cloud Networking

    Inter-Cloud Interface Specification Draft

    Cloud Resource Data Model

    Inter-Cloud Protocol

    Collaboration with DMTF and SNIA

    Support for joint projects and international intercloud testbeds

  • Global Inter-Cloud Technology Forum

    GICTF http://www.gictf.jp/

Recommended

View more >