[Mar AWS 201] Technical Workshop

amazon web services

AWS 201T Workshop DAY-1 (March 20):

James Saull / Solutions Architect / [email protected] DAY-2 (March 21):

Session-1: 이수형 / Solutions Architect / [email protected] Session-2: 김일호 / Solutions Architect / [email protected] Session-3: 정윤진 / Solutions Architect / [email protected]

Agenda •  House Keeping & Setup •  Introduction to Amazon Web

Services •  Lab: Building a Web

Property on AWS –  Storage: S3, EBS –  Compute –  Content Delivery –  Relational Database –  Dynamo DB

•  Lab: Scalability and Availability –  Snapshots –  Load Balancing –  Auto Scaling –  Security

•  Log Processing Scenario –  Logging to AWS –  Elastic Map Reduce

Setup •  aws.amazon.com •  Be sure you have permissions and are

active •  Latest browser •  Windows download Putty: www.putty.org

Watch out for unexpected Costs When the Technical Workshop comes to an end, to avoid unwanted costs: •  Delete your S3 objects •  Destroy your CloudFront distributions •  Stop or Shut Down your EC2 and RDS instances

The customer is responsible for the resources he’s using. AWS declines any responsibility if the customer forgets to shut down resources.



Property on AWS –  Storage: S3, EBS –  Compute –  Content Delivery –  Relational Database


•  Log Processing Scenario –  Logging to AWS –  Elastic Map Reduce –  Dynamo DB

6

Region GovCloud Region

Global AWS Infrastructure ] [

7

Availability Zone


8

9

Edge Loca8ons


Dallas (2) St.Louis

Miami

Jacksonville Los Angeles (2)

Palo Alto

SeaEle

Ashburn (2)

Newark New York (3)

Dublin London (2) Amsterdam (2)

Stockholm

Frankfurt (2) Paris (2)

Singapore (2)

Hong Kong (2)

Tokyo (2)

Sao Paulo

South Bend

San Jose

Osaka Milan

Sydney

Hayward

Madrid

10

The Amazon Web Services universe [ ]

amazon web services

11

The Amazon Web Services universe [ ]

SDK

API

Web Console

Command Line

12

[ ]

Management + Interface

Cross Service Features

CloudForma8on Simple WorkFlow

CloudWatch

Elas8c BeanStalk

IAM

The Amazon Web Services universe

(some services are omiEed here)

Simple No8fica8on Service

13

[ ]


Platform Building blocks

RDS

EMR

DynamoDB CloudFront SQS

CloudSearch SES




RedshiX

14

Infrastructure Building Blocks

[ ]




Glacier EBS S3 Virtual Private Cloud



EC2

[ ]


RDS CloudSearch SES CloudFront SQS EMR DynamoDB

Elas8c Beanstalk Simple WorkFlow CloudForma8on

CloudWatch IAM

VPC EC2 EBS S3


Infrastructure Building Blocks


Cross Service Features Command Line

Web Console APIs SDK







Labs During this workshop, we will build from

scratch a highly available, redundant, scalable web property on AWS.

Storage Amazon Simple Storage Service (S3)

Amazon S3 Bucket Volume Snapshot Amazon Elastic Block Storage

(EBS)

Amazon Elas8c Block Storage (Amazon EBS)

Our Building Blocks

Compute Our Building Blocks

Amazon Elas8c Compute Cloud (EC2)

Amazon EC2 Instance Instances AMI DB on Instance

Instance with CloudWatch

Elas8c IP

Content Delivery & Database Our Building Blocks

Amazon Database Services RDS/DDB

Amazon RDS MySQL DB Instance DynamoDB

Amazon Cloudfront

Amazon Cloudfront

Download Distribu8on

Edge Loca8on Streaming Distribu8on

•  Object-based storage �(no Filesystem)

•  Easily store/retrieve data •  Durability of 99.999999999%

or 99.99% •  Integrated with other AWS

Services

•  Scalable •  Redundancy is managed

transparently •  File (Object): up to 5 TB each •  HTTP, HTTPS, BitTorrent

protocols

21

Amazon S3 Simple Storage Service

•  Backup •  Archive (Glacier) •  Disaster Recovery (DR) •  Content Storage •  Website

22

Amazon S3 Use cases

Lab Exercise •  Create an S3 Bucket •  Upload and Download a File •  Check out properties and permissions

Questions ?

•  Delivery of content worldwide

•  Static: HTTP / HTTPS •  Streaming: RTMP •  Dynamic Content

•  Origin server: S3, EC2, non-AWS

•  Public/Private Distributions

25

Amazon CloudFront Content Delivery Network

Regions, Availability Zones, Edge Locations

Dallas

St.Louis Miami

Jacksonville

Los Angeles (2)

Palo Alto

SeaEle

Ashburn

Newark New York (2)

Dublin London

Amsterdam

Stockholm

Frankfurt Paris

Singapore

Hong Kong

Tokyo

Sao Paulo

South Bend

San Jose

38 Edge Loca8ons in total (as of Dec 2012)

Sydney

Let’s simplify a bit: we consider only a few of them

Stockholm

Hong Kong

Sao Paulo

San Jose

Your web servers in Singapore

Stockholm

Hong Kong

Sao Paulo

San Jose

Content Delivery Network: How it works

Dynamic pages (PHP, Java) (from web servers) Sta8c content or streaming (with CloudFront)

Your web servers in Singapore

•  Accelerated web content delivery •  Off-load traffic from web servers •  Big spikes in traffic •  Event streaming •  Marketing campaigns

29

Amazon CloudFront Use cases

Lab Exercise •  Create a New Distribution •  Test the Content

Questions ?

•  Linux / Windows virtual machines

•  Elastic (scale in minutes) •  Many Instance types

(micro, small, large...) •  AMI: Amazon Machine

Image •  VPC: Virtual Private Cloud •  Security (Groups, Keys...) •  VM Import •  Auto Scaling, Elastic Load

Balancing

32

Amazon EC2 Elastic Compute Cloud

33

Availability Zone

I see 20 Availability Zones in total (as of Feb 2013)

•  Media •  Hosting •  High Performance

Computing •  Dev & Test �

•  Internal Applications •  Gaming •  ... Everything that

needs computing!

34

EC2 Use cases

Lab Exercise •  Create a new Security Group •  Launch an Amazon EC2 instance (Linux) •  Log in with SSH as ec2-user@ •  Install a web server •  Create a simple web page •  Test it on a browser: it works! •  Create and attach an Elastic IP

Questions ?

•  Block Level Storage for use with EC2

•  Volume: 1 GB to 1 TB •  Raw unformatted block

device •  Local to an Availability Zone

•  Redundant •  Persistent •  Point-in-time snapshots to

Amazon S3 •  Integration with CloudWatch

37

EBS Elastic Block Storage

Demo EBS •  Create a new EBS volume •  Attach it to an EC2 Instance •  Partition it and create a File System •  Mount it; create a file in it •  Create an EBS Snapshot •  Create a new volume from the Snapshot

Questions ?

•  Relational Database “as a Service”

•  Simple to Deploy •  Managed by the AWS team •  MySQL, SQLServer or

Oracle (as of 2012) •  Scalable

•  Optional: automatic Standby Replica

•  Optional: multiple Read-Only copies

•  Easy DB Snapshots and automated backup

40

RDS Relational Database Service

Demo RDS •  Create a DB Instance on RDS (MySQL) •  Enable Multi-AZ Deployment •  Enable one Read Replica •  Optional: connect to the DB Instance

Questions ?






Let’s take our Web Application to the next

level

EBS Elastic Load Balancer

•  Automatically distribute incoming traffic to multiple Amazon EC2 instances (in the same Region).

•  Automatic Health check •  IPv6 support •  Can be integrated with AutoScaling

Questions ?

Snapshots & AMIs •  Copies of EBS Volumes •  Essential to Reusability •  Copy between Regions •  Durability in S3

Lab Exercise •  Duplicate your entire architecture by

making an AMI •  Increase your availability by spreading

your application across availability zones •  Bring up an ELB in front of your website •  Optional – Create a CNAME to the ELB

Questions ?

CloudWatch •  Integrated across AWS platform •  Nothing to install •  Custom Metrics •  Set Alarms •  View graphs and statistics •  Feeds into Autoscaling

AutoScaling •  Auto Up and Auto Down •  Runs on CloudWatch metrics •  Notifications via SNS •  Spot or On-demand •  No additional Fees

AutoScaling •  Launch config: AMI to be used •  Autoscaling group: where/how to launch •  Autoscaling policy: what should AS do •  Autoscaling trigger: what will activate AS

Demo Autoscaling

JMETER

Questions ?

Security •  Security Groups •  Granular tiered secure architecture •  Roles for services •  Best Practices - Bastions

Before we get started on the controls…

•  AWS Reports, Certifications & Accreditations •  SOC 1, Type 2 report •  SOC 2 report •  ISO 27001 •  PCI DSS Level 1 service provider •  FISMA Moderate •  MPAA

•  Look at http://aws.amazon.com/security

A Protected Infrastructure:

IAM VPC ACL ELB AZ SG

TMG RDP WSUS AV

Lab Exercise •  Secure your website by creating a DMZ

between the ELB and your application •  Create a bastion host

Questions ?






Getting your Data into S3

S3 Console Upload

FTP

S3 API

AWS Import / Export

Direct Connect

Tsunami UDP

Storage Gateway

3rd Party Commercial Applications

CloudFront Flume

AWS Data Pipeline

S3 and Big Data •  Why S3? •  Hadoop Overview •  Hadoop on the Cloud •  Hadoop File System

Introducing Apache Hadoop

•  Apache Hadoop •  Software for distributed data analysis •  Map/Reduce framework •  Focus on data

•  But •  Complex •  Hard to setup •  Cap-ex intensive •  Difficult to manage

EMR is Hadoop in the Cloud

Hadoop is an open-‐source framework for parallel processing huge amounts of data on a cluster of machines

What is Amazon Elas3c MapReduce (EMR)?

How does it work?

EMR

EMR Cluster S3

Put the data into S3

Choose: Hadoop distribu8on, # of nodes, types of nodes, custom

configs, Hive/Pig/etc.

Get the output from S3

Launch the cluster using the EMR console, CLI, SDK,

or APIs

You can also store everything in HDFS

011001101

DynamoDB and Big Data •  What is Dynamo? •  Dynamo in Big Data – Volume & Velocity

Questions ?

Watch out for unexpected Costs When the Technical Workshop comes to an end, to avoid unwanted costs: •  Delete your S3 objects •  Destroy your CloudFront distributions •  Stop or Shut Down your EC2 and RDS instances

The customer is responsible for the resources he’s using. AWS declines any responsibility if the customer forgets to shut down resources.

amazon web services

Thank You DAY-1 (March 20):

James Saull / Solutions Architect / [email protected] DAY-2 (March 21):

Session-1: 이수형 / Solutions Architect / [email protected] Session-2: 김일호 / Solutions Architect / [email protected] Session-3: 정윤진 / Solutions Architect / [email protected]

Technology

[Mar AWS 201] Technical Workshop