CloudTutorial:AWSEC2andAWSIoT

TAforclassCSE520S,Fall,Aug/30/2017HaoranLi

Agenda

Ø AWS EC2: IaaSq  Project 0: Part 1

Ø AWS IoT: A Platform-as-a-Service exampleq  Project 0: Part 2

2

Pointers

Ø Amazon EC2q  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/

EC2_GetStarted.html

Ø Amazon IoTq  http://docs.aws.amazon.com/iot/latest/developerguide/what-is-

aws-iot.html

Ø  Resource list for course projectsq  http://cps.cse.wustl.edu/index.php/List_of_Projects

Ø Apply for $40 credits for Amazon AWS q  https://aws.amazon.com/education/awseducate/apply/

3

AWSEC2

Today’s“Mainframe”

Service Provider Handles the Hardware.You consume the computational resource.

5 [1]source:hBp://www.alamy.com/stock-photo-icl-2900-series-mainframe-computer-installaLon-and-programmers-at-92958461.html[2]source:hBps://www.stratacore.com/the-advisor/data-center-m-a-recap-2016

Ø Datacenter (2016)Ø Multiuser shares multiple

computers (A datacenter)

Ø Mainframe Computer (1970s)q  Multiuser shares one Computers

InfrastructureasaService(IaaS)Ø  IaaS

q  "physical server box”q  Virtual Machine

•  Memory•  Storage•  CPU•  Network

Ø  Exampleq  AWS EC2q  AWS HPC

Ø  Usecaseq  Build up you VM cluster

6

YouManage

APP

Data

Runtime

Middleware

OS

Virtualization

Server

Storage

Network

ServiceProviderManages

WebService

Tutorial:BigPicture

Ø Create q  Two EC2 Ubuntu Instances

•  T2.micro

q  Ensure the connectivity

Ø Micro benchmarkq  Latency

•  ping

q  Bandwidth•  iperf3

7

AWS

EC2t2.micro EC2t2.micro

EC2:t2.micro

Step1:Create/LaunchInstances

Ø USE AWS Console WebUIØ Create t2.micro Instance

q  1 Xeon vCPU @ 2.5GHz

q  1GB Memoryq  8GB EBS Storage

q  ??? Network

Ø Create Operating Systemq  AMI: Ubuntu 16.04 Server

Ø Get its Public DNS (HostName)

Ø Get the Private Key

8 Source:hBps://www.shareicon.net/tag/network?&&cl=darkslategray&&s=glyph

AccessAWSEC2Service

Ø  Sign Up and Sign In

Ø Choose EC2 Service

9 Source:hBps://aws.amazon.com/console

Launchinstances

Ø  Launch instance in EC2 Dashboard

10

ChooseAMI

Ø Choose Your Image Type (Template Operating System)

11

InstanceType(“Hardware”ofVM)

Ø Choose your Virtual CPU number and Memoryq  T2.micro 1 vCPU + 1GB @ $0.012 per Hour (running state)

12

ATTENTION:{CPU,Mem,Net}andStorage(EBS)arechargedSeparately!

Storage(“HardDrive”)

13

MountPoint Default8GB:$0.10perGB-month

Review&Launch:GeneratePrivateKey

14

PrivateKey:UseittologinyourVM

ChecktheDashboard

Ø Check the State: RunningØ Check the Public DNS

15

Wewillsshintotheinstanceviathisaddress

Step2:SSHintotheVM

Ø Use SSH and Key to login the VM

16

AWSEC2t2.micro EC2t2.microSSH

ChangethePermissionofthePrivateKey

Ø Change it to “owner read only”, i.e. Permission 400q  sudo chmod 400 {Your_Private_Key}.pem

17

LoginIntoit

Ø Host Address: Public DNSØ Username: ubuntu[1]

q  ssh –i {Your_Private_Key} ubuntu@{Pub_DNS_Name}

18 [1]ManageUserAccount:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html[2]UsingpuBytologin:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/puBy.html

WindowsUser:UsePuWyasaSSHclient[2]

Step3:LatencyandBandwidthTest

19

AWS

EC2t2.micro EC2t2.micro

What’stheRTTdelay?What’stheThroughput?

ConnecYvitySeZng

Ø  Security Group SettingsØ  Edit Inbound Rules

20

AllowInboundConnecYons

Ø  Rulesq  Enable All TCP/UDP/ICMP

21

Testthelatency

Ø Using Pingq  Get RTT Min/Avg/Max/Dev

22

PingstaLsLcs

TesttheBandwidth

Ø Using iperf3q  Install: sudo apt-get install iperf3

q  Server: iperf3 –s

q  Client: iperf3 –c {server_ip}

23

1GbpsBandwidth

Step4:RemembertoStop/Terminate

24

Ø All about Moneyq  E.g. t2.micro 1-VCPU + 1GB Mem @ $0.012 per Hour (running

state)[1]

[1]EC2Pricing:hBps://aws.amazon.com/ec2/pricing/on-demand/

RemembertoStopYourVM

Ø  Either shutdown from CLI:q  sudo shutdown –h now

Ø Or Stop your instances from WebUI:

25

Stopped:However,theywillsYllchargeyou

Ø Charge for EBS Storage[1]

Ø  Pricingq  E.g. gp2 volume: $0.10 per GB-month

26

[1]WhychargedbyEBS:hBps://aws.amazon.com/premiumsupport/knowledge-center/ebs-charge-stopped-instance/[2]EBSPricing:hBps://aws.amazon.com/ebs/pricing/

StoppedV.S.TerminatedØ  “Terminate" your instance: PERMANENTLY DELETES the

instance, and free the EBS storage.Ø  “Stop" the instance: Release the run time computational

resource, but keep the disk.Ø Note: Frequently committing your work to your repository.

Ø More info: Read “Differences Between Reboot, Stop, and Terminate” in [1]

27 [1]LifeCycle:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-lifecycle.html[2]EBSPricing:hBps://aws.amazon.com/ebs/pricing/

State Stop TerminateRoot volume (e.g. /dev/sda1)

The volume(EBS) is preserved The volume(EBS) is deleted.

Billing You stop incurring charges for an instance as soon as its state changes to stopping.

You stop incurring charges for an instance as soon as its state changes to shutting-down.

Project0:Part1AWSEC2Ø  Create Your AWS accountØ  Build up two EC2 instances and Setting up the ConnectivityØ  Run ping / iperf3 and get latency / bandwidth statistics

q  1. Repeat this experiment (inbound bandwidth test)q  2. Create Instance in different zone (e.g. us-west-2a/b/c), test the

bandwidth between your PC and the EC2 Instance•  Which Zone shows the best bandwidth / latency?

q  3. Pick up one “Zone”, write a script to repeat test every 2 hours (or more frequent, if you want) in a consecutive 24 hours

•  Plot a figure to show the latency against time•  Plot a figure to show the bandwidth against time

q  4. Run a single test for 10 minute, will the stats change?•  Note: iperf3 –c {server_ip} –t {seconds}

Ø  Email your results (inline, you don’t need to write a report) to lihaoran@email.wustl.edu

28

AWSIoT:APaaSExample

Pla`ormasaService(PaaS)

Ø  PaaSq  You get a framework

q  Host Application

q  Tools

Ø  Exampleq  AWS IoT

Ø Usecaseq  Build up you’re smart A/C

controller

8/30/17 30

YouManage

ServiceProviderManages

APP

Data

Runtime

Middleware

OS

Virtualization

Server

Storage

Network

Internet-of-Things

8/30/17 31

Ø  Things (Devices)q  Many of them

•  Different Types

•  Isolated Systems

q  Data and Command•  Sensing the world

•  Give Response

q  Challenge•  United: Connected + Communication

•  Smart: Data Analytics + Strategy

Source:hBps://aws.amazon.com/iot-plakorm/hBp://www.brain-smart.net/smart-brain-health-blog/page/2/#axzz4W4oSp8a6

SoluYon:AWSIoT

8/30/17 32

United:Connect+CommunicaLon Smart:OtherCloudServiceDataStorageMachineLearning

Source:hBps://aws.amazon.com/iot-plakorm/

Tutorial:HelloAWSIoT!

8/30/17 33Source:hBps://aws.amazon.com/iot-plakorm/

RandomInteger[1,100]

Publish

AmazonSNS

Forward

AWSEC2t2.micro

Step1:CreateaVirtual"Thing”

8/30/17 34

AWSIoT

Virtual“Thing”/Shadow

GetintoAWSManageConsole

Ø Create your own AWS accountØ  Sign In IoT Manage Console

q  https://aws.amazon.com/iot/

8/30/17 35

Createathing

Ø  1. AWS IoT Menuq  Registry

•  Things è Create

Ø  2. Give a name

8/30/17 36

BasicInteract:Publish

Ø Using Embedded MQTT Client to Test

Ø Check the Things Shadow

8/30/17 37

BasicInteract:Subscribe

8/30/17 38

AWSEC2t2.micro

Step2:Connecta“Physical”Device

8/30/17 39

AWSIoT

Virtual“Thing”/Shadow

ABach

CerLficate

Policy Copy

MQTTClient

RandomInteger[1,100]

CreateandgetCerYficates

Ø Create Certificatesq  Security è Certificates è Create

Ø Download Cert Files•  1. public & private key•  2. thing cert

•  3. Root CA for AWS

8/30/17 40

CreatePolicyandaWachittocert

Ø Create Policy

Ø Attach Policy to Certificates

8/30/17 41

ConnectyourDevice

Ø Copy certificates to your EC2 Instanceq  Note: through scp utility

Ø Choose your AWS SDK (support MQTT)q  Node JS

q  Pythonq  Java

Ø  You can also use third party MQTT toolsq  Python (paho-mqtt library)

8/30/17 42

SomeNotes

Ø  1. You will need these certification when setting up the TLS1.2 verification

Ø  2. You will need the endpoint and port (8883) when connect to AWS IoT Gateway

8/30/17 43

More:RuleEngine,LinkwithSNSservices

Ø  Simple Notification Service

8/30/17 44

Publish

AmazonSNS

AWSIoT

Virtual“Thing”/Shadow

Forward

Topic:CSE520_Tutorial

Subscribe

Subscribe

AWSEC2t2.micro

CreateaRuleinAmazonIoT

Ø Add a query to filter your inteseting topic (event)

Ø Add an Action:q  Forward this message to SNSq  Specify Dest ARN

q  Enable Rule

8/30/17 45

NoYficaYononSMS&Email

8/30/17 46

AWSIoT:APaaSExample

8/30/17 47Source:hBps://aws.amazon.com/iot-plakorm/

Publish

AmazonSNS

Forward

AWSEC2t2.micro

AWSEC2t2.micro

Subscribe

AWSIoTasaMQTTMessageBroker

AWSIoTstoragesThingsStates

AWSIoTprovidesthecapabilitytolinkAgainstotherservices

ChooseyourProjecttopic:Pickuptheserviceyouinterest:Tryitfirst.Andcanyoubuildyourownservicebyusingopensourcecomponents?

Project0Part2:AWSIoT

Ø Create a Thing in AWS IoTØ Use Web MQTT tool to update Thing Shadow State

q  i.e. publish to shadow/update topic

Ø  Setup MQTT client in EC2 instanceØ  Subscribe the shadow/update topic in EC2 instance

Ø  Email the screenshots to lihaoran@email.wustl.edu

48

AWSEC2t2.micro

Subscribe

Project0:Part2

Ø  Sample Screenshot:q  1. Updated Shadow State

in WebUIq  2. Printed message on EC2

terminal

49

Project0:

Ø  Part 1 AWS EC2q  Bandwidth Test

Ø  Part 2 AWS IoTq  Subscribe to shadow/update

Ø Due on: 10AM Sep/6 (Wednesday)

50

Pointers

Ø Amazon EC2q  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/

EC2_GetStarted.html

Ø Amazon IoTq  http://docs.aws.amazon.com/iot/latest/developerguide/what-is-

aws-iot.html

Ø  Resource list for course projectsq  http://cps.cse.wustl.edu/index.php/List_of_Projects

Ø Apply for $40 credits for Amazon AWS q  https://aws.amazon.com/education/awseducate/apply/

51

Thanks!

HaoranLi

Aug/30/2017

Project0:

Ø  Part 1 AWS EC2q  Bandwidth Test

Ø  Part 2 AWS IoTq  Subscribe to shadow/update

Ø Due on: 10AM Sep/6 (Wednesday)

53

Project0:Part1AWSEC2Ø  Create Your AWS accountØ  Build up two EC2 instances and Setting up the ConnectivityØ  Run ping / iperf3 and get latency / bandwidth statistics

q  1. Repeat this experiment (inbound bandwidth test)q  2. Create Instance in different zone (e.g. us-west-2a/b/c), test the

bandwidth between your PC and the EC2 Instance•  Which Zone shows the best bandwidth / latency?

q  3. Pick up one “Zone”, write a script to repeat test every 2 hours (or more frequent, if you want) in a consecutive 24 hours

•  Plot a figure to show the latency against time•  Plot a figure to show the bandwidth against time

q  4. Run a single test for 10 minute, will the stats change?•  Note: iperf3 –c {server_ip} –t {seconds}

Ø  Email your results (inline, you don’t need to write a report) to lihaoran@email.wustl.edu

54

Project0Part2:AWSIoT

Ø Create a Thing in AWS IoTØ Use Web MQTT tool to update Thing Shadow State

q  i.e. publish to shadow/update topic

Ø  Setup MQTT client in EC2 instanceØ  Subscribe the shadow/update topic in EC2 instance

Ø  Email the screenshots to lihaoran@email.wustl.edu

55

AWSEC2t2.micro

Subscribe

Project0:Part2

Ø  Sample Screenshot:q  1. Updated Shadow State

in WebUIq  2. Printed message on EC2

terminal

56