View
235
Download
4
Category
Preview:
Citation preview
CloudTutorial:AWSEC2andAWSIoT
TAforclassCSE520S,Fall,Aug/30/2017HaoranLi
Agenda
Ø AWS EC2: IaaSq Project 0: Part 1
Ø AWS IoT: A Platform-as-a-Service exampleq Project 0: Part 2
2
Pointers
Ø Amazon EC2q http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
EC2_GetStarted.html
Ø Amazon IoTq http://docs.aws.amazon.com/iot/latest/developerguide/what-is-
aws-iot.html
Ø Resource list for course projectsq http://cps.cse.wustl.edu/index.php/List_of_Projects
Ø Apply for $40 credits for Amazon AWS q https://aws.amazon.com/education/awseducate/apply/
3
AWSEC2
Today’s“Mainframe”
Service Provider Handles the Hardware.You consume the computational resource.
5 [1]source:hBp://www.alamy.com/stock-photo-icl-2900-series-mainframe-computer-installaLon-and-programmers-at-92958461.html[2]source:hBps://www.stratacore.com/the-advisor/data-center-m-a-recap-2016
Ø Datacenter (2016)Ø Multiuser shares multiple
computers (A datacenter)
Ø Mainframe Computer (1970s)q Multiuser shares one Computers
InfrastructureasaService(IaaS)Ø IaaS
q "physical server box”q Virtual Machine
• Memory• Storage• CPU• Network
Ø Exampleq AWS EC2q AWS HPC
Ø Usecaseq Build up you VM cluster
6
YouManage
APP
Data
Runtime
Middleware
OS
Virtualization
Server
Storage
Network
ServiceProviderManages
WebService
Tutorial:BigPicture
Ø Create q Two EC2 Ubuntu Instances
• T2.micro
q Ensure the connectivity
Ø Micro benchmarkq Latency
• ping
q Bandwidth• iperf3
7
AWS
EC2t2.micro EC2t2.micro
EC2:t2.micro
Step1:Create/LaunchInstances
Ø USE AWS Console WebUIØ Create t2.micro Instance
q 1 Xeon vCPU @ 2.5GHz
q 1GB Memoryq 8GB EBS Storage
q ??? Network
Ø Create Operating Systemq AMI: Ubuntu 16.04 Server
Ø Get its Public DNS (HostName)
Ø Get the Private Key
8 Source:hBps://www.shareicon.net/tag/network?&&cl=darkslategray&&s=glyph
AccessAWSEC2Service
Ø Sign Up and Sign In
Ø Choose EC2 Service
9 Source:hBps://aws.amazon.com/console
Launchinstances
Ø Launch instance in EC2 Dashboard
10
ChooseAMI
Ø Choose Your Image Type (Template Operating System)
11
InstanceType(“Hardware”ofVM)
Ø Choose your Virtual CPU number and Memoryq T2.micro 1 vCPU + 1GB @ $0.012 per Hour (running state)
12
ATTENTION:{CPU,Mem,Net}andStorage(EBS)arechargedSeparately!
Storage(“HardDrive”)
13
MountPoint Default8GB:$0.10perGB-month
Review&Launch:GeneratePrivateKey
14
PrivateKey:UseittologinyourVM
ChecktheDashboard
Ø Check the State: RunningØ Check the Public DNS
15
Wewillsshintotheinstanceviathisaddress
Step2:SSHintotheVM
Ø Use SSH and Key to login the VM
16
AWSEC2t2.micro EC2t2.microSSH
ChangethePermissionofthePrivateKey
Ø Change it to “owner read only”, i.e. Permission 400q sudo chmod 400 {Your_Private_Key}.pem
17
LoginIntoit
Ø Host Address: Public DNSØ Username: ubuntu[1]
q ssh –i {Your_Private_Key} ubuntu@{Pub_DNS_Name}
18 [1]ManageUserAccount:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html[2]UsingpuBytologin:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/puBy.html
WindowsUser:UsePuWyasaSSHclient[2]
Step3:LatencyandBandwidthTest
19
AWS
EC2t2.micro EC2t2.micro
What’stheRTTdelay?What’stheThroughput?
ConnecYvitySeZng
Ø Security Group SettingsØ Edit Inbound Rules
20
AllowInboundConnecYons
Ø Rulesq Enable All TCP/UDP/ICMP
21
Testthelatency
Ø Using Pingq Get RTT Min/Avg/Max/Dev
22
PingstaLsLcs
TesttheBandwidth
Ø Using iperf3q Install: sudo apt-get install iperf3
q Server: iperf3 –s
q Client: iperf3 –c {server_ip}
23
1GbpsBandwidth
Step4:RemembertoStop/Terminate
24
Ø All about Moneyq E.g. t2.micro 1-VCPU + 1GB Mem @ $0.012 per Hour (running
state)[1]
[1]EC2Pricing:hBps://aws.amazon.com/ec2/pricing/on-demand/
RemembertoStopYourVM
Ø Either shutdown from CLI:q sudo shutdown –h now
Ø Or Stop your instances from WebUI:
25
Stopped:However,theywillsYllchargeyou
Ø Charge for EBS Storage[1]
Ø Pricingq E.g. gp2 volume: $0.10 per GB-month
26
[1]WhychargedbyEBS:hBps://aws.amazon.com/premiumsupport/knowledge-center/ebs-charge-stopped-instance/[2]EBSPricing:hBps://aws.amazon.com/ebs/pricing/
StoppedV.S.TerminatedØ “Terminate" your instance: PERMANENTLY DELETES the
instance, and free the EBS storage.Ø “Stop" the instance: Release the run time computational
resource, but keep the disk.Ø Note: Frequently committing your work to your repository.
Ø More info: Read “Differences Between Reboot, Stop, and Terminate” in [1]
27 [1]LifeCycle:hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-lifecycle.html[2]EBSPricing:hBps://aws.amazon.com/ebs/pricing/
State Stop TerminateRoot volume (e.g. /dev/sda1)
The volume(EBS) is preserved The volume(EBS) is deleted.
Billing You stop incurring charges for an instance as soon as its state changes to stopping.
You stop incurring charges for an instance as soon as its state changes to shutting-down.
Project0:Part1AWSEC2Ø Create Your AWS accountØ Build up two EC2 instances and Setting up the ConnectivityØ Run ping / iperf3 and get latency / bandwidth statistics
q 1. Repeat this experiment (inbound bandwidth test)q 2. Create Instance in different zone (e.g. us-west-2a/b/c), test the
bandwidth between your PC and the EC2 Instance• Which Zone shows the best bandwidth / latency?
q 3. Pick up one “Zone”, write a script to repeat test every 2 hours (or more frequent, if you want) in a consecutive 24 hours
• Plot a figure to show the latency against time• Plot a figure to show the bandwidth against time
q 4. Run a single test for 10 minute, will the stats change?• Note: iperf3 –c {server_ip} –t {seconds}
Ø Email your results (inline, you don’t need to write a report) to lihaoran@email.wustl.edu
28
AWSIoT:APaaSExample
Pla`ormasaService(PaaS)
Ø PaaSq You get a framework
q Host Application
q Tools
Ø Exampleq AWS IoT
Ø Usecaseq Build up you’re smart A/C
controller
8/30/17 30
YouManage
ServiceProviderManages
APP
Data
Runtime
Middleware
OS
Virtualization
Server
Storage
Network
Internet-of-Things
8/30/17 31
Ø Things (Devices)q Many of them
• Different Types
• Isolated Systems
q Data and Command• Sensing the world
• Give Response
q Challenge• United: Connected + Communication
• Smart: Data Analytics + Strategy
Source:hBps://aws.amazon.com/iot-plakorm/hBp://www.brain-smart.net/smart-brain-health-blog/page/2/#axzz4W4oSp8a6
SoluYon:AWSIoT
8/30/17 32
United:Connect+CommunicaLon Smart:OtherCloudServiceDataStorageMachineLearning
Source:hBps://aws.amazon.com/iot-plakorm/
Tutorial:HelloAWSIoT!
8/30/17 33Source:hBps://aws.amazon.com/iot-plakorm/
RandomInteger[1,100]
Publish
AmazonSNS
Forward
AWSEC2t2.micro
Step1:CreateaVirtual"Thing”
8/30/17 34
AWSIoT
Virtual“Thing”/Shadow
GetintoAWSManageConsole
Ø Create your own AWS accountØ Sign In IoT Manage Console
q https://aws.amazon.com/iot/
8/30/17 35
Createathing
Ø 1. AWS IoT Menuq Registry
• Things è Create
Ø 2. Give a name
8/30/17 36
BasicInteract:Publish
Ø Using Embedded MQTT Client to Test
Ø Check the Things Shadow
8/30/17 37
BasicInteract:Subscribe
8/30/17 38
AWSEC2t2.micro
Step2:Connecta“Physical”Device
8/30/17 39
AWSIoT
Virtual“Thing”/Shadow
ABach
CerLficate
Policy Copy
MQTTClient
RandomInteger[1,100]
CreateandgetCerYficates
Ø Create Certificatesq Security è Certificates è Create
Ø Download Cert Files• 1. public & private key• 2. thing cert
• 3. Root CA for AWS
8/30/17 40
CreatePolicyandaWachittocert
Ø Create Policy
Ø Attach Policy to Certificates
8/30/17 41
ConnectyourDevice
Ø Copy certificates to your EC2 Instanceq Note: through scp utility
Ø Choose your AWS SDK (support MQTT)q Node JS
q Pythonq Java
Ø You can also use third party MQTT toolsq Python (paho-mqtt library)
8/30/17 42
SomeNotes
Ø 1. You will need these certification when setting up the TLS1.2 verification
Ø 2. You will need the endpoint and port (8883) when connect to AWS IoT Gateway
8/30/17 43
More:RuleEngine,LinkwithSNSservices
Ø Simple Notification Service
8/30/17 44
Publish
AmazonSNS
AWSIoT
Virtual“Thing”/Shadow
Forward
Topic:CSE520_Tutorial
Subscribe
Subscribe
AWSEC2t2.micro
CreateaRuleinAmazonIoT
Ø Add a query to filter your inteseting topic (event)
Ø Add an Action:q Forward this message to SNSq Specify Dest ARN
q Enable Rule
8/30/17 45
NoYficaYononSMS&Email
8/30/17 46
AWSIoT:APaaSExample
8/30/17 47Source:hBps://aws.amazon.com/iot-plakorm/
Publish
AmazonSNS
Forward
AWSEC2t2.micro
AWSEC2t2.micro
Subscribe
AWSIoTasaMQTTMessageBroker
AWSIoTstoragesThingsStates
AWSIoTprovidesthecapabilitytolinkAgainstotherservices
ChooseyourProjecttopic:Pickuptheserviceyouinterest:Tryitfirst.Andcanyoubuildyourownservicebyusingopensourcecomponents?
Project0Part2:AWSIoT
Ø Create a Thing in AWS IoTØ Use Web MQTT tool to update Thing Shadow State
q i.e. publish to shadow/update topic
Ø Setup MQTT client in EC2 instanceØ Subscribe the shadow/update topic in EC2 instance
Ø Email the screenshots to lihaoran@email.wustl.edu
48
AWSEC2t2.micro
Subscribe
Project0:Part2
Ø Sample Screenshot:q 1. Updated Shadow State
in WebUIq 2. Printed message on EC2
terminal
49
Project0:
Ø Part 1 AWS EC2q Bandwidth Test
Ø Part 2 AWS IoTq Subscribe to shadow/update
Ø Due on: 10AM Sep/6 (Wednesday)
50
Pointers
Ø Amazon EC2q http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
EC2_GetStarted.html
Ø Amazon IoTq http://docs.aws.amazon.com/iot/latest/developerguide/what-is-
aws-iot.html
Ø Resource list for course projectsq http://cps.cse.wustl.edu/index.php/List_of_Projects
Ø Apply for $40 credits for Amazon AWS q https://aws.amazon.com/education/awseducate/apply/
51
Thanks!
HaoranLi
Aug/30/2017
Project0:
Ø Part 1 AWS EC2q Bandwidth Test
Ø Part 2 AWS IoTq Subscribe to shadow/update
Ø Due on: 10AM Sep/6 (Wednesday)
53
Project0:Part1AWSEC2Ø Create Your AWS accountØ Build up two EC2 instances and Setting up the ConnectivityØ Run ping / iperf3 and get latency / bandwidth statistics
q 1. Repeat this experiment (inbound bandwidth test)q 2. Create Instance in different zone (e.g. us-west-2a/b/c), test the
bandwidth between your PC and the EC2 Instance• Which Zone shows the best bandwidth / latency?
q 3. Pick up one “Zone”, write a script to repeat test every 2 hours (or more frequent, if you want) in a consecutive 24 hours
• Plot a figure to show the latency against time• Plot a figure to show the bandwidth against time
q 4. Run a single test for 10 minute, will the stats change?• Note: iperf3 –c {server_ip} –t {seconds}
Ø Email your results (inline, you don’t need to write a report) to lihaoran@email.wustl.edu
54
Project0Part2:AWSIoT
Ø Create a Thing in AWS IoTØ Use Web MQTT tool to update Thing Shadow State
q i.e. publish to shadow/update topic
Ø Setup MQTT client in EC2 instanceØ Subscribe the shadow/update topic in EC2 instance
Ø Email the screenshots to lihaoran@email.wustl.edu
55
AWSEC2t2.micro
Subscribe
Project0:Part2
Ø Sample Screenshot:q 1. Updated Shadow State
in WebUIq 2. Printed message on EC2
terminal
56
Recommended