Serverless Patterns: “No server is easier to manage than no server” - AWS Security Day

Karthi Thyagarajan karthit@amazon.com

Maitreya Ranganath maitreyr@amazon.com

AWS Solutions Architects

Why Serverless?

AWS

Lambda

Amazon

Kinesis

Amazon

S3

Amazon API

Gateway

Amazon

SQS

Amazon

DynamoDB

AWS IoT

Amazon

EMR

Amazon

ElastiCache

Amazon

RDSAmazon

Redshift

Amazon

Elasticsearch Service

Managed Serverless

Amazon

EC2

“On EC2”

3 Tier Web Application

http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf

Serverless 3 Tier Web Architecture

Amazon API

Gateway

AWS

Lambda

Amazon

DynamoDB

Data tier App tier Presentation

tier

Amazon

S3

Amazon

CloudFront

https://aws.amazon.com/blogs/compute/the-squirrelbin-

architecture-a-serverless-microservice-using-aws-lambda/

Stream Processing & Analytics

SOURCE of

data

Fleet of servers poll for

changes

Listening to source Pushes to queue Pull off queue

Fleet of servers act as workers

to process the data

Auto-scale

worker nodes

to adjust with

load

S3 objects

Amazon Kinesis records

DynamoDB tables

SNS messages

IoT devices

…

Cross-AZ

replication for

high

availability

Load balancer

to distribute

traffic

Application code

Sensors

Amazon Kinesis:

Stream

AWS Lambda:

Stream Processor

Amazon DynamoDB:

Aggregates Table

Amazon S3:

Visualization Web App

https://blogs.aws.amazon.com/bigdata/post/Tx148NMGPIJ6F6F/Analyze-a-Time-Series-

in-Real-Time-with-AWS-Lambda-Amazon-Kinesis-and-Amazon-Dyn

http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/Subscriptions.h

tml

Customers running real-time data stream processing on

Kinesis+Lambda

AWS

Lambda

Aggregate

statistics

Real-time

analyticsKinesis Stream

“I want to apply custom logic to process

data being uploaded through my Kinesis

stream”.

• Client activity tracking

• Metrics generation

• Data cleansing

• Log filtering

• Indexing and searching

• Log routing

• Live alarms and notifications

https://aws.amazon.com/blogs/compute/indexing-amazon-dynamodb-content-with-amazon-elasticsearch-service-using-

aws-lambda/

Operations Automation

AWS Lambda:

Update R53

Amazon CloudWatch Events:

Rule Triggered

Amazon EC2 Instance

State Changes

Amazon DynamoDB:

EC2 Instance Properties

Amazon Route53:

Private Hosted Zone

Tag:

CNAME = ‘xyz.example.com’

xyz.example.com A 10.2.0.134

https://serverlesscode.com/post/lambda-schedule-ebs-

snapshot-backups/

https://serverlesscode.com/post/lambda-schedule-ebs-snapshot-backups-2/

https://aws.amazon.com/blogs/compute/scheduling-ssh-jobs-

using-aws-lambda/

Rule Repository https://github.com/awslabs/aws-config-

rules

Ensure IAM User has MFA Enabled https://github.com/awslabs/aws-config-

rules/blob/master/node/iam_mfa_require-triggered.js

Ensure all EC2 Instances are of a Given Type https://github.com/awslabs/aws-

config-rules/blob/master/python/ec2_desired_instance_type-triggered.py

Building a Dynamic DNS for Route 53 using CloudWatch Events and Lambda

https://aws.amazon.com/blogs/compute/building-a-dynamic-dns-for-route-53-using-cloudwatch-events-and-

lambda/

Scaling Amazon ECS Services Automatically Using Amazon CloudWatch and AWS Lambda

https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-

and-aws-lambda/

Using AWS Lambda with Auto Scaling Lifecycle Hooks

https://aws.amazon.com/blogs/compute/using-aws-lambda-with-auto-scaling-lifecycle-hooks/

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-custom-

resources-lambda-lookup-amiids.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-custom-resources-lambda-

cross-stack-ref.html

Dynamic Data Triggers

Tutorial: Using AWS Lambda with S3 http://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html

https://blogs.aws.amazon.com/bigdata/post/Tx2YRX3Y16CVQFZ/Building-and-Maintaining-an-Amazon-S3-

Metadata-Index-without-Servers

https://blogs.aws.amazon.com/security/post/Tx1ZTM4DT0HRH0K/How-to-Configure-Rate-Based-Blacklisting-

with-AWS-WAF-and-AWS-Lambda

https://blogs.aws.amazon.com/security/post/Tx223ZW25YRPRKV/How-to-Use-AWS-WAF-to-Block-IP-

Addresses-That-Generate-Bad-Requests

Customers running dynamic data ingestion and processing

using S3+Lambda

AWS

Lambda

Indexing

tables or

notifications

“I want to apply custom logic to process content being uploaded to my data store”. • Watermarking / thumbnailing• Transcoding• Indexing and deduplication• Aggregation and filtering• Pre processing• Content validation

Amazon S3

Bucket

Events

Processed

files

http://www.allthingsdistributed.com/2016/06/aws-lambda-serverless-reference-

architectures.html

http://www.serverless.com/

https://aws.amazon.com/blogs/compute/getting-started-with-jaws-on-amazon-web-services/

https://d0.awsstatic.com/whitepapers/AWS_Serverless_Multi-Tier_Architectures.pdf

https://aws.amazon.com/blogs/compute/microservices-without-the-servers/

http://highscalability.com/blog/2015/12/7/the-serverless-start-up-down-with-servers.html

https://youtu.be/U8ODkSCJpJU

https://youtu.be/pBLdMCksM3A

https://youtu.be/GnaO-LwdSuU

https://youtu.be/JT2xOYOdUvM

https://youtu.be/FhJxTIq81AU

https://youtu.be/hU25CIRPIJo

https://youtu.be/ZBxWZ9bgd44

https://youtu.be/WWDHxy4zuqg

https://youtu.be/D_U6luQ6I90