www.immobilienscout24.de

Berlin | 28.04.2016 | Schlomo SchapiroSystems Architect / Open Source Evangelist

http://creativecommons.org/licenses/by-nd/4.0

Hybrid Cloud

A Cloud Migration Strategy

@schlomoschapiro

go.schapiro.org/slides

Why should I care?

The cloud is here - let's

make the best out of it!

Our goal is to joina vibrant technical eco-

system to accelerate our own innovation speed.

PROJECT

Cloud Migration - Management View

Just ask about ...

Timeline

Budget

Engineered for the CloudSecurity

Resilience

Time & Money

Data Center Costs

SAN Storage Server Hardware

Server Hardware Core & Rack Switches

SAN StorageBackup Solution

Core & Rack SwitchesServer Hardware

SAN Storage

Backup Solution

Backup Solution

SAN Storage

5 years writing off

BU

DG

ET

Cloud Costs - Quick MigrationB

UD

GET

1st year 2nd year 3rd year

Cloud Migration - Costs Journey

Data Center Costs

Cloud Costs

Total CostsB

UD

GET

Invest

Save

ROI

How many years?

Engineering

Cloud = Scale Out.

Automate or Die.

Test Driven Development.

Everything will fail.

Live Staging Play Search User ...

InternalCommunication◉ No transport encryption◉ Trust based on IP◉ Easy Dev/Ops access to

debug and admin ports◉ Low latency (LAN)◉ Static service discovery

works

ExternalCommunication

◉ Must use HTTPS◉ Trust based on

authentication◉ Need secure back door for

debug and admin access◉ Medium / high latency◉ Effort for service discovery

Cloud Migration≈

Microservices Migration

AutomateAutomate

AutomateAutomate

AutomateAutomate

Automate

Data Center

Hardware Network Storage

Virtualization

Operating System

Application

Configuration

Load Balancer

Autom

ation

CodeCloud (AWS)

Hardware Network Storage

Virtualization

Operating System

Application

Configuration

Load Balancer

Cloud Form

ationEC2 VPC S3

ECS / Lambda / Bean Stalk

Docker AMI ZIP / S3

ELB

Route53 Cloud Front

RDS / SNS / SQS / IAM / EMRApi Gateway / Dynamo DB / ...

Resilience

Cloud Formation StackRegion

VPCRDS

A typical web application on AWS ...

Autoscaling Group

EC2 EC2 EC2

ELB

RDS

SPOF

More resilience

Cloud Formation Stack

Region

VPC RDS

Autoscaling Group

EC2 EC2 EC2

ELB

RDS

Cloud Formation Stack

Region

VPC RDS

Autoscaling Group

EC2 EC2 EC2

ELB

RDS

Static credentials

are just broken by

design!

Static Credentials

◉ SSH keys - copy and crack at home➨ SSH HostbasedAuthentication➨ Consider IP trust & rsh for automation and clusters➨ Use ssh-agent, personal keys should never leave the desktop

◉ AWS key & secret - you won't notice me using them➨ Use temporary credentials (secret, key, token)➨ Watch your Cloud Trail logs

◉ Username & password - thanks!➨ Federated logins for people➨ Certs for machines (although still static credentials)➨ IP trust may be good enough

...

Private Connec-

tion to DCNo Authenti-

cation

Perimeter Security

Blind Trust

Firewall=

Security

Federatedemployee login

Watch logs for anomalies

App is fully responsible for security

Jump host for dev & admin access

Local firewalls everywhere, explicit access only.

AWS:Security Groups

Service⇔Service Communication

over public Internet

HTTPS only. Setup identity management for services (OAuth2)

HybridCloud

Hybrid Cloud?

My Virtual Machine / Docker Container can run on premise or in the cloud.

1

Use the best tool for the job:Some apps run better on premise and some apps benefit more from the cloud.Embrace Cloud services as part of our applications and integrate with them.

2

Hybrid Cloud Comparison

Run VMs/Docker anywhere+ No vendor lock in+ Write once, run anywhere+ Easily support multiple

platforms+ Unified tooling over all

platforms+ Unified tooling also for data

center hosting+ Shift workloads based on

cost and demand

Use best tool for the job+ Benefit from external

innovation+ Ready-made services instead

of roll-your-own+ "Serverless" applications+ Significantly reduce OPS+ Use platform migration to

refactor applications+ Costs scale well with

application usage+ Small things are very cheap+ More options to optimize costs

80% 20%

Benefit Work

Work Benefit

AWS Managed Services

VM Hosting (EC2, ECS)

CloudEnablement

A Cloud Migration Strategy

1. Establish Cloud platform besides data center

2. Integrate Cloud platform with data center

3. Build new applications into the cloud

4. Migrate existing services into the cloud

5. Repeat until done

1. Establish Cloud platform besides data center

1. Solve common problems:security, compliance and cost control

2. Provide basic solution forlogging, monitoring, deployment

3. Easy & secure access to Cloud platform for all employees, using temporary credentials

4. Decide upon macro architecture,e.g. many AWS accounts, communication over public Internet without VPN, OAuth2 everywhere

2. Integrate Cloud platform with data center

1. Provide temporary Cloud credentials to every server2. Provide secure communication framework between

services running in the data center and in the cloud3. Use Cloud managed services from the data center,

e.g. SNS, SQS, EMR, Data Pipeline, Kinesis, SWF4. Migrate persistent storage to Cloud where beneficial,

e.g. S3, DynamoDB5. Improve automation and gather operational experience

3. Build new applications into the cloud

1. Learn working with full stack responsibility2. Learn how to architect and develop to benefit

from cloud platform3. Learn how to optimize development and

operational costs4. Improve automation and gather operational

experience

4. Migrate existing services into the cloud

1. Keep total cost (data center + cloud) in check,e.g. prioritize service migrations by data center hardware replacement / investment plan

2. Prioritize cloud migration against feature development3. Migrate application into Cloud together with new feature4. Improve automation and gather operational experience

5. Repeat until done

1. After the migration is before the next migration,e.g. to the next Cloud platform

2. "Remaining" services in data center have to pay for all the data center

3. Optimize between costs and availability requirements4. Improve automation and gather operational experience

………

5. Always change the running system

The ImmobilienScout24 Cloud Toolbox

The ImmobilienScout24 Cloud Toolbox

◉ Compliance: AWS resources should only run in the EUhttps://github.com/ImmobilienScout24/aws-monocyte

◉ Security: Provide AWS credentials to humans and machineshttp://immobilienscout24.github.io/afp/

◉ Security: SSH jump host with OpenID Connect authenticationhttps://github.com/ImmobilienScout24/c-bastion

◉ Automation: Cloud Formation cross-stack managementhttps://github.com/ImmobilienScout24/cfn-sphere

◉ Development: Automate Python Lambda packaginghttps://github.com/ImmobilienScout24/pybuilder_aws_plugin

go.schapiro.org/slides@schlomoschapiro www.schapiro.org/schlomo/publications