CUMULONIMBUS FORTIFICATION - SECURE YOUR DATA IN THE CLOUD

David Busby | Information Security Architect | PerconaDavid Busby | Information Security Architect | Percona

Threat ModelsQuantifying Threats To Your DeploymentsQuantifying Threats To Your Deployments

3

Threat Models

What is Threat Modeling?What is Threat Modeling?

A Threat Model Is• Prioritized list of security

enhancements for● Concepts● Requirments● Design● Implementation

• Identify & Isolate● Areas of Risk● Potential Threats

A Threat Model Is• Prioritized list of security

enhancements for● Concepts● Requirments● Design● Implementation

• Identify & Isolate● Areas of Risk● Potential Threats

...• Defining Scope• Understanding

● Possible Attack Vectors● Countermeasures

• Reduction of Risk• Some Examples

● OWASP● Microsoft SDL● Apple

...• Defining Scope• Understanding

● Possible Attack Vectors● Countermeasures

• Reduction of Risk• Some Examples

● OWASP● Microsoft SDL● Apple

4

Threat Models

What is a Side Channel Attack?What is a Side Channel Attack?

A Side Channel Attack Is• Indirect attacks to reveal secrets• In Cryptography

● Power Analysis● Accoustic Analysis● E.M Analysis● Cache Timing

• Children● Are the masters of side channel

attacks

A Side Channel Attack Is• Indirect attacks to reveal secrets• In Cryptography

● Power Analysis● Accoustic Analysis● E.M Analysis● Cache Timing

• Children● Are the masters of side channel

attacks

...• In General

● Power Analysis● Accoustic Analysis

● Keyboard Accoustics● Inaudible frequencies

● E.M Analysis● Noise Floor

● Weaponizing your pets

...• In General

● Power Analysis● Accoustic Analysis

● Keyboard Accoustics● Inaudible frequencies

● E.M Analysis● Noise Floor

● Weaponizing your pets

5

Threat Models

What is a Co-Residency Attack?What is a Co-Residency Attack?

A Co-Residency Attack is• Indirect attacks to reveal secrets

● Against virtual guests on the hypervisor● Pre-req for “Side Channel”

attacks such as● Cache Timing

● AWS EC2● White papers claim some 40%

success rate● Defated by dedicated EC2 option

A Co-Residency Attack is• Indirect attacks to reveal secrets

● Against virtual guests on the hypervisor● Pre-req for “Side Channel”

attacks such as● Cache Timing

● AWS EC2● White papers claim some 40%

success rate● Defated by dedicated EC2 option

...• AWS EC2

● Dedicated Instances Option● Prevents Co-Residency

• Openstack● Instances can be weighted

● Dedicate HW pool for “sensitive” trusted instances

● Dedicated a pool for everything else

...• AWS EC2

● Dedicated Instances Option● Prevents Co-Residency

• Openstack● Instances can be weighted

● Dedicate HW pool for “sensitive” trusted instances

● Dedicated a pool for everything else

Amazon AWSCompliance DocumentationCompliance Documentation

7

Amazon AWS

Why should I care about compliance?Why should I care about compliance?

A strong foundation• aws.amazon.com/compliance• You can't control the underlying

infrastructure● You want some assurance

● PCI DSS Level 1● No this doesn't make you PCI

compliant● Shared responsibility model

A strong foundation• aws.amazon.com/compliance• You can't control the underlying

infrastructure● You want some assurance

● PCI DSS Level 1● No this doesn't make you PCI

compliant● Shared responsibility model

...• “A foolish man, which built his

house upon the sand”● No VM is secure if …

● Hypervisor is insecure● Network is insecure● DC is insecure● Support staff are insecure

...• “A foolish man, which built his

house upon the sand”● No VM is secure if …

● Hypervisor is insecure● Network is insecure● DC is insecure● Support staff are insecure

Amazon AWSFeatures / Services to Secure your DeploymentsFeatures / Services to Secure your Deployments

9

Amazon AWS

Key Management ServiceKey Management Service

Create, Store, Control Keys• Encryption Support for

● EBS, RDS*, S3, RedShift, …● *not MySQL RDS

• Key management● Yearly rotation

● Retired not removed● Service auto detects correct key

for use

Create, Store, Control Keys• Encryption Support for

● EBS, RDS*, S3, RedShift, …● *not MySQL RDS

• Key management● Yearly rotation

● Retired not removed● Service auto detects correct key

for use

...• Key access controled through IAM

● Define key Administrators, Users• AES-GCM-256• Hardened Security Appliance (HSA)

● HSM backed• Auditable usage

● CloudTrail

...• Key access controled through IAM

● Define key Administrators, Users• AES-GCM-256• Hardened Security Appliance (HSA)

● HSM backed• Auditable usage

● CloudTrail

10

Amazon AWS

Virtual Private CloudVirtual Private Cloud

Isolated Cloud Resources• VPN

● IPSec VPN Tunnel Support● Peer with

● DC's, Office, Other VPC• Routers can be configured

● internet access● NAT

• EIPs still work!

Isolated Cloud Resources• VPN

● IPSec VPN Tunnel Support● Peer with

● DC's, Office, Other VPC• Routers can be configured

● internet access● NAT

• EIPs still work!

...• Flow Logs

● Usefull for basic analytics ● Src dst srcport dstport bytes

● Can be pushed into Splunk● CloudWatch + E.L.K

...• Flow Logs

● Usefull for basic analytics ● Src dst srcport dstport bytes

● Can be pushed into Splunk● CloudWatch + E.L.K

11

Amazon AWS

Identity and Access ManagementIdentity and Access Management

User ACL• Can't restrict the root account

● Stop using it!● Delete API keys!

• Deploy MFA● On all users● Especially the root account

User ACL• Can't restrict the root account

● Stop using it!● Delete API keys!

• Deploy MFA● On all users● Especially the root account

...• Create Groups

● Assign Users● Ensure P.O.L.P

●

• Advisory tools● Netflix Security Monkey● AWS Trusted Advisor● Nimbsotratus

...• Create Groups

● Assign Users● Ensure P.O.L.P

●

• Advisory tools● Netflix Security Monkey● AWS Trusted Advisor● Nimbsotratus

12

Amazon AWS

Identity and Access ManagementIdentity and Access Management

API Access• Create & Retire keys• API keys must be protected

● Disclosure can be● Expensive

● Bit/Lite/Other Coin Mining● Malware distribution● DoS “stresser”● Phishing● Complete nuke

API Access• Create & Retire keys• API keys must be protected

● Disclosure can be● Expensive

● Bit/Lite/Other Coin Mining● Malware distribution● DoS “stresser”● Phishing● Complete nuke

...• Do not need instance access

● Snapshot● Export

● Can even export to OVA● Or attach to another instance● Zero indication in traditional

controls● Deploy CloudTrail

● Even RDS

...• Do not need instance access

● Snapshot● Export

● Can even export to OVA● Or attach to another instance● Zero indication in traditional

controls● Deploy CloudTrail

● Even RDS

13

Amazon AWS

API keys in Pastebin / GithubAPI keys in Pastebin / Github

OpenstackWhat is Openstack Bandit?What is Openstack Bandit?

15

Openstack

BanditBandit

“Security Linter”• Can be configured to error on

● “known bad”● Default passwords● Weak hashes● Insecure methods

● Yaml.load● Pickle.loads

“Security Linter”• Can be configured to error on

● “known bad”● Default passwords● Weak hashes● Insecure methods

● Yaml.load● Pickle.loads

...• Deployed as part of CI process

● Similar to unit tests● Force “build” failiure if

● Known insecure● Insecure method● Insecure use input

...• Deployed as part of CI process

● Similar to unit tests● Force “build” failiure if

● Known insecure● Insecure method● Insecure use input

OpenstackFeatures / Services to Secure your DeploymentsFeatures / Services to Secure your Deployments

17

Openstack

NeutronNeutron

Networking as a Service (NaaS)• VPNaaS

● IPSec VPN (similar to VPC)• “technology-agnostic, network

abstraction”• Can leverage OpenVSwitch• TL;DR Virtualized switching

infrastructure

Networking as a Service (NaaS)• VPNaaS

● IPSec VPN (similar to VPC)• “technology-agnostic, network

abstraction”• Can leverage OpenVSwitch• TL;DR Virtualized switching

infrastructure

18

Openstack

BarbicanBarbican

Secure secrets management• REST API

● Cinder● Kilo

● Glance● Not yet

● Swift● Blueprints exist

● Nova● Blueprints exist

Secure secrets management• REST API

● Cinder● Kilo

● Glance● Not yet

● Swift● Blueprints exist

● Nova● Blueprints exist

...• Looks to replicate KMS functionality• Can back onto HSM appliances• Currently an “emerging” feature

...• Looks to replicate KMS functionality• Can back onto HSM appliances• Currently an “emerging” feature

DockerWhat is docker?What is docker?

20

Docker

What is conatiner virtualization?What is conatiner virtualization?

RunC (formerly libcontainer)• Layered filesystem auFS

● Share read-only components● Mount write per container

• Namespacing & Groups● Similar to LXC● Cgroups control resources● Namespaces helps provide

isolation

RunC (formerly libcontainer)• Layered filesystem auFS

● Share read-only components● Mount write per container

• Namespacing & Groups● Similar to LXC● Cgroups control resources● Namespaces helps provide

isolation

...• Each container gets

● Its own network stack

...• Each container gets

● Its own network stack

21

Docker

How is this different?How is this different?

Containers• All containers on a host run the

same● Host OS

● Kernel● Some binaries & libs

• Rightscale blog post

Containers• All containers on a host run the

same● Host OS

● Kernel● Some binaries & libs

• Rightscale blog post

22

Docker

Does this affect my attack surface?Does this affect my attack surface?

Some caveats to docker• The daemon requires root

● Users in the docker group have access to the daemon● Therefor docker group users

should be considered as having root access

● Container breakout is entirely possible● And has been proven before

Some caveats to docker• The daemon requires root

● Users in the docker group have access to the daemon● Therefor docker group users

should be considered as having root access

● Container breakout is entirely possible● And has been proven before

...• Possible to craft

● Malicious images● Same as any VM

● Docker Security Pages

...• Possible to craft

● Malicious images● Same as any VM

● Docker Security Pages

23

Docker

Is it production ready?Is it production ready?

If Properly Configured• As with any other technology

● Research the caveats● And limitations● Produce your threat-model

● And secure accordingly

If Properly Configured• As with any other technology

● Research the caveats● And limitations● Produce your threat-model

● And secure accordingly

Maybe ...• Docker & SELinux

● Dan Walsh (RedHat)• Docker Security Page• AWS Container Service

Maybe ...• Docker & SELinux

● Dan Walsh (RedHat)• Docker Security Page• AWS Container Service

Federated cloudsUnited federation of … cloud technologies (admit it you thought planets)United federation of … cloud technologies (admit it you thought planets)

25

Federated Clouds

What is it?What is it?

Taking cloud $vendors• Amazon• Rackspace• Google• HP• Digital Ocean• Linode• Etc ...

Taking cloud $vendors• Amazon• Rackspace• Google• HP• Digital Ocean• Linode• Etc ...

And through API's integrate with

• Private cloud deployments● Openstack● Docker Swarm● VMWare● Etc ...

And through API's integrate with

• Private cloud deployments● Openstack● Docker Swarm● VMWare● Etc ...

26

Federated Clouds

Why do I need one?Why do I need one?

Develop & QA• On known common stack

● OS● Application stack

• Automate QA● Spin instance / Container● Deploy code from SCM● Run tests

• Automate deployment● Build passes use Apis to push

Develop & QA• On known common stack

● OS● Application stack

• Automate QA● Spin instance / Container● Deploy code from SCM● Run tests

• Automate deployment● Build passes use Apis to push

Production• Some services can import entire

images● AWS

● OVA (VMDK)● Openstack

● Glance● QCOW (preffered)● Not supported by some

$vendors

Production• Some services can import entire

images● AWS

● OVA (VMDK)● Openstack

● Glance● QCOW (preffered)● Not supported by some

$vendors

27

Federated Clouds

Ensuring a secure “chain of custody”Ensuring a secure “chain of custody”

Develop & QA• Builds OK

● Store image● Sign the image

● e.g. GPG● “Appliance” can now be deployed

Develop & QA• Builds OK

● Store image● Sign the image

● e.g. GPG● “Appliance” can now be deployed

Production• Deploy appliance

● Verify signature• Post-deploy integration

● Ansible/puppet/chef• Fail over to new appliance• Retire old appliance• Retain API Audit logs

Production• Deploy appliance

● Verify signature• Post-deploy integration

● Ansible/puppet/chef• Fail over to new appliance• Retire old appliance• Retain API Audit logs

Security CIHow Security can be part of your CI processHow Security can be part of your CI process

29

Security CI

Integrating Security in your CIIntegrating Security in your CI

Extend your unit tests• Only allow “safe” methods

● For SQL● Sanitize user input● Test sanitization methods● e.g.

● known “good” class / method● Require compile args

● -pie -fPIE

Extend your unit tests• Only allow “safe” methods

● For SQL● Sanitize user input● Test sanitization methods● e.g.

● known “good” class / method● Require compile args

● -pie -fPIE

Fail securely• Fail builds on unsafe / non standard

methods• Enforce security as a development

standard

Fail securely• Fail builds on unsafe / non standard

methods• Enforce security as a development

standard

Telemtry ProcessingWhy is it your most important data source?Why is it your most important data source?

31

Telemtry Processing

In the ether no one can hear you scream ...In the ether no one can hear you scream ...

In AWS API calls are “invisible”• Sort of ...

● CloudTrail + SNS● Alarm on specific API activity

● Instances● Launch / Stop / Terminate● Snapshot

● S3● Create / Delete / ACL changes

In AWS API calls are “invisible”• Sort of ...

● CloudTrail + SNS● Alarm on specific API activity

● Instances● Launch / Stop / Terminate● Snapshot

● S3● Create / Delete / ACL changes

...• KMS

● Keys● Administration

● Retire / create● Access

● IAM● Add / delete MFA● Keys generation / removal● Etc ...

...• KMS

● Keys● Administration

● Retire / create● Access

● IAM● Add / delete MFA● Keys generation / removal● Etc ...

32

Telemtry Processing

In the ether no one can hear you scream ...In the ether no one can hear you scream ...

In openstack• Logging is configured per

component● Cinder● Nova● Neutron

● Aka Quantum● Keystone● Barbican

In openstack• Logging is configured per

component● Cinder● Nova● Neutron

● Aka Quantum● Keystone● Barbican

...• API calls are “invisible”

● To traditional IDS● Push logs onto your own

configuration● Alert on set conditions

● Instances up / terminate● Key creation / deletion● Snapshots● Network configuration

...• API calls are “invisible”

● To traditional IDS● Push logs onto your own

configuration● Alert on set conditions

● Instances up / terminate● Key creation / deletion● Snapshots● Network configuration

33

Telemtry Processing

In the ether no one can hear you scream ...In the ether no one can hear you scream ...

Traditional telemetry• Resource metrics

● CPU / RAM / IO● Should include GPU

● Network● IDS / IPS

● Host events● Network Events

Traditional telemetry• Resource metrics

● CPU / RAM / IO● Should include GPU

● Network● IDS / IPS

● Host events● Network Events

...• Service metrics

● MySQL● Running queries● AHI● Buffer pool

● Queue services● Queue length● Message size

● HTTPD● Request load

...• Service metrics

● MySQL● Running queries● AHI● Buffer pool

● Queue services● Queue length● Message size

● HTTPD● Request load

34

Telemtry Processing

Data overload, handeling many lines/sData overload, handeling many lines/s

ELK• ElasticSearch

● Indexing & Search● Lucene

• LogStash● Log aggregation● Mutation

• Kibana● Visualing interface for

ElasticSearch

ELK• ElasticSearch

● Indexing & Search● Lucene

• LogStash● Log aggregation● Mutation

• Kibana● Visualing interface for

ElasticSearch

...• LogStash

● Can feed alerts to Nagios• Make it modular

● Deploy components on seperate nodes, where possible

● Also ensures availability

...• LogStash

● Can feed alerts to Nagios• Make it modular

● Deploy components on seperate nodes, where possible

● Also ensures availability

35

Telemtry Processing

Data overload, handeling many lines/sData overload, handeling many lines/s

36

Telemtry Processing

Data overload, handeling many lines/sData overload, handeling many lines/s

Hadoop• OpenSOC

● 1.2M packets/sec RealTime● Flume

● Ships log data● Kafka

● Messaging system● Storm

● Distributed job processing● Runs “enrichment”

●

Hadoop• OpenSOC

● 1.2M packets/sec RealTime● Flume

● Ships log data● Kafka

● Messaging system● Storm

● Distributed job processing● Runs “enrichment”

●

...• ElasticSearch

● ElasticSearch can back onto HDFS● Greater analytics variety

● Map reduce• Alerting

● Storm jobs could run analytics, alert on set conditions.

...• ElasticSearch

● ElasticSearch can back onto HDFS● Greater analytics variety

● Map reduce• Alerting

● Storm jobs could run analytics, alert on set conditions.

37

Telemtry Processing

Don't over-engineer things!Don't over-engineer things!

Emerging TechnologiesProjects to keep an eye on, to help in your security.Projects to keep an eye on, to help in your security.

39

Emerging Tech

Vaultproject.ioVaultproject.io

Secret storage• API driven access to

● Secrets● Dynamic secrets

● Aids auto-rotation● Encryption service

● Encrypt / Decrypt data via API● Leasing & Renewal

Secret storage• API driven access to

● Secrets● Dynamic secrets

● Aids auto-rotation● Encryption service

● Encrypt / Decrypt data via API● Leasing & Renewal

...• Similar to Barbican• HA Configurable

● Consul• Audit backend• Multiple integrations

● AWS● MySQL● PostgreSQL

...• Similar to Barbican• HA Configurable

● Consul• Audit backend• Multiple integrations

● AWS● MySQL● PostgreSQL

40

Emerging Tech

Haka-Security.orgHaka-Security.org

Developer friendly network security ?

• LUA DSL● Object Orientated● Kibana suport

● Hakabana• Also for analytics

● Can analyse pcap files

Developer friendly network security ?

• LUA DSL● Object Orientated● Kibana suport

● Hakabana• Also for analytics

● Can analyse pcap files

Building The CastleOk, I've got the idea. But how do I proceed?Ok, I've got the idea. But how do I proceed?

42

Building the Castle

“Hardening” tips for the private cloud“Hardening” tips for the private cloud

Tuning you'll want to do• Disable Pci Passthrough

● DMA• Openstack Nova

● Disable Soft Delete• Openstack Glance

● Disable delayed delete

Tuning you'll want to do• Disable Pci Passthrough

● DMA• Openstack Nova

● Disable Soft Delete• Openstack Glance

● Disable delayed delete

...• Openstack cinder

● Enable volume encryption● ISCSI packets● Backups encrypted

• Openstack Barbican● Cinder support● Can back onto a HSM

...• Openstack cinder

● Enable volume encryption● ISCSI packets● Backups encrypted

• Openstack Barbican● Cinder support● Can back onto a HSM

43

Building the Castle

“Hardening” tips for the private cloud“Hardening” tips for the private cloud

Tuning you'll want to do● Entropy sources

● Most use /dev/random● Invest in HWRNG

● Rngd conf & deploy● Feed /dev/random●

• Define instance assignment criteria• Define “trusted” images criteria

Tuning you'll want to do● Entropy sources

● Most use /dev/random● Invest in HWRNG

● Rngd conf & deploy● Feed /dev/random●

• Define instance assignment criteria• Define “trusted” images criteria

...• Disable “live migration”

● Copies memory, data etc over the network● Libvirtd can be configured to

encrypt transport manually● No Horizon support at the time

of writing

...• Disable “live migration”

● Copies memory, data etc over the network● Libvirtd can be configured to

encrypt transport manually● No Horizon support at the time

of writing

44

Building the Castle

Closing thoughts & QAClosing thoughts & QA