73
Fast Forward Reflecting on a Life of Watching Movies and a Career in Security Jason Chan VP, Information Security @ Netflix @chanjbs

Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Fast Forward

Reflecting on a Life of Watching Movies and a Career in Security

Jason ChanVP, Information Security @ Netflix@chanjbs

Page 2: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 3: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 4: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Credit: @LoulouHoltz

Page 5: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 6: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 7: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 8: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 9: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

So . . . what does this have to do with security?

Page 10: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 11: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 12: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 13: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 14: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 15: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 16: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 17: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Credit: @matt_tesauro

Page 18: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Back to the movies . . .

Page 19: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 20: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Core Functionality

Security

MigrationsUpgrades

Other Campaigns

Change

DeploymentInfrastructure

Technology Standards Operations

Basics

Observability

Performance Reliability

Scalability

Non-Functional Requirements

Page 21: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Reducing Cognitive Load for Developers

Page 22: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 23: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 24: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 25: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Simplifying the Security Interface for Developers

Page 26: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Are you trying to make your engineers security experts?

Or do you just want them to build and operate secure systems?

Page 27: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

What security functions can we abstract to simplify the developer experience?

Page 28: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Netflix Studio Engineering

Page 29: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 30: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 31: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Netflix Studio EngineeringOptimize production from “pitch to play”Lots of innovation and iteration

Page 32: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Netflix Studio User

Studio LOB App A

Studio LOB App N

Netflix Studio Apps

Page 33: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Simplify and Improve Security through Functionality Abstraction

Page 34: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Leverage Netflix OSS - Zuul“built to enable dynamic routing, monitoring, resiliency and security”

https://github.com/Netflix/zuul/wiki

Page 35: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Netflix Studio User

Studio LOB App A

Studio LOB App N

Netflix Studio Apps with Zuul and Wall-E

Wall-E

Rate LimitIP Blacklist

AuthenticationAuthorizationSigSci WAF

Schema Check

Schema CheckSec Headers

DLP

Pre-Filters Post-Filters

Page 36: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 37: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

ResultsLower cognitive load for onboarding security Centralized and managed functionalityFrees developers to build the Netflix Studio!

Page 38: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 39: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Monolith to microservices: networkImmutable infra: OS, custom app, middleware Infra as code: Everything!

Blurring Lines: App and Infra

Page 40: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Tackling App and Infra Integration:Seamless Least Privilege

Page 41: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Instances

Hadoop

Email Services

Storage

Database

Message Queue

The Magic of IaaS

Page 42: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Ex: Cloud Based Word Processor

Page 43: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Ex: Cloud Based Word Processor{

"Effect": "Allow",

"Action": ["*:*"]

"Resource": "*"

}

Page 44: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Ex: Cloud Based Word Processor{

"Effect": "Allow",

"Action": ["s3:*"]

"Resource": "*"

}

Page 45: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Ex: Cloud Based Word Processor{

"Effect": "Allow",

"Action": ["s3:GetObject", "s3:PutObject"]

"Resource": "*"

}

Page 46: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Ex: Cloud Based Word Processor{

"Effect": "Allow",

"Action": ["s3:GetObject", "s3:PutObject"]

"Resource": "arn:aws:s3:::wp_bucket"

}

Page 47: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 48: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

AWS provides data about API use

This data acts as a basis for action

Page 49: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

When a new application is created, we provide a base set of permissions

s3:GetObjects3:PutObject............sqs:ReceiveMessage

Page 50: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

We observe the application to see which permissions are actually used

Page 51: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

We then remove unused permissions

s3:GetObjects3:PutObject............sqs:ReceiveMessage

Page 52: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

We then remove unused permissions

s3:GetObjects3:PutObject............sqs:ReceiveMessage

Page 53: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Available as OSS - Repokid

https://github.com/Netflix/repokid

Page 54: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 55: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

ResultsLow-risk access reductionTransparent and versioned opsInnovation and high-velocity development without friction

Page 56: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 57: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Potential for “Controlled” AnarchyMicroservices YBIYRI Polyglot and multiple tech stacks Independent deployments

Intentionally decentralized governance leads to increased attack surface

Page 58: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 59: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 60: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Managing the Anarchy

Page 61: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 62: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

● Well-supported solutions from central teams● Clarifies and evangelizes successful patterns and

practices● Automated observation and evaluation of adoption● Provides a standard way of interfacing with engineering

teams about security● Uncover risk and reward operational excellence

The Security Paved Road

Page 63: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Security Paved Road (ex.)Example Solutions & Measures

Per-app IAM rolePer-app Security GroupNo secrets in codeInstance identityUpdated machine image

Page 64: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Security Paved RoadQuarterly Change Cycle

Commit to update once per quarter to pull in upgrades, library changes, and modifications to paved road components

Page 65: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 66: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Security Paved RoadSecurity Brain

Make our expectations, asks, and recommendations explicit and easy to navigate

Page 67: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Customized view for the user

Open security issuesRecommended

practices

Page 68: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Most security backlog is standard;explicitly limit bespoke/custom backlog

Page 69: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 70: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app
Page 71: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

In closing . . .

Page 72: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Overall TakeawaysStay attuned to trendsSimplify and standardize Favor transparent decisionsMeasure adoption and uptakeGet comfortable with tradeoffs

Page 73: Fast Forward - QCon · Wall-E Rate Limit IP Blacklist Authentication Authorization SigSci WAF Schema Check Schema Check Sec Headers DLP Pre-Filters Post-Filters. Results ... Per-app

Thank you!@chanjbs