Real-Time Web AnalyticsOn

Amazon Web Services

@TheDeveloper

Google CampusFebruary 2013

Real-time web analytics

James GillCEO + Co-founder

Geoff WagstaffCTO + Co-founder

James TaylorDeveloper + Co-founder

LiveStats

In 2009, our Server melted.

Conventional Hosting

• Contractual lock-in. Fixed term subscription

• Laborious and slow to provision resources

• Restricted

• Dedicated, but not bespoke

• Boring

Business constraints

• Experimental technology

• Agility

• Low budget

• Inexperience

• High barrier to entry in market

GET TO THE CLOUD!!!11!!one!

Why AWS?

Diverse + growing range of services

EC2S3

CloudFront

CloudFormation

CloudWatch

RDS

Route53IAM

CloudSearch

ElasticBeanstalk

ElastiCache

DynamoDBSES

SNS

SQS Glacier

RedShift ElasticTranscoder

Data Pipeline

StorageGateway

VPC

SWF

Why AWS?

Flexible & Scalable

Why AWS?

Pay-as-you-use Pricing

Why AWS?

Platform agnostic

Why AWS?

Reputable

Challenges

• More moving parts

• Daunting. Requires new skills and role changes

• Architecting for cloud

• Deployment - resources, software, data

• Automation - scaling, failover, backup

Advantages

•Full control - freedom, bespoke

•Fosters good system design paradigms & avoids complacency

•Abundant tools, services, community + documentation

•Lowered barriers to entry: infrastructure for anyone

•Reduced time to market

Gap Year project: Trends

Angels (2010)

Series A (2011)

6 Team Members

Average age: 21

25,000 sites and counting

Traffic

10s Thousands

100s Thousands 100s Millions

99.9%

concurrent visitors pageviews/month

Requests/second Uptime

Our Stack (tidier)

Load Balancer

Reverse Proxy

Services

Data Stores

ELB

Redis, MongoDB,MySQL (RDS), EBS, S3

Service-oriented architecture:

Tiered, decoupled, load-balanced

clusters

Fault tolerance, failover, backups

Auto-scaling. Spot instances

Collectors, processing, API,

Traffic

AWS Services: EC2, S3, RDS, Route53, CloudWatch, SNS, IAM

Applications, Website

How we do...Monitoring

Ganglia

Graphite - http://graphite.wikidot.com/

ServerDensity

CloudWatch

PagerDuty

Best PracticesScale only when necessary

Decouple using APIs + messaging

Automate wisely

Monitor and log everything

Destroy Everything

Thanks!

Geoff Wagstaff

@TheDeveloper

https://engineering.gosquared.com