Butter Web Browsing with Margarine

margarine dissection

alex brandtsr developer

wayne wallscloud evangelist

Wednesday, August 21, 13

2

WHAT IS MARGARINE?


3

3 PARTS


4

BLEND

TINGE

SPREAD


5

BLEND

TINGE

SPREAD


6

BLEND

TINGE

SPREAD


7

WHAT DO WE CARE ABOUT?


8

SIMPLICITY!


9

DESIGN DISCLAIMER


10

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE


11

BUT WHAT DOES IT ALL MEAN?!


12

DATA TIER


13

WHAT KIND OF DATA DO WE NEED TO STORE?


14

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes


15



16



17



18



19

DATA TIER DECISIONS


20

Where to Store the DataHow to Organize the DataHow to Manage UsersAPI Strategy


21



Data Schema

•articles• url

• text

• tags-index

• notations

• location

• note

• votes

• created_at-index

• etag

• parsed_at

•users• username-unique index

• email

• name

• hash (md5)

22

•subscribers• uuid4-user

• uuid4-bookmark

• subscribed_at


23



24



25

APPLICATION TIER


26

APPLICATION TIER ACTION!


27

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS & ARTICLES


28




29




30

APPLICATION TIER DECISIONS


31

HOW IS DATA PROCESSED?WHERE IS DATA PROCESSED?


32




33

A LOT OF WORK HERE...


34

SPLIT THE APPLICATION TIER!


34

SPLIT THE APPLICATION TIER!

BLEND SPREAD


35

ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES


BLEND

SPREAD


36

HOW DOES ONE SPLIT THE APPLICATION TIER?


37

MESSAGE QUEUE, BUT WHICH?


38

NOW THAT WE HAVE THAT FIGURED OUT


39

BLEND

SPREAD



RabbitMQ


40

BLEND

SEVEN CONSUMER/WORKER SERVICES

Create Ar(cleUpdate References Sani(ze HTMLCreate UserUpdate UserPassword E-‐mailPassword Change


41

BLEND

SPREAD



RabbitMQ


42

WEB TIER


43

WHAT ARE WE SERVING OUR CUSTOMERS?


44

WEB PORTAL FOR...


44

WEB PORTAL FOR...

Logins


44

WEB PORTAL FOR...

LoginsPassword Resets


44

WEB PORTAL FOR...

LoginsPassword ResetsViewing and Tagging of Articles


44

WEB PORTAL FOR...

LoginsPassword ResetsViewing and Tagging of Articles Submission of Articles


45

WEB TIER DECISIONS


46

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY


47

Ubuntu 12.04python; python-flask

nginx in front of uWSGI


48


LOAD BALANCINGDNS

CONTENT DELIVERY


49

CloudFiles as Key/Value StoreObjectRocket as Document Store

Exceptional.io Redis2Go as Key/Value Store


50

CloudFiles as Key/Value StoreObjectRocket as Document Store



51

CloudFiles as Key/Value StoreObjectRocket as DocumentStore



52


LOAD BALANCINGDNS

CONTENT DELIVERY


53

RACKSPACE


54


LOAD BALANCINGDNS

CONTENT DELIVERY


55


56


LOAD BALANCINGDNS

CONTENT DELIVERY


57

Tinge to CloudFilesSEO Management


58

Tinge to CloudFilesSEO Management


59

CANONICAL ARCHITECTUREWITH PILLARS


LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize

OBJECTSTORE


LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize

OBJECTSTORE Parallel

Modular

Horizontally scalable

Agile

Secure


LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize


Modular


Agile

Secure


LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize


Modular


Agile

Secure


LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize


Modular


Agile

Secure


KEY/VALUESTORE

OBJECTSTORE

LB

FRONT END API

MQ

CONSUMER

LB

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize


Consumers

•create user

•password user


•email user

•create article

•article sanitize

Parallel

Modular


Agile

Secure


66

https://github.com/raxsavvy/margarine

Contribute




Technology

Butter Web Browsing with Margarine