Scaling Drupal“UnDrupaling Drupal”

Michael T. SmithThrillist

October 4, 2011

Drupal 5 to 6• Two month rebuild;

October to December 2010• Converted >50 modules to D6;

20+ new modules• 8gb+ database:

2.3m nodes, 3m+ user records

Our Mindset:“UnDrupaling Drupal”• Ethan Kaplan, former VP Emerging Tech

@ Warner Bros. Entertainment Group• 120 artist sites

• Out of the box: Not able to scale• Huge issues with loading node objects

from DB• Hooks are nice, but can allow too much to

load

MongoDB• MySQL & node_load

• Every module potentially called• Huge overhead• Mongo:

• Fast lookup, fast results• Not for user records

MongoDB

MongoDB

Memcached• Cache All The Things!• Memcached with bins set up for page,

content, mobile data and blocks

• Default page and block cache, kinda• Edition and Logged In states

• Special keys for edition content• Case:

Viewing NY article from ATL “edition”

SCSS &Clean Theming

• Holy disgusting code, Batman• Old jQuery version—updated• Custom theme functions

• Preprocessors to clean• Compass:

• Auto-generated sprites• Auto-minimized code• UberTags

Users & Stats• Huge data, huge concern, huge codebase• Extremely custom marketing and promo

system for signups

• No default signups• Customized modules for mixing

forms/surveys• Reporting is a challenge—separate

systems for reporting and data dumps, all based “around” Drupal

API• Two servers:• One for people• One for machines

• Services module for JS requests• Future? DB as datastore, Drupal only for

serving content to people

Idempotence• Make as much as possible repeatable

without harm• Not perfectly idempotent, but close

enough• Mainly user systems

• Thrillist/JackThreads/Rewards subscriptions updates are queued

People• Team• RackSpace• MongoHQ

Thanks!(questions?)