Surviving a Plane Crash, a NU.nl case-study

Peter C. VerhageJune 12th 2009

Surviving a Plane Crasha NU.nl case-study

Surviving a Plane Crash

NU.nl

• most popular Dutch news website

• 10th anniversary

• 4.5 million unique visitors each month

• 7-15 million page views each day

Me

The plane crash

• 20.2 million page views

• 1.83 Gbit/s traffic

2

Surviving a Plane Crash

CMS

Front-end

Performance

QA

3

CMS

4

CMS

100% custom

PHP 5.1 / MySQL 5

ATK framework

• in-house developed

• Open Source (LGPL)

• specializes in CMS-like applications

Prototype / Scriptaculous

5

CMS

6

vs

CMS

7

CMS

8

CMS

9

CMS

10

CMS

11

Front-end

12

Front-end

Multiple sites (NU, NUvideo, NUfoto, NUsport, ...)

CodeIgniter framework

• PHP 4 :(

• lightweight MVC

• extensible

Extensions

• Snippets

• Globalization (i18n / l10n)

• Code sharing between sites

13

Front-endSnippets

14

Front-end: Snippets

15

Front-end: Snippets

16

header

Front-end: Snippets

17

navigation

Front-end: Snippets

18

sidebar

Front-end: Snippets

19

other news

Front-end: Snippets

20

article

Front-end: Snippets

snippets

• are like ZF partials, but more powerful

• contain logic that doesn’t belong in the template / view

• have full access to the model

• render a view for their result

• so a lot like a controller action

• but more re-usable

21

abstract class Snippet { public function __construct($params); protected function getParam(); protected function renderView($name, $params); public abstract function render(); public function getPath(); ...}

Front-endGlobalization

22

Front-end: Globalization

Dnews.de; german version of NU.nl

Same code base, different configuration

Default CodeIgniter globalization not sufficient

Custom globalization consists of:

• translations

• template overrides

• snippet overrides

• css / javascript overrides

• routing overrides

23

Front-endCode Sharing

24

Front-end: Code Sharing

Good developers are lazy developers

Lazy developers are good developers

Avoid code duplication

Each site has its own code base

One shared code base

Class not found in site code base? → Try shared code base

Small variations can be tweaked in the site configuration

25

Performance

26

Performance

Static HTML

Going more dynamic

A Layer of Varnish

Hardware setup

27

PerformanceStatic HTML

28

Performance: Static HTML

Nothing beats static HTML

Snippets; the problem

Task Scheduler

Degradation

29

PerformanceGoing more dynamic

30

Performance: Going more dynamic

NUfoto / NUvideo

Lists ordered by date

Problem

• latest photo / video is pushed on top

• pagination continuously changes

Solution

• use a proxy server

• use Memcache

• generate HTML for first few pages

31

Performance: Going more dynamic

Showing the logged in user

Problem

• every page is customized for the user

Solution

• set cookie with display data

• use JavaScript to inject

• don’t cache real user specific pages

• never trust the cookie!

32

PerformanceA Layer of Varnish

33

Performance: A Layer of Varnish

Open Source reverse proxy server

Serves from memory

Blazingly fast

Snippets; the solution → ESI

• Edge Side Includes (W3C standard)

• In-memory

• Remember Snippet::getPath()?

• Example:<esi:include src=”/includes/site/sections/algemeen.html”/>

34

PerformanceHardware setup

35

Performance: Hardware setup

2 back-end servers (apache / master db), only 1 active

4 front-end servers (apache / slave db)

4 front-end Varnish proxy servers

4 media Varnish proxy servers

Juniper load balancers

36

Questions ?

37

Thank you!

Contact details:

E-mail: peter@ibuildings.nlSkype: peter-ibuildings

Links & sources

39

http://www.nu.nlhttp://www.nufoto.nlhttp://www.nuvideo.nlhttp://www.nusport.nlhttp://www.dnews.de

http://www.atk-framework.comhttp://www.codeigniter.comhttp://varnish.projects.linpro.no