Symfony2 for legacy app rejuvenation: the eZ Publish case study

It’s all about eXperience

Gaetano GiuntaDaniel Clements

The UK Symfony meetup

July 2015

Symfony2 for legacy app rejuvenation:

the eZPublish case study

2

1. The Story

3

Do you speak eeazee publish?

An Enterprise-grade Web Content management System

Figures•20K commits, first one in 1999•3200 php files in default install•Licensed both under GPL and commercially•45K community members, 200 partners, 500+ Paying customers!•Full-time employees!•Not your average php project

( This is not Product Placement, I swear )( And it’s not going to last more than 3 slides, I promise )

4





FEATURE

S

5





FEATURE

S

STAB

ILIT

Y

6



Figures•20K commits, first one in 2002•3200 php files in default install•Licensed both under GPL and commercially•45K commPaying customers!•Full-time employees!•Not your average php project


7

Do you speak eeazee publish? II

Features•Flexible content model

• Versioning• Translation

•Separation of content and presentation (templates & themes)•Wysiwyg editor; in-site editing; import from Office documents•Advanced search engine (based on SOLR)•Built-in webshop•Workflow engine•Fine grained RBAC•Cluster-mode for HA•RSS feeds, SEO, multimedia, geolocation, and much, much more…

8

Do you speak eeazee publish? II

Features•Flexible content model

• Versioning• Translation

•Separation of content and presentation (templates & themes)•Wysiwyg editor; in-site editing; import from Office documents•Advanced search engine (based on SOLR)•Built-in webshop•Workflow engine•Fine grained RBAC•Cluster-mode for HA•RSS feeds, SEO, multimedia, and much, much more…

9

Do you speak eeazee publish? III

Patterns• Entity-Attribute-Value•ActiveRecord•DBAL

• Supports MySQL, PostgreSQL, Oracle

•MVC• Data can be fetched from templates, making it HMVC in practice

•Templates• Bespoke language

10

If it works… Why change?Existing codebase is 10 years old

High maintenance cost• Started with no unit tests• Layers and roles not properly defined / documented

OOP before php had• Private/protected/static• Closures• Namespaces• Late static binding• And much more

Not built for an Ajax and REST world

11

If it works… Why change?Existing codebase is 10 years old

High maintenance cost• Started with no unit tests• Layers and roles not properly defined / documented

OOP before php had• Private/protected/static• Closures• Namespaces• Late static binding• And much more

Not built for an Ajax and REST world

12

If it works… Why change?

Existing codebase is 10 years old

Widely deployed• Well debugged• Pitfalls have probably been uncovered by now

Proven to scale

Well known:• Documentation improved over years• Tutorials, forums, blogs, aggregators• Active community of practitioners• Official training courses and consulting

13

If it works… Why change?

Existing codebase is 10 years old

Widely deployed• Well debugged• Pitfalls have probably been uncovered by now

Proven to scale

Well known:• Documentation improved over years• Tutorials, forums, blogs, aggregators• Active community of practitioners• Official training courses and consulting

14

Making (good) decisions

The first rewrite phase• Started beginning of 2011• Focused on Content Engine• Using in-house components (Apache Zeta Components)

The second rewrite phase• Started beginning of 2012• Base the CMS on an existing framework

15

Prerequisites - functional

• Durable Architecture• API stability guaranteed• Battle tested / not (only) the latest trend• Should still be there in 10 years

• Speed and Scalability

• Lively Community• Documentation available• Evolving and getting bugfixes

16

Prerequisites - technical

• Simple Integration with existing API

• HMVC (Hierarchical Model View Controller) stack

• Decoupled Components

• Dependency Injection

• Good Template Engine

• Extensible, Open, Reliable (TM ;-)

17

Candidates

• Java / non-PHP

• Home brew

• Zeta Components

• Zend Framework 2

• Symfony 2 (Full Stack!)

18

And the winner is…

• Java / non-PHP

• Home brew

• Zeta Components

• Zend Framework 2

• Symfony 2 (Full Stack!)

19

1. The Story2. The Challenge

20

Backwards compatibility

Product Management SCRUM Story:«As an existing user, I don’t want to be pissed off by a new

#@!$% version!»

21



#@!$% version!»

• 100% Data Compatible (same DB scheme)• Routing fallback to legacy controllers• Possibility to include legacy templates in the new ones• Access legacy code from Symfony controllers• Settings compatibility

• Bonus: access Symfony services from legacy modules

22



#@!$% version!»


• Bonus: access Symfony services from legacy modules

23

And while the devs are busy…

Things the boss has to account for• having to maintain 2 codebases for a long time =>• dev resources split =>• not a lot of new features for a while

24

And while the devs are busy…

More things the boss has to account for• Double infrastructure:

• Testing / Support / Packages system / …

• Training your support team / consulting team / sales team / partners

• Evangelization of the community• Possibility of a fork of the legacy version ('hostile'

community takeover)• Customers wanting to pay A LOT to stay on old version and

be supported longer

25

Making (good) decisions, part II

• Coming up with a perfect-out-of-the-box v2 means dedicating the whole company to the reimplementation

• By the time that version is ready, the market might have moved on to something else

Solution: take the Minimum Viable Product approach• Deliver v2 before reaching feature parity• Iterate quickly• Downside: keep the v1 around for longer

• But you have to provide the compatibility layer anyway, right?

26

Making (good) decisions, part II


#@!$% version!»


• Bonus: access Symfony services from legacy modulesChallenge accepted

27

1. The Story2. The Challenge3. A New Architecture

28

Dual-core architecture

29

Icing on the cake

The legacy version still works perfectly standalone

30


Packing it up

31

Directory layout

New Kernel: a standard Symfony App

Legacy Kernel is isolated in a subdirectory** = symlinks still necessary for making assets available

32

Bundles everywhere!

New Kernel: 100% in Sf bundles

33

Integrated plugin management

New “Kernel”: Bundles are installed via composer + config editing

Legacy Kernel: Extensions are installed via tarball unzip (manual) + config editing

Solutions:1. write a custom Composer installer plugin for Legacy

Extensions2. Allow an eZP Bundle to include a Legacy Extension

34

Plugin management

New “Kernel”: Bundles are installed via composer + config editing

Legacy Kernel: Extensions are installed via tarball unzip (manual) + config editing

Solutions:1. write a custom Composer installer plugin for Legacy

Extensions2. Allow an eZP Bundle to include a Legacy Extension

35


Front Controller

36

The old Front Controller

Not the nicest code ever written• All logic in the php file – no classes• 1100 lines of code• Sets up a huge amount of global variables• Incorporates tons of logic

• module redirection / re-execution logic• permission checking• running the setup wizard if needed• checking for pending database transactions • etc…

• Not clearly structured in setup / execute / teardown phases

37

The new Front Controlleruse Symfony\Component\HttpFoundation\Request;

require_once __DIR__ . '/../ezpublish/autoload.php'; // set up class autoloadingrequire_once __DIR__ . '/../ezpublish/EzPublishKernel.php';

$kernel = new EzPublishKernel( 'dev', true ); // extends the Sf Kernel

$kernel->loadClassCache(); // a method from parent

$request = Request::createFromGlobals();$response = $kernel->handle( $request );$response->send();$kernel->terminate( $request, $response );

The Kernel extends the Symfony HTTPKernel• It adds configuration for the Service Container

• It allows to register bundles via registerBundles()

38

Refactoring

Previous index.php had to be refactored• All logic moved into a class• 20 lines of code in index.php!• Better separation of setup and teardown of the environment

from execution of application logic

=>

• Can now be used as sub-controller• Or to run any legacy code from any New Kernel context

39

Bridging legacy code

It is sufficient to run it in a closure• with some performance penalty

40


Routing

41

Legacy RoutingeZPublish 4 uses a custom MVC implementation

• Front controller: index.php• Controllers are “plain php” files, properly declared• Url syntax: http:// site / module / controller / parameters• Parameters use a custom format instead of the query string• Virtual aliases can be added on top• For all content nodes, a “nice” alias is always generated by the system

• Good for SEO

Technical debt• No DIC anywhere (registry pattern used)• No nested controllers• No provision for REST / AJAX

• Implemented ad-hoc in many plugins (code/functionality duplication)

• Policies are tied to controllers, not to the underlying content model

43

Seamless integration

The ChainRouter from the CMF project is used

44

Seamless integration

The ChainRouter from the CMF project is used

45


Caching

46

Legacy cacheseZ Publish 4 has a complicated advanced caching system

For viewing content, cache is generated on access, invalidated on editing• TTL = infinite• When editing a content, cache is also invalidated for all related contents• Extra invalidation rules can be configured• Can be set up to be pregenerated at editing time (tradeoff: editing speed)• Cache keys include policies of current user, query string, custom session data

“Cache-blocks” can also be added anywhere in the templates• Expiry rules can be set on each block, TTL-based or content-editing based• Breaks mvc principle

Most powerful AND misunderstood feature in the CMS

47

Legacy caches: up to elevenHistorically: built-in “full-page cache” (stores html on disk)

Currently deprecated, in favour of using a caching Reverse Proxy• Performances same if not better• Delegate maintenance of part of the stack (Varnish, Squid)

Holy grail of caching: high TTL and support for PURGE command1. When RP requests page from server, he gets a high TTL => cache page forever2. When page changes, server tells to RP to purge that url from cache• Best reduction in number of requests to server while always showing fresh data• Downside: extremely hard to cache pages for connected users

ESI support as well• Hard to make efficient, as eZ can not regenerate an ESI block without full page

48

Legacy caches: up to elevenHistorically: built-in “full-page cache” (stores html on disk)

Currently deprecated, in favour of using a caching Reverse Proxy• Performances same if not better• Delegate maintenance of part of the stack (Varnish, Squid)

Holy grail of caching: high TTL and support for PURGE command1. When RP requests page from server, he gets a high TTL => cache page forever2. When page changes, server tells to RP to purge that url from cache• Best reduction in number of requests to server while always showing fresh data• Downside: extremely hard to cache pages for connected users

ESI support as well• Hard to make efficient, as eZ can not regenerate an ESI block without full page

49

Integration Wholesale ReplacementSymfony has one of the nicest caching systems for web appsBecause it adopts the HTTP model

HTTP Expiration and Validation are used• By setting caching headers on response object

Integrates with a Gateway Cache (a.k.a. Reverse Proxy)• Native (built-in, php)

$kernel = new Kernel('prod', false);$kernel = new HTTPCache($kernel);

• External (Varnish, Squid, ...)

Native support for ESI• Using {{ render_esi() }} in twig

50

1. The Story2. The Challenge3. A New ArchitectureReaping the

benefits

51

REST API

eZ4 has an incomplete REST API• Only functionality available: reading content• Based on Zeta Components MVC component

=>A new API has been implemented

• Full reading and writing of content is possible• All “dictionary” data is also available• Content-type for response can be JSON or XML (with an XSD!)• Fully restful

• Usage of all HTTP verbs (and then some: PATCH)• Respect http headers of request (eg: “Accept”)• HATEOAS: use urls as resource ids

• No separate request handling framework needed: pure Symfony routing

52

Database Access Layer

• eZPublish 4 comes with its own DBAL• The Content Engine was rewritten on top of Zeta

Components Database (before Symfony adoption)

• Dwindling development• Shaky support for Oracle, MS SQLServer, etc

• Most Symfony apps use Doctrine• Fast enough (not the ORM)• Good support for many databases

=>• Adopt Doctrine• Write a stub layer implementing Zeta DB API on top of it

53

Improving performances

• The queries generated by eZPublish 4 have been hand tweaked for years

• The new Content Engine not so much• Designed to run on non-sql storage• Many more layers of separation• Generated queries are suboptimal

=>• Introduce a Content Cache

• Built using Stash (www.stashphp.com)• Multiple storage backends• Fully tested

http://www.stashphp.com/

54

Improving performances

• The queries generated by eZPublish 4 have been hand tweaked for years

• The new Content Engine not so much• Designed to run on non-sql storage• Many more layers of separation• Generated queries are suboptimal

=>• Introduce a Content Cache

• Built using Stash (www.stashphp.com)• Multiple storage backends• Fully tested

http://www.stashphp.com/

55

Templating

• eZPublish 4 comes with its own template language• Syntax similar to Smarty• Compiler has very limited capabilities• Support in IDEs is spotty

• Symfony comes with TWIG template engine• Fast enough• Good way to support inheritance• Extensible

=>• Adopt TWIG• Implement custom helpers (filters / tag / functions)

56

Managing Assets

• eZPublish 4 comes with its own assets plugin• Minifies and compresses CSS, JS• Doubles as core plugin for all things AJAX• Hard dependency on JQuery and YUI versions• No support for LESS,SASS, etc…

=>• Adopt Assetic

• Currently working on: support for multiple site themes

57

Community Bundles

Replacing - and improving - existing functionality

• csrf token Symfony• Reverse Proxy integration FosHttpCacheBundle• Image manipulation LiipImagineBundle• Pagination WhiteOctoberPagerFanta• Menu building KnpMenuBundle• Breadcrumbs BreadcrumbsBundle• IO/storage League\Flysystem

58

Testing

eZ4 has an incomplete test suite• No unit testing• Selenium for functional testing• Jenkins as CI server

• Manual labour still involved• High maintenance cost

=>• Phpunit for unit testing

• 100% API Coverage is the goal

• Behat for all the rest• Travis for good platform coverage

59

Testing

eZ4 has an incomplete test suite• No unit testing• Selenium for functional testing• Jenkins as CI server

• Manual labour still involved• High maintenance cost

=>• Phpunit for unit testing

• 100% API Coverage is the goal

• Behat for all the rest• Travis for good platform coverage

60

The story so far

• First release: November 2012• 8 major releases• Legacy Kernel not included anymore since 2015• New modern Administration UI in beta• Will be launched as eZ Platform this fall• Good collaboration with the Symfony ecosystem

61

We’ll be here all night

THANK YOU<link to slideshare>

• @gggeek• @declemo• www.kaliop.co.uk

• www.ez.no• share.ez.no• doc.ez.no• github.com/ezsystems

62

And Now For Something Completely Different

Internet

Symfony2 for legacy app rejuvenation: the eZ Publish case study