1

Case Study, Government of Flanders

Tomas Fülöpp (Vacilando)

Decoupled & Uprooted

2

Decoupled & Uprooted

Builder Track: Case Study

Building a Challenging Web Platform at the Flemish Government

Tomas Fülöpp (Vacilando)

3

Your Speaker

What to Expect

4

Tomas Fülöppa.k.a. Vacilando

ICT architect at the Flanders Information AgencyWorking for the Government of Flanders in Belgium since 2012

Drupal aficionado since 2006„Vacilando“ on drupal.org

Born in Czechoslovakia, Belgian by choice, European at heart

https://www.drupal.org/u/vacilandohttps://www.linkedin.com/in/vacilando/

5

The Next ½ Hour

Your speaker

Government of Flanders

8 years of Drupal 7

Time to Reinvent

Context

Building Blocks

Development

Takeaways

Thanks Pages

Questions?

6

Government of Flanders

Belgium, Europe

7

BelgiumEurope

That speck on the map

Federal constitutional monarchy divided into three highly autonomous regions:

• Flanders, • Wallonia, • and Brussels.

Famous for:

• Beer• Chocolate• French fries• Dries Buytaert

8

FlandersBelgium

Tiny by any standards• 6.6 million inhabitants• 13.5 thousand km²

Or to put it in U.S. context:• about the size of Puerto Rico and• with the population of Indiana

9

Flemish Public Administrationa.k.a. Government of Flanders

… or Flemish Government

Prepares and executes policies of the Flemish Region and the Flemish Community.

Main organs• Flemish Parliament• Flemish Government

Main public portalhttps://www.vlaanderen.beResponsibility of the Flanders Information Agency

That’s what we’re talking about here!10

8 Years of Drupal 7

www.vlaanderen.be

11

Early Adopterof Drupal 7

Drupal 7 was released in 2011The Flemish Government embraced it as CMS of choice still in the same year

Main citizen portal www.vlaanderen.be in D7 was a great success after the previous CMS(FatWire).• Modern, comfortable and fast editing• Separate interfaces thanks to Domain Access• Plenty of data source integrations• Lots of custom code but reasonably clean• Several redesigns over the years• Inspirational for great many websites around the Government (also as open source)

• Some 80k pages, permanent editing team keeping the information current and accurate• 2 million sessions per month (3.5 million page views)

12

Vlaanderen.be in 2011

13

Vlaanderen.be in 2018

14

8 Years of Drupal 7Time for a Next Step

Usual Drupal 7 pains, especially:

Rigid; one large monolith• Editing environment + front-end + data synchronized from external sources• Clumsy and risky deployment (manual configuration -> Features); limited speed of

development / time to production

Difficult performance scaling• Non-cached performance dismal = particularly frustrating for the Editorial Team (front

blazing fast thanks to Varnish and smart invalidation of all related pages)

Simply old• After 8 long years time to retire

15

Time to Reinvent

From Portal to a Web Platform

16

From Portal to a Web Platformwww.vlaanderen.be

Much more than yet another redesign.

Transition from a simple portal to a Web Platform• able to maintain and publish information from any and all Government

departments (API-First, No Wrong Door)• for citizens and professionals.

17

Requirements

• Scalability: front-end, editing environment. Be able to grow to an information platform forany and all government entities. Accommodate many diverse Editorial Team.

• Flexible content model. Editors must be able to use a wide variety of content structures.• Maintainability. Microservices over monoliths. Continuous integration and deployment.

• Accessibility. Everybody needs to be able to get the information they need.• Performance. Best practically attainable.• Security. Uncompromising.

• Reuse. Avoid duplication of content, code, systems. Use the open source alternativewherever possible – starting with Drupal.

• Integrate. Respect existing integrations.

• Sustainability. Emanate stability, reliability, trustworthiness to the citizens. Not a short-termproduct or campaign site.

• Standards. Commitment to industry as well as organizational standards.

18

Fundamental Questions

1. Is Drupal - still - the best CMS?• After 8 years Drupal perceived as “old technology”.• Appetite for something radically different, for the cutting edge.• Drupal 8 seemed nice, but not convincing (for non-Drupalers!)

2. Go decoupled (API-first)? Or is it perhaps just a hype?

3. Can we decouple (uproot) the content as well? Can we use Drupal forediting but not store the data in its database?

19

Fundamental DecisionsBeginning of 2018

1. CMS: We continue to trust Drupal (8)• Review of competitors (Adobe Experience Manager, Contentful, GraphCMS, Prismic , WP, etc.)• Drupal still the most feature-complete and mature• Lots of expertise and enthusiasm in-house

2. Front-end: Go decoupled, using data from Drupal API• Added complexity, risk of hype but attraction of modularity / freedom; more than one front-end• Huge enthusiasm from the front-end team

3. Remote / uprooted data sources• We concluded we could not detach all content; it would make little if any sense to use Drupal

(decided with contribution of Wim Leers)• We can however use remote sources of data and manipulate them just as if they were stored in

Drupal. Not well documented but D8 can do this!• Use an asset library (DAM) instead of Drupal for storing images, videos, PDFs, etc.

4. Set up and move to a new, scalable hosting system.20

Context

Integrate & Reuse

21

Existing IntegrationsContext

Even a green field (project) is part of a wider ecosystem. A.k.a. “brown field”.

1. Dependent on a number of data sources• Organizations• Publications• Press releases• Vacancies• Assets (DAM)• ...

2. Data source for several existing applications• Call Center Knowledge Base• Catalogue of Government Products• Reporting• …

22

Library of Web Components

Library of Web Components (specific for the Flemish Government)• CSS & JS snippets (and also VueJS)• Conform to the corporate identity of the Government• Mobile-first• Tested for accessibility• Best design practices

Available for use by other Flemish Government entitiesDocumentation at https://overheid.vlaanderen.be/webuniversum/v3/

23

Global Header & Footer Widgets

Not Web Components as such but “widgets”Highly configurable JS applications

Conform to the corporate identity of the Government

Easy integration (embed script or SSI)

Gateway to a growing secured area where each citizen can findpersonalized and private information related to the Government (familyinformation, degrees obtained, status of subsidy requests, etc.)

24

Global Header & Footer

25

Global Header & Footer

26

Building Blocks

CMS, Front, Content API

27

I. Heart of GoldDrupal 8 as the CMS

Content structure: heavy use of Paragraphs• Much more flexible than just using plain content types• Alternatives: Bricks (underdeveloped), or within WYSIWYG

Internal search: Search API with Solr backend• Alternative: Elasticsearch, but the integration seemed to still lag behind

API: GraphQL (https://www.drupal.org/project/graphql). Could as well have been REST or JSON.POST requests, yet cached by Varnish → very little load on Drupal.

Site API: Decoupled Router (https://www.drupal.org/project/decoupled_router) provides anendpoint that will help you resolve path aliases and redirects for entity related routes.

Internal caching: Redis• Proven slower than DB caching on HA infrastructure (network latency). Switched off to

increase performance!

28

Remote / Uprooted Data SourcesExternal Entities

Module External Entities allows you to seamlessly integrate datasets from external databases:https://www.drupal.org/project/external_entities

While the content lives external, Drupal sees those remote entities as internal. This makes itpossible to alter the field displays, add references, comments, path aliases, share buttons, andmore.

New, significantly reworked branch 8.x-2.x by Raf @rp7, with contributions from Wim Leers.See https://www.drupal.org/project/external_entities/issues/2995140

Featured in a presentation at Drupal Europe in Darmstadt in September 2018, checkhttps://www.drupaleurope.org/session/how-cope-external-entities

29

Drupal ContributionsBy Our Team

External entities — improved and refactored branch 8.x-2.x by Raf @rp7,with contributions from Wim Leers• https://www.drupal.org/project/external_entities

Entity Usage Integrity — new module by Maciej @gugalamaciek• https://www.drupal.org/project/entity_usage_integrity• Protects integrity of references based on entity usage, preventing

inaccessible references

Issuu Viewer – new module by Tamar @tamarpe• https://www.drupal.org/project/issuu_viewer

30

Drupal ContributionsBy Our Team

Patches to a host of d.o. projects:• https://www.drupal.org/project/entity_browser• https://www.drupal.org/project/bynder• https://www.drupal.org/project/entity_usage• https://www.drupal.org/project/decoupled_router• https://www.drupal.org/project/openid_connect• https://www.drupal.org/project/anonymous_redirect• https://www.drupal.org/project/unique_field_ajax• https://www.drupal.org/project/field_group• https://www.drupal.org/project/inline_entity_form• https://www.drupal.org/project/purge• ...

Full list of contributions:• https://www.drupal.org/government-of-flanders

31

II. Decoupled Front-endVueJS & NuxtJS

Opinions divided between ReactJS and VueJS

VueJS for client side selected mainly because of• Further reuse that technology choices within the Agency• The Front-end Team existing experience and huge motivation

NuxtJS for server-side rendering (analogous to NextJS for React)Vuex for state management (analogous to Redux for React)

CDN (Akamai) with smart invalidation. Not just HTML but also JSON data.

Largest, but just one of the front-ends possible thanks to the API.

32

Front-end Caching

Caching at the edge: Akamai• World class CDN, many more edge locations than competitors• Smart invalidation = custom implementation• Cache tags possible although rather new

WAF (preventing attacks such as DOS, XSS, SQL-injection, ...) – on the edge

L7 load balancer – on the edge

Fastly was a viable alternative, mainly because it’s in essence a Varnish at the edge, but has less locations and support options.

33

Performance TestingUsing CDN

34

Why Do We Need WAFWatching For Suspect Requests

35

III. Remote / Uprooted Data Sources

New building block we call Content API

• Aggregation and equalization of data coming from the individual remote data source APIs

• Caching (slow or unstable APIs) in MongoDB (vs memory – not permanent, or Elasticsearch)

• Synchronization scripts (common base but tailored improvements). Particular complicationswith data that need to be enriched by Drupal.

• Filtering, sorting (not available in all APIs)

• Microservices, separation of concerns, modularity. Small blocks, each with clear purpose.

• For the Content API is the decoupled front-end just one of the data consumers. Front-end, Drupal, and other integrators talk to one single interface

• For the Content API is Drupal just another data source

• Output using GraphQL instead of REST or JSON

Messaging queue using RabbitMQ36

Open-source data query language for APIs.Ideal for heterogenous sources.

Here‘s a simple query (to Content API, not Drupal) that provides:

• Publications (an external source) with matching title

• Total number of items in the result list• Specific items• Publication date item will be formatted as needed

Use of GraphQL

37

Example GraphQL QueryResults

38

Digital Asset Management (DAM)

Images, videos, and also files live in the DAM. Use Drupal files only for generated stuff.

One of the Content API sources.

Chosen for Bynder• Existing modules not suited for our use case (no need to sync to media entities) → custom

module• Inadequate support for image derivatives (limited number, batch-created, hardcoded specs)

Cloudinary chosen as a solution for the front end• Significant complexity due to synchronization from Bynder to Cloudinary (technical debt)

39

Building Blocks

40

Hosting Infrastructure

Proudly using Amazee.io hosting

• Very scalable, based on managed Kubernetes and OpenShift (EKS soon)• Able to run virtually any application thanks to containerization (far more

than just Drupal hosting). Able to mix containers with cloud services (egwe use AuroraDB instead of MariaDB containers)

• Can be run locally, on premise or in the cloud (we use AWS)• Own choice of infrastructure, you choose the specs• Deployment based on GitOps along the whole development path, from

local through testing to production• Excellent support, invaluable for deep Drupal-specific hosting issues• Enormous flexibility; rather a partnership than just a hosting service

41

Development Happens

April 2018 – March 2019

42

Eagle has LandedMarch 21, 2019

Two major events on one day

1. Spring is here! (in the Northern Hemisphere only, sorry)

2. Launch of the Web Platform at www.vlaanderen.be @ 21:55 CET

45

Eagle has Landedwww.vlaanderen.be

46

For the FutureWhat We Did Not Do

Drop Bynder and choose Cloudinary or an even better DAM with full capabilities

Add a non-public section to the same website (for now the whole site is publicly accessible)

Preview and visual editing (NB: in the decoupled front-end)

Robust roles and permissions management system

Writing to (some) remote sources through the Content API

Improve Content API (the aggregator) so that it becomes a full-fledged data source

Implement a powerful custom search across the diverse sources (now Google CSE)

Write many more tests (unit testing, end-to end scripts).

Remove technical debt built up as the deadline approached.

47

Takeaways

Well Done | Bleeding Edge Can Hurt

48

TakeawaysDecoupled Drupal

Advantages• Your Drupal team does not need to worry about front-end• API out of the box or using mature modules• Fast, and the API can easily be cached for even better performance

Disadvantages• Some of basic Drupal functionality and contrib modules won‘t work with decoupled front

simply by definition• Drupal also used for routing (Site API); so it’s more than simply a data source. However, there

seem to be no good routing solutions for this in NuxtJS (perhaps generally in front-end JS apps), so it may be the best solution (Egg of Columbus).

Watch out when using Redis or Memcache on HA infrastructure. Measure if it actually speeds up the system.

Use Drupal core and quality contrib to the fullest. Deviate only if necessary as it will cost you resources.

49

TakeawaysDecoupled Front

Advantages• Separately maintainable, lighter deployments• Easily replaceable with every new hype – or business requirement• Your front-end team will love you forever (freedom!)• No worries about finding good front-enders that are familiar with Drupal theming

Disadvantages• A LOT of added complexity to maintain• The most natural things, like viewing a saved page, cannot be taken for granted any more

Don‘t decouple unless you are sure that the advantages dwarf the disadvantages for your use case.

Watch out what data manipulation is in the front end, and which stays in the aggregator.

Invest not just in a good CDN but also WAF. We’ve been seeing attacks from the very first day.

50

TakeawaysRemote / Uprooted Data Sources

Advantages• Perfect reuse, no mirroring of external data in Drupal• Easily doable thanks to External Entities 2.x

Disadvantages (over native Drupal)• Every API is different and we had to creatively patch the holes (in sync scripts)• Selective cache invalidation must be custom built• More complex: roles and permissions, internal search, sitemap generation, multilingual

setup, ...• Reading still relatively simple as opposed to writing

Beware of Bynder as DAM. Cloudinary may be a better solution when it comes to front-end.

Complications in cases where external content is not used directly but as a part of Drupal entities (enrichment).

51

TakeawaysFinal Reminders

Remember the ecosystem. No green field exists in isolation.

Follow architecture ideals but don‘t become a purist (common sense, prioritizing).

Respect your team‘s expertise and motivation.

Invest heavily in reuse: code, data, architecture, processes, teams. The open source community is your extended team.

Stay agile. The waterfall is poisoned.

“Bleeding edge can hurt you” (credit: Dirk Stevens!)

Dare to dream. Dare to fail. Dare.

52

Thanks Pages

Questions Welcome

53

Thanks to Our Wonderful Team

+ all those I forgot!

54

Thanks for Your AttentionQuestions?

Questions, comments, or suggestions welcome!

Happy to bring you in contact with the tech leads for deeper discussions.

Contact me atTomas.Fulopp@kb.vlaanderen.behttps://www.drupal.org/u/vacilandohttps://www.linkedin.com/in/vacilando/

Visit https://www.vlaanderen.be/

And visit Flanders, Belgium!

55

Join us for

contribution opportunitiesFriday, April 12, 2019

9:00-18:00Room: 602

Mentored

Contribution

First Time

Contributor

Workshop

General

Contribution

#DrupalContributions

9:00-12:00Room: 606

9:00-18:00Room: 6A

56

What did you think?

Locate this session at the DrupalCon Seattle website:

http://seattle2019.drupal.org/schedule

Take the Survey!

https://www.surveymonkey.com/r/DrupalConSeattle

57