Marko Heijnen CODEKITCHEN
Node.js to the rescue
Let Node.js do things when WordPress/PHP isn’t enough
Marko Heijnen• Founder of CodeKitchen
• Lead developer of GlotPress
• Core contributor for WordPress
• Plugin developer
• Organizer for WordCamp Belgrade 2015
• Technologist
Why something else?
Yes, you can build almost everything in WordPress
But is that the smart thing to do?
Can you trust on WordPress to be always stable?
This summer I started moving things away from WordPress
WP Central
WP Central• Showing download history
• Showing version usage history
• List all locales and their current state (need update)
• Showing contributors data (currently API only)
• Collects history of locale progress
• Getting checksums for plugins & themes
How it works• A lot of data handling by wp_remote_get
• Scrapping profiles.WordPress.org to read data
• Multiple API calls to api.WordPress.org
• Combine data so it can be presented
The problem• Most things happened through WP Cron
• Some things happens on the front end
• Resulting a load on the webserver that could and should be prevented
New server setup
Loadbalancer
Memcached
Elasticsearch
MariaDB
New server setup
Micro services
Webserver 1
Webserver 2
Thumbor
Public Private
Service server• MariaDB as database
• Memcached as object cache
• Moving to Redis when the PHP7 version is out
• Elasticsearch to make search better/faster
The microservices server• Handles all cronjobs for the network site
• Node.js services running for WP Central
• Like getting checksums for plugins/themes
• Soon merging other WP cronjob calls for getting all the stats
Microservices
Microservices• Microservices are small, autonomous services that
work together
• Small, and Focused on Doing One Thing Well
Benefits• Different services can use different programming
languages
• High level separation
• If WordPress breaks, the services still keep running
• Ease of Deployment
• Scale services that require more resources
Benefits• In general they have an (REST) API
• Reusable
• Other microservices could call the service to run a task
Node.js
What is Node.js• JavaScript platform
• Uses an event-driven, non-blocking I/O model
• Lightweight and efficient
• Ideal for real time application
• Lot’s of modules you can use
• Manage with NPM - https://www.npmjs.org
Why to use it• Internal webserver
• No configuration needed outside it’s code base
• You could use nginx as a proxy but not needed
• You get what you see approach
Who is using it• Netflix
Need to know modules• Express / Restify -> Webserver
• Socket.io -> Real time
• Request -> Doing internet requests
• async -> async calls with callback
• mysql -> MySQL driver with Pool support
• node-cmd -> Command line
Checksums for plugins/themes
What is does• Request checksum of a certain version from a
plugin or theme
• Download the zip and unzips it.
• Reads it in memory and get the checksum per entry
• After everything is retrieved stores it in MySQL
Modules used• Build in modules
FSCrypto
• NPM ModulesExpress MySQLRequestYauzl for unzipping
• And build a little queue class
Calling the API• http://wpcentral.io/api/checksums/plugin/tabify-
edit-screen/0.8.3 (REST API)
• Calls nginx by IP (10.10.10.10) which handles as a fallback when the node.js application is down
• nginx calls then internally the node.js application like proxy_pass http://127.0.0.1:8080
API calls• /plugin/:slug/:version
http://10.10.10.10/checksums/plugin/:slug/:version http://wpcentral.io/api/checksums/plugin/tabify-edit-screen/0.8.3
• /theme/:slug/:version http://10.10.10.10/checksums/theme/:slug/:version http://wpcentral.io/api/checksums/theme/twentyfourteen/1.2
nginx ruleserror_page 404 @404;error_page 500 @500;error_page 502 @502;
location @404 { internal; add_header Content-Type application/json always; return 404 '{ "status": "Route Not Found" }'; }
return 500 '{ "status": "Service is down" }';return 502 '{ "status": "Service is down" }';
Going over the code
Basic setup// set variables for environment var express = require('express'), app = express(), mysql = require('mysql'), request = require('request'), fs = require('fs'), crypto = require('crypto'), yauzl = require("yauzl");
MySQL connectionvar pool = mysql.createPool({ connectionLimit : 10, host : ’10.10.10.11’, user : 'checksums', password : 'checksums', database : 'checksums' });
pool.on('enqueue', function () { log_error('Waiting for available connection slot'); });
Serverapp.listen(4000);
app.get( '/plugin/:slug/:version', function(req, res) { if ( ! queue.add( 'plugin', req.params.slug, req.params.version, res ) ) {
res.json({ 'success': false, 'error': 'Generating checksums’
}); }
});
Server 404app.use(function(req, res, next) {
res.status(404).json({ 'success': false, 'error': "Route doesn;'t exist”
}); });
Lets check the rest
Starting the server• The default way is: node server.js
• The production server way could be: pm2 start server.js -u www-data --name “Cool service”
The new situation
The new situation• No more unneeded logic in WordPress
• WordPress simple pipes the calls
• Small services that replacing it
• APIs can easily be reused
• Pushing new updates becomes easier
• Currently no caching but easily added
Other things you could use node.js for
See my presentation: Extending WordPress as a pro
Describing an idea of using Socket.io with WordPress
Other ideas• Scheduling tasks or url
calls
• Build a central cache point for external sources like getting tweets
• Real time support
• git2svn sync
• Backup service
• Real time logger
• Perform heavy tasks
Thank you for listening
Questions?@markoheijnen
markoheijnen.com
codekitchen.eu