Upload
chris-richardson
View
1.339
Download
2
Embed Size (px)
DESCRIPTION
JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
Citation preview
@crichardson
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com
NodeJS: the good parts? A skeptic’s view
@crichardson
Presentation goal
How a grumpy, gray-haired server-side Java developer discovered an appreciation for NodeJS and JavaScript
@crichardson
VIEWER DISCRETION IS ADVISED
WARNING!
@crichardson
1982 1986
RPG 3 BCPLPascal
C
About Chris
1983
LispWorks
1980 1984 1985 1987 1988 19891981
Z806502Assembler
Basic
@crichardson
C++
EJB
1992 199619931990 1994 1995 1997 1998 19991991
@crichardson
CloudFoundry.com
2002 200620032000 2004 2005 2007 2008 20092001
@crichardson
?About Chris
2012 201620132010 2014 2015 2017 2018 20192011
@crichardson
Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
What’s NodeJS?
Designed for DIRTy apps
@crichardson
Growing rapidly
Busy!
@crichardson
NodeJS Hello Worldapp.js
$ node app.js$ curl http://localhost:1337
http://nodejs.org/
Load a module
request handler
No complex configuration: simple!
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
Dynamic and weakly-typedDynamic:
Types are associated with values - not variables
Define new program elements at runtime
Weakly typed:
Leave out arguments to methods
Read non-existent object properties
Add new properties by simply setting them
@crichardson
JavaScript is object-oriented> var fred = {name: “Fred”, gender: “Male”};undefined> fred.name“Fred”> console.log("reading age=" + fred.age);reading age=undefinedundefined> fred.age = 99;99> fred{ name: 'Fred', gender: 'Male', age: 99 }> delete fred.agetrue> fred{ name: 'Fred', gender: 'Male' }
Unordered key-value pairs
Keys = properties
Add property
Delete property
@crichardson
overrides
JavaScript is a prototypal language
__proto__name “Chris”
__proto__sayHello function
... ...
inherited
Prototype
Person
Chris
“CER”nicknameobject specific
@crichardson
Prototypal code$ node> var person = { sayHello: function () { console.log("Hello " + this.name); }};[Function]> var chris = Object.create(person, {name: {value: "Chris"}});undefined> var sarah = Object.create(person, {name: {value: "Sarah"}});undefined> chris.sayHello();Hello Chrisundefined> sarah.sayHello();Hello Sarahundefined> chris.sayHello = function () { console.log("Hello mate: " + this.name); };[Function]> chris.sayHello();Hello mate: Chrisundefined
Not defined here
create using prototype properties
@crichardson
JavaScript is Functionalfunction makeGenerator(nextFunction) {
var value = 0;
return function() { var current = value; value = nextFunction(value); return current; };
}
var inc = makeGenerator(function (x) {return x + 1; });
> inc()0> inc()1
Pass function as an argument
Return a function closure
@crichardson
But JavaScript was created in a hurry
The ‘Java...’ name creates expectations that it can’t satisfy
Fake classes: Hides prototypes BUT still seems weird
global namespace
scope of vars is confusingMissing return statement = confusion
‘function’ is really verbose
‘this’ is dynamically scoped
Unexpected implicit conversions: 99 == “99”!
truthy and falsy values52-bit ints
Dynamic + weakly-typed (+ event-driven) code
+ misspelt property names
⇒
lots of time spent in the abyss
Essential: Use IDE integrated with JSLint/JSHint + tests
@crichardson
Prototypal languages have benefits BUT
Developers really like classes
JavaScript prototypes lack the powerful features from the Self language
e.g. Multiple (and dynamic) inheritance
http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html
@crichardson
Verbose function syntax> var numbers = [1,2,3,4,5]> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })[ 4, 16 ]>
scala> val numbers = 1..5scala> numbers filter { _ % 2 == 0} map { n => n * n }Vector(4, 16)
VersusPrelude> let numbers = [1,2,3,4,5]Prelude> map (\n -> n * n) (filter (\n -> mod n 2 == 0) numbers)[4,16]
Or
@crichardson
Verbose DSLsdescribe('SomeEntity', function () {
beforeEach(function () { ... some initialization ... });
it('should do something', function () { ... expect(someExpression).toBe(someValue); });});
class SomeScalaTest ...{
before { ... some initialization ... }
it should "do something" in { ... someExpression should be(someValue)}
Versus
Jasmine
Scalatest
@crichardson
JavaScript is the language of the web
“You have to use the programming language you have, not the one that you
might want”
@crichardson
It works but the result is lost opportunities
and impeded progress
@crichardson
But if you think that this isn’t a problem then perhaps ....
“Stockholm syndrome ... is a psychological phenomenon in which hostages ... have
positive feelings toward their captors, sometimes to the point of defending them...”
http://en.wikipedia.org/wiki/Stockholm_syndrome
@crichardson
Martin Fowler once said:
"...I'm one of those who despairs that a language with such deep flaws plays such an
important role in computation. Still the consequence of this is that we must take
javascript seriously as a first-class language and concentrate on how to limit the damage
its flaws cause. ...."
http://martinfowler.com/bliki/gotoAarhus2012.html
@crichardson
Use just the good parts
http://www.crockford.com/
Douglas Crockford
@crichardson
Use a language that compiles to JavaScript
TypeScript
Classes and interfaces (dynamic structural typing)
Typed parameters and fields
Dart
Class-based OO
Optional static typing
Bidirectional binding with DOM elements
Less backwards compatibility with JavaScript
Also has it’s own VM
@crichardson
CoffeeScript Hello Worldhttp = require('http')
class HttpRequestHandler constructor: (@message) ->
handle: (req, res) => res.writeHead(200, {'Content-Type': 'text/plain'}) res.end(@message + '\n')
handler = new HttpRequestHandler "Hi There from CoffeeScript"
server = http.createServer(handler.handle)
server.listen(1338, '127.0.0.1')
console.log('Server running at http://127.0.0.1:1338/')
Classes :-)
Bound method
Concise
@crichardson
No escaping JavaScript
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on handles (e.g. sockets, file descriptors) to event handlers
@crichardson
Reactor pattern structure
Event Handlerhandle_event(type)get_handle()
Initiation Dispatcherhandle_events() register_handler(h)
select(handlers)for each h in handlers h.handle_event(type)end loop
handleSynchronous Event
Demultiplexerselect()
owns
notifies
uses
handlers
Applicationregister_handler(h1)register_handler(h2)handle_events()
@crichardson
Benefits
Separation of concerns - event handlers separated from low-level mechanism
More efficient - no thread context switching
Simplified concurrency - single threaded = no possibility of concurrent access to shared state
@crichardson
DrawbacksNon-pre-emptive - handlers must not take a long time
Difficult to understand and debug:
Inverted flow of control
Can’t single step through code easily
Limited stack traces
No stack-based context, e.g. thread locals, exception handlers
How to enforce try {} finally {} behavior?
@crichardson
Application code
NodeJS app = layers of event handlers
NodeJS event loop
Basic networking/file-system/etc.
HTTP DB driver ...
Event listener
Callback function
One time events:async
operation completion
Recurring events from Event
Emitters
@crichardson
Async code = callback hell
Scenarios:
Sequential: A ⇒ B ⇒ C
Scatter/Gather: A and B ⇒ C
Code quickly becomes very messy
@crichardson
Messy callback codegetProductDetails = (productId, callback) -> productId = req.params.productId result = {productId: productId} makeCallbackFor = (key) -> (error, x) -> if error
callback(error) else result[key] = x if (result.productInfo and result.recommendations and result.reviews) callback(undefined, result)
getProductInfo(productId, makeCallbackFor('productInfo')) getRecommendations(productId, makeCallbackFor('recommendations')) getReviews(makeCallbackFor('reviews'))
The result of getProductDetails
Gather
Scatter
Update result
Propagate error
@crichardson
Simplifying code with Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
when.js (part of cujo.js by SpringSource) is a popular implementation
Crockford’s RQ library is another option
@crichardson
Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity
responses = [getProductInfo(productId), getRecommendations(productId),
getReviews(productId)]
all(responses).spread(makeProductDetails)all(responses) spread(makeProductDetails)
responses = [getProductInfo(productId), getRecommendations(productId),
getReviews(productId)]
@crichardson
Not bad but lacks Scala’s syntactic sugar
class ProductDetailsService .... {
def getProductDetails(productId: Long) = {
for (((productInfo, recommendations), reviews) <- getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)) yield ProductDetails(productInfo, recommendations, reviews) }
}
getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)
yield ProductDetails(productInfo, recommendations, reviews)
for (((productInfo, recommendations), reviews) <-
@crichardson
Long running computations
Long running computation ⇒ blocks event loop for other requests
Need to run outside of main event loop
Options:
Community: web workers threads
Built-in: NodeJS child processes
@crichardson
Using child processesvar child = require('child_process').fork('child.js');
function sayHelloToChild() { child.send({hello: "child"});}
setTimeout(sayHelloToChild, 1000);
child.on('message', function(m) { console.log('parent received:', m);});
function kill() { child.kill();}
setTimeout(kill, 2000);
process.on('message', function (m) { console.log("child received message=", m); process.send({ihateyou: "you ruined my life"})});
parent.js
child.js
Create child process
Send message to child
@crichardson
Modern multi-core machines vs. single-threaded runtime
Many components of many applications
Don’t need the scalability of the Reactor pattern
Request-level thread-based parallelism works fine
There are other concurrency options
Actors, Software transactional memory, ...
Go goroutines, Erlang processes, ...
Imposing a single-threaded complexity tax on the entire application is questionable
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
Core built-in modules
Basic networking
HTTP(S)
Filesystem
Events
Timers
...
@crichardson
Thousands of community developed modules
https://npmjs.org/
web frameworks, SQL/NoSQL database drivers, messaging, utilities...
@crichardson
What’s a module?
One or more JavaScript files
Optional native code:
Compiled during installation
JavaScript != systems programming language
Package.json - metadata including dependencies
exports.sayHello = function () { console.log(“Hello”);}
foo.js
@crichardson
Easy to install
$ npm install package-name --save
@crichardson
Easy to use
var http = require(“http”)var server = http.createServer...
Core module ORPath to file ORmodule in node_modules
Module’s exports
@crichardson
Developing with NodeJS modules
Core modules
Community modules
Your modules
Application code
@crichardson
There is a module for that...
Modules + glue code =
rapid/easy application development
AWESOME!...
@crichardson
... BUT
Variable quality
Multiple incomplete/competing modules, e.g. MySQL drivers without connection pooling!
Often abandoned
No notion of a Maven-style local repository/cache = repeated downloads
...
@crichardson
To summarize
NodeJS
JavaScript
Reactor patternModules
Flawed and misunderstood
Scalable yet costly and
often unnecessary
Rich but variable quality
@crichardson
How will future history view NodeJS?
C++EJB
?
@crichardson
Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
So why care about NodeJS?
Easy to write scalable network services
Easy to push events to the browser
Easy to get (small) stuff done
It has a role to play in modern application architecture
@crichardson
Evolving from a monolithic architecture....
WAR
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
@crichardson
... to a micro-service architecture
Store front application
reviews application
recommendations application
product info application
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
orders application
@crichardson
Browser
WAR
StoreFrontUI
Model
View Controller
Presentation layer evolution....
HTML / HTTP
+ JavaScript
@crichardson
Browser Web application
RESTfulEndpointsModel
View Controller
...Presentation layer evolution
JSON-REST
HTML 5/JavaScriptIOS/Android clients
Event publisher
Events
Static content
@crichardson
Directly connecting the front-end to the backend
Model
View Controller Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
Model
View Controller
Browser/Native App
Traditional web application
Chatty API
Web unfriendly protocols
@crichardson
NodeJS as an API gatewayBrowser
Model
View Controller
HTML 5 - JavaScript
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
APIGateway
Native App
Model
View Controller
Single entry point
Optimized Client specific APIs
Protocol translation
RESTproxy
Event publishing
NodeJS
@crichardson
Serving static content with the Express web framework
var express = require('express') , http = require('http') , app = express() , server = http.createServer(app) ;
app.configure(function(){ ... app.use(express.static(__dirname + '/public'));});
server.listen(8081);
From public sub directory
@crichardson
RESTful web services
@crichardson
Proxying to backend serverexpress = require('express')request = require('request')
app = express.createServer()
proxyToBackend = (baseUrl) -> (req, res) -> callback = (error, response, body) -> console.log("error=", error) originRequest = request(baseUrl + req.url, callback) req.pipe(originRequest) originRequest.pipe(res)
app.get('/productinfo/*', proxyToBackend('http://productinfo....'))
app.get('/recommendations/*', proxyToBackend(''http://recommendations...'))
app.get('/reviews/*', proxyToBackend('http://reviews...'))
Returns a request handler that proxies to baseUrl
@crichardson
Implementing coarse-grained mobile API
var express = require('express'), ...;
app.get('/productdetails/:productId', function (req, res) { getProductDetails(req.params. productId).then( function (productDetails) { res.json(productDetails); }});
@crichardson
Delivering events to the browser
@crichardson
Socket.io server-sidevar express = require('express') , http = require('http') , amqp = require(‘amqp’) ....;
server.listen(8081);...var amqpCon = amqp.createConnection(...);
io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); });});
Handle socket.io
connection
Subscribe to AMQP queue
Republish as socket.io
event
https://github.com/cer/nodejs-clock
@crichardson
Socket.io - client side
var socket = io.connect(location.hostname);
function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); });};
ko.applyBindings(new ClockModel());
<html><body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script>
</body></html>
clock.js
Connect to socket.io
Subscribe to tick event
Bind to model
Update model
@crichardson
NodeJS is also great for writing backend micro-services
“Network elements”
Simply ‘route, filter and transform packets’
Have minimal business logic
@crichardson
NodeJS-powered home security
Upload2S3 UploadQueueProcessor
SQS Queue DynamoDBS3
FTP ServerLog file
FTP ServerUpload directory
@crichardson
SummaryJavaScript is a very flawed language
The asynchronous model is often unnecessary; very constraining; and adds complexity
BUT despite those problems
Today, NodeJS is remarkably useful for building network-focussed components