Embed Size (px)
DESCRIPTION
Two years ago Ardoq set out to build the documentation platform that the IT-professional wants to use. Starting a “greenfield” project, we were free to choose what we considered to be the best technology for the job, so we ended up using Clojure for all backend components. In this talk I will share our experience using Clojure to build the http://ardoq.com/ documentation platform. A few highlights - Building comprehensive REST-APIs in Clojure - Rapid feedback using REPL driven development - Building a “realtime web” backend using Clojure, WebSockets and Redis. I will also share some of the mistakes we made along the way, as well as lessons learned and tips for anyone starting a new Clojure project.
Citation preview
Ardoq.com @ebaxt
Erik Bakstad
Clojure at
Ardoq
Oslo Socially Functional Programmers
Ardoq.com @ebaxt
! 6 years as consultant (finance/telecom)
! Mostly Java & JavaScript professionally
! Discovered Clojure in 2009
! Groovy, Ruby, Python, SML, Racket, Scala …
! CTO & Co-founder of Ardoq
My background
Ardoq.com @ebaxt
AGENDA
! What is Ardoq?
! Why we decided to use Clojure
! Overview Clojure at Ardoq + 2 Cases
! REPL/development environment
! Experience so far / Q&A / Discussions
Ardoq.com @ebaxt
What is Ardoq?
Ardoq.com @ebaxt
We‘re building larger, more complex systems
! Many different “sub-systems“
! Different technologies
! Wide range of integrations
! System-of-systems problem
Ardoq.com @ebaxt
To achieve conceptual understanding we create artifacts
! Documents
! System maps
! Diagrams
! Flowcharts
These artifacts are hard to maintain!
Ardoq.com @ebaxt
Existing tools are either ...
Enterprise architecture software
Very Complex
Word
Not powerfull enough
Excel PowerPoint
Wikis
V isio
Ardoq.com @ebaxt
Ardoq is a documentation platform that offers
! Explorable, “Living” documentation
! Automatically visualizes dependencies and hierarchies
! REST-api for data import/export
! Supports custom visualizations via plugins
Ardoq.com @ebaxt
DEMO
Ardoq.com @ebaxt
Why Clojure?
Ardoq.com @ebaxt
Ardoq.com @ebaxt
Ardoq.com @ebaxt
Ship software
Quickly Fun!
C onsistently (over time)
Quality
Try out different solutions
Understand How it works!
Leverage
Ardoq.com @ebaxt
JavaScript
MongoDB
Clojure
BSON
JSON
{ "firstname": "Erik", "lastname": "Bakstad", "age": 30, "hobby": ["hockey", "cycling"] }
{ "firstname": "Erik", "lastname": "Bakstad", "age": 30, "hobby": ["hockey", "cycling"] }
{ :firstname "Erik" :lastname "Bakstad" :age 30 :hobby ["hockey", "cycling"] }
EDN
Ardoq.com @ebaxt
! Extensive REST-api
! “Version control” on top of MongoDB
! Real-time web backend
! 3-party integrations (recurly, jira, segment.io, +++)
! Attachment service with pluggable storage provider
Stuff we‘ve built in Clojure
Ardoq.com @ebaxt
Architecture
Ardoq.com @ebaxt
Backbone.js !D3.js!jQuery!Require.js! JavaScript-app!
Clojure!backend!
MongoDB!! Redis!
H T TP Websockets SNS!S3!
Segment.io!Mailchimp!Mandrill!Recurly!JIRA!Tarsnap!!
H T TP
TCP
Plugins!JavaDoc / Excel! H T TP
Ardoq.com @ebaxt
api!
com.ardoq.*
utils! service!
gateway!persistence!
MongoDB!!
SNS!S3!
Segment.io!Mailchimp!Mandrill!Recurly!JIRA!!!
hAp://www.infoq.com/presentaGons/DDDIClojure$
Ardoq.com @ebaxt
Libraries
! Carmine
! amazonica
! core.async
! Pantomime
! clj-http/chime/clostach
! Ring/compojure/Liberator
! http-kit/clj-wamp
! Friend/oauthentic/clj-oauth
! clj-schema/antisamy
! Monger/ragtime
Ardoq.com @ebaxt
Testing/debugging
! Started out with midje
! Now using clojure.test
! ring.mock
! Lein test selectors
! Mongodb + redis
! clojure.tools.trace
! IntelliJ with Cursive
! Embedded REPL
! criterium
Ardoq.com @ebaxt
Editor/IDE
! Emacs + Cider
! clojure-mode + paredit
! clojure-test-mode
! clojure-refactor (just started)
! company-cider
Ardoq.com @ebaxt
Infrastructure/monitoring
! S3 (wagon)
! SNS
! Zapier
! Tarsnap
! HipChat
! AWS / on-site
! Ansible /python/bash
! Circle-CI
! Logstash + Kibana
! Pingdom
Ardoq.com @ebaxt
Case 1
REST-api
Ardoq.com @ebaxt
Request Response
Servlet C ontainer Servlet AP I
Ring
Application
Ardoq.com @ebaxt
{:scheme(:http,((:session({}((:request3method(:get,((:query3string("q=hello"((:cookies({}((:uri("/test"((:headers(({"accept3encoding"("gzip,(deflate"}((:body(...}$
(defn%handler([request](...()$
{:status(200((:headers({"Content3Type"((((((((((((("text/plain"}((:body("Hello(World!"}$
Ardoq.com @ebaxt
(defn%middleware([handler]((((()$
Ardoq.com @ebaxt
(defn%middleware([handler]((((fn%[req](((...((((handler(req)(((...())$
Ardoq.com @ebaxt
(defroutes(api((((GET("/"([](...)((((POST("/"([](...))(((def%handler(((((logging((session((security(app))))((((run3jetty(handler({:port(8080}))$
Ardoq.com @ebaxt
(defroutes(api((((GET("/"([](...))(
?
Ardoq.com @ebaxt
Ardoq.com @ebaxt
Liberator
hAp://clojureIliberator.github.io/liberator/$
(defroutes(api((((GET("/"([](...))(
Decision graph
Ardoq.com @ebaxt
Decision graph
hAp://clojureIliberator.github.io/liberator/assets/img/decisionIgraph.svg$
(resource((((((((((:allowed3methods([:get]((((((((((:allowed?((fn%[_]((permission/is3admin?))((((((((((:available3media3types(["application/json"]((((((((((:exists?((fn%[_]({::result(...)((((((((((:handle3ok(::result)))$
Ardoq.com @ebaxt
Authentication
hAps://github.com/cemerick/friend$
! Basic Auth
! Token Auth
! Oauth (Log in with Github, Google, Live Connect)
Ardoq.com @ebaxt
Friend
hAps://github.com/cemerick/friend$
Ardoq.com @ebaxt
Friend
hAps://github.com/cemerick/friend$
{:workflows([(ajax3login)(((((((((((((((token3workflow(:realm("Ardoq"(:credential3fn(...)(((((((((((((((workflows/http3basic(:realm("Ardoq")(((((((((((((((github3oauth(system)(((((((((((((((live3connect3oauth(system)(((((((((((((((google3oauth(system)]((:credential3fn((partial(verify3credentials(system)((:login3uri("/login"((:unauthorized3redirect3uri("/login"((:default3landing3uri("/index.html"}$
Ardoq.com @ebaxt
Middleware
! wrap-impersonate
! wrap-organization
! wrap-session-expiry
! clojure-metrics
Ardoq.com @ebaxt
Case 2
Real-time web
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!POST$/api/component${“name”:$“foo”}$
NoGfy$user$of$BrowserI1’s$POST$
Ardoq.com @ebaxt
Why not use something like Pusher.app or firebase?
Ardoq.com @ebaxt
The Web Application Messaging Protocol
hAp://wamp.ws/$
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!
Websocket + Wamp
Http-kit + cl j-wamp
Autobahn.js + f lash-fal l back
?
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!
Redis PubSub + core.async Redis!
Ardoq.com @ebaxt
Publish!Event! Redis!
Carmine!listener!
Core.async!mult-channel!
(loop! …! (wamp/send-event …)! (recur))!!
Browser-2!Browser-1!
Workspace-event Organization- event
dao!
Ardoq.com @ebaxt
Publish!Event! Redis!
Carmine!listener!
Core.async!mult-channel!
(loop! …! (wamp/send-event …)! (recur))!!
Analytics!
Testing!
dao!
Ardoq.com @ebaxt
REPL
Driven Development
Ardoq.com @ebaxt
http://thinkrelevance.com/blog/2013/06/04/clojure-workflow-reloaded
https://github.com/juxt/jig
https://github.com/stuartsierra/component
https://www.youtube.com/watch?v=13cmHf_kt-Q
Ardoq.com @ebaxt
The Clojure experience
So far
Ardoq.com @ebaxt
Ardoq.com @ebaxt
Other benefits
Tiny codebase Fast feedback Libraries (both Java and Clojure) Concurrency
Ardoq.com @ebaxt
Challenges
Dynamic language - tradeoffs Lack of signatures LISP – with great power comes…
Ardoq.com @ebaxt
Thank you!
Questions?
Ardoq.com @ebaxt
THANK YOU!
