Clojure at DataStax: The Long Road From Python to Clojure

CLOJURE AT DATASTAX: THE LONG ROAD FROM PYTHON TO CLOJURENick Bailey @nickmbailey

© 2016 DataStax, All Rights Reserved.

1 Introduction

2 Starting with Python

3 Clojure Enters the Ring

4 Clojure Everywhere!

5 Takeaways

2

© 2016 DataStax, All Rights Reserved. 3

1. Introduction


Me• DataStax • Clojure • Apache Cassandra • Austin

4


DataStax• DSE (distributed database) • Java

• OpsCenter • Python • Clojure

5


This talk• DataStax Clojure experience report • Python shop interested in Clojure? • Have a large python application you wish was written in Clojure?

6


2. Starting With Python


A long time ago in an galaxy far far away…


A long time ago in an office not far away…

(5 years ago)


Building a Database Monitoring Tool

10



11



12


Ok… but why?


Why Twisted?

14

• 2010 • 10ish person startup • 1 front-end dev • 1 back-end dev


Product Complexity Grows

15

• Single process isn’t enough • Need an agent to run on the cluster


Product Growth

16


Things go wrong

17

• The JVM doesn’t play nice • Python may not be the best choice for monitoring a JVM… • … but there is definitely a JVM available…


3. Clojure Enters the Ring


3. Clojure Enters the Ring (pun intended)


(still 5 years ago)


Replacement Needed

21

?

?

?

?


Responsibilities

22

• Gather and report metric data


Requirements

23

• Need to interact heavily with a JVM based database • Maintainability • Developer happiness


Contenders

24

• Java • Clojure • Scala • Jython


Why Clojure

25

• Isolated piece of the architecture • Immutability was a good fit • Great interoperability • An experiment worth taking a risk on


New Architecture

26


Experiment Results…


Smashing Success!

28

• Clojure fit all of our needs • Great recruitment tool • Clojure inspires passion • Picked up by new developers


On Learning Clojure

29

“…the most important transition will be opening your soul to Truth and Beauty, if you do so, Clojure will come naturally…”

—Philip Doctor


A Note on Hiring

30

• Datastax has a distributed engineering organization • Our clojure engineers span the globe


The rough edges

31

• Deployed on 1.2, immediate painful migration • JMX library unmaintained • Less mature libraries


Ok… why didn’t you just rewrite it all?


Hindsight is 20/20…

33

• Clojure was definitely considered an experiment/risk • Still a tiny startup • Bulk of code and functionality in the Twisted code


4. Clojure Everywhere!


Back to the future!

(ish)


What’s changed?

36

• 2 person team to 20 person team • Clojure codebase has grown significantly • JVM usage has increased across the company • Sorry Python, Clojure is the favorite child


Time to Reevaluate

37

?


Reevaluating Twisted

38

• Scalability requirements have grown • GIL rears it’s ugly head • Overlapping responsibilities


Reevaluating Twisted

39

• Need a path the JVM • Need to address scalability concerns


Possible Solutions

40

• Rewrite completely • Multiple processes • Jython?


Rewrite completely

41

• Benefit of hindsight when starting from the ground up • 4+ years invested • Pause feature development • Inevitable rewrite bugs introduced


Multiple Processes

42

• Less time intensive than a re-write • Can gradually move things out of python • Multi-processing is difficult • Untangling Twisted is extremely difficult • Adding IPC • Additional customer burden


Port to Jython

43

• Theoretically completely behind the scenes • Shortest path to being completely on the JVM • No out of the box support for Twisted • Introduction of a “third language” • The road less traveled


Jython Wins!


The Present

45


Migrating to Jython1. Make Twisted run on Jython 2. Replace all broken python libraries 3. Fix all the Jython* bugs you found 4. Write Jython <-> Clojure translation layer

46


1. Make Twisted run on Jython

47

• Upgrade Twisted • Start application on twisted • Get a stack trace and fix the issue • Rinse • Repeat


2. Replace all broken python libraries

48

• C extensions won’t run on Jython • SSL, LDAP, sqlite, crypto • Jython doesn’t support python sub processing


3. Fix all the Jython* bugs you found

49

• Brace yourselves…


3. Fix all the Jython* bugs you found

50

• Socket leaks • Deadlocks (including during garbage collection) • Incorrect socket implementations • Lots of SSL bugs • *And also 1 bug in Netty


If you thought stack traces and heap dumps were bad in Clojure…


Write Jython <-> Clojure translation layer

52

• The fun part • Clojure: “What’s a PyFunction object!?” • Jython: “Shut up, what’s an IPersistentVector!?” • Luckily, everyone speaks Java • https://gist.github.com/nickmbailey/3f11300e944e17d92edd6d42da785ab3

https://gist.github.com/nickmbailey/3f11300e944e17d92edd6d42da785ab3


The Present

53


Success?

54

• There were some bumps a long the road • Took a lot longer than hoped • It was definitely the right choice


Over the Rainbow

55


Takeaways


Should I use Clojure?

57

• Yes, open yourself to Truth and Beauty


Should I not use Python?

58

• No one said that! • But if you are mainly interacting with the JVM… probably not.


Should I migrate my Python app to Clojure via Jython?

59

• Maybe? • Do you need to be on the JVM? • How much C code will you need to replace? • Are you brave enough to venture into the Jython codebase itself?


Questions? [email protected]

mailto:[email protected]

Software

Clojure at DataStax: The Long Road From Python to Clojure