Upload
jonathan-felch
View
1.785
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Introductionary talk and tutorial on Groovy, discussion of its success and popularity, issues with the future of Java with examples
Citation preview
Groovy On The Trading Desk
Jonathan FelchQuantitative Strategies
Volant Trading LLC
Overview● Motivations for Polyglot Software with Groovy● Introducing Groovy: Scripting on the VM
● Skills To Take With You: Learning Groovy● Groovy DSLs: Creating the syntax you need● Unlearning Java, C++, C#● Groovy Performance and Concurrency● Relearning Java: The Future of the VM ● Questions
Motivations
● High Speed OO Tools + Dynamic / Functional Tools
● Re-use existing assets
● Lazy Evaluation
● Externalize Logic and Business Rules
● Dynamic Evaluation
Why Groovy ?● Meet Requirements
– Lazy & Dynamic Evaluation, – Concise Synatx, Rapid Development, Iterative Prototyping– Supported But Did Not Require Functional Style, Support for
Closures / Lamda Expressions– Tool Support, Attractive Adoption Pattern– Human Readable / Machine Executable Data Interchange– Natural Data Integration Platform
● Strong and Focused Community● Familiar Syntax / Ease of Adoption● Expensible Syntax : DSLs
Scripting on the VM
Evolution of the VM: Towards Dynamic Programming
– Features Popular in Most Used 'Frameworks' Exploited Specific Benefits and Uses of Dynamic Programming for a general-purpose or 'cross-cutting' concern
● Hibernate, Spring Framework, Jboss AOP, AspectJ
– Introduction of Dynamic Programming for Testing,Configuration and Business Rules
● BeanShell, Jython, JRuby
– Next Generation Scripting: Groovy, Scala, Clojure● Specific to VM, not a straight port of another language● Emphasis on Functional Programming● Tight Integration of FP / DP with a high performance OO Language
Introducing Groovy● The Blue Pill: Syntactically compatible with Java
– Easy Adoption for Java Programmers– Virtually seamless / virtually flawless integration with Java– Natural Support for Multiple Styles– Compatibility allows for remarkable performance tuning
● The Red Pill: A Language On Its Own– Type-system: Dynamically / Optionally Strong Typing– Closures: Lambda Expressions / Functions as Objects– GDK: Object and Collection Enhancements– Dynam Eval: Code is Data, Data is Code– Dynam Syntax: Write your own syntax– Enhancements: New operators, data literals, new basic types
Job Trends / Adoption Hints for new JVM Languages
Groovy Use Cases: ● 'Super Glue': I/O and Integration
● 'Code as Data': Configuration, Process Scripting, Business Rules
● 'Live Analysis': Ad-Hoc Queries & Expressions
● 'Language Building': New Syntax and DSLs
● 'Half-Baked Ideas'
● 'House Elf': Cleanup and Maintenance
Learning Groovy
Im proving o n O b je c tdef abstract class OptionPxModel {
def vol, carry, strike, expiry, pc def abstract px (def spot)
} OptionPxModel model = new BlackScholesModel(strike : 100, pc : 'C' ) model.vol = 0.15 model.expiry = Date.parse('MM/dd/yyyy', '11/18/2010') println “The Strike 100 Nov 2010 Call is worth ${model.px(102.0)}”
C lo s ure s : def r = new Random() def normRand = { r.nextGaussian() } def stockPrice = { px, vol, rate, time → px * exp((rate – vol**2) * time + normRand() * vol * time) }
Learning Groovy: CollectionsL is tsdef list = [5, 6, 7, 8]assert list.size == list.size() assert list[2] == list.getAt(2) == list.get(2) == 7 assert [1,2,3,4,5][-1] == 5 && [1,2,3,4,5][-2] == 4assert list.find { it > 6 } == 7 && list.findAll { it > 6 } == [7, 8]list.each { println “$it ${it * 2}” }['a', 'b', 'c'].eachWithIndex{ it, i -> println "$i: $it" }
Multiple A s s ig nm e ntdef (int a, int b) = [1, 2](a, b) = [b, a]R a ng e sassert ('a'..'d') == ['a','b','c','d']assert [ 1, *3..5, 7, *9..<12 ] == [1,3,4,5,7,9,10,11]def oneWeekForward = (new Date() .. (new Date() + 7))
Ma ps[ 'id': 'FX-17', name: 'Turnip', 99: 123, (-97): 987, "tail's": true ].each { key, value → println “$key => $value” }
Groovy One (or 2, 3, 4) Liners AppsExecuting A Process: ”processMyFile ./output.txt $Destination”.execute()
Adding A She-Bang To All Groovy Files: groovy -i .bak -pe "if (count == 1) println '#!/usr/bin/groovy'" *.groovy
Print the 1st column and 2nd-to-last column from filegroovy -a -pe "split[0] split[-2]" some.filename // or set of files
Create and Shuffle a Deck of Cardsdef deck = [ [ *2..10, *('J Q K A'.split()) ], 'C D H S'.split() ]
.combinations().collect { it.join('') }; Collections.shuffle(deck)
Generate 21-Character OSI Symbol for Option def osi = { symbol, year, month, day, cp, strike -> sprintf('%1$s%2$2d%3$2d%4$2d%5$s%6$08.3f', [ symbol.padLeft(6), year as Integer, month as Integer, day as Integer, cp as String, strike as Double ] as Object[] )}
Learning Groovy: Operators'A s O pe ra to r'
"3.14159" as Double def readable = { it.put("12 34".reverse()); 5 } as ReadableAs O pe ra to r O n C lo s ure s + Ma ps Ob je c ts ? →
def map; map = [ i: 10, hasNext: { map.i > 0 }, next: { map.i-- } ] as Iterator
while ( map.hasNext() ) println map.next()
E lvis , S a fe Re fe re nc e , S pre a d & S pa c e s hip
def lang = data?.favoriteLanguage ?: 'Groovy' assert ['cat', 'elephant']*.size() == [3, 8] assert (3 <=> 4) == -1 && (3 <=> 3) == 0 && (5 <=> 3) == 1
Re g e x
def list = ["foo", "bar", "moo"] def pattern = ~/foo/
assert "foo" =~ list.find { it =~ pattern }assert ["foo", "moo"] == list.findAll { it ==~ /.*oo/ }
One Slide Apps: RSS Reader
def url ='http://www.groovyblogs.org/feed/rss'def items = new XmlParser().parse(url).channel.itemdef cols = 'pubDate title description'.tokenize()groovy.swing.SwingBuilder.build { frame(id:'f', title: 'Groovy RSS', visible :true) {
scrollPane {table {
tableModel(list: items) {cols.each { col →
closureColumn header: col,read: { it[col].text() }
} } } } }f.pack()
}.show()
Credit: Dierk König, GR8 Conference 2009, Copenhagen
One Slide Apps: Monte Carlo
def px = 100, rate = 0.05, vol = 0.15, time = 1.0def strikes = [ 8..12 ].collect { it * 10 }def w = RandomNumbers.getNormDist(1000,1000)def S = px * exp((rate-vol ** 2)* time + sqrt(time)* vol * w)strikes.each { K → def optionValue = max(0, S – K) def df = exp(-rate * time) println “${K} : ${df * optionValue as Number}”})
One Slide Apps: RESTful Web Serverimport com.sun.net.httpserver.*;
def isPrime = { Long N -> if (N < 2) return false; for (long i = 2; i*i < N; i++) if (N % i == 0) return false return true }
def pws = { HttpExchange exch -> exch.sendResponseHeaders(200,0); def pe = exch.requestURI.path.split('/') if (pe.size() == 3 && pe[1] == 'prime') { exch.responseBody.write("${isPrime(pe[2] as Long)}".bytes) } else exch.responseBody.write("${exch.requestURI.path}".bytes) exch.responseBody.close();} as HttpHandlerdef run = { handler, minutes -> def server = HttpServer.create(new InetSocketAddress(8888),0) server.createContext('/', handler) server.start()}run(pws,1)
Unlearning Java: Taking the Red Pill● Meta-Programming
● Design Patterns
● Compile Time Meta-Programming
● Power of Lazy Evaluation
Dynamic Language: Evaluation or Syntax
“In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program” - Paul Graham
“When [Smalltalk] is used to describe an application system, the developer extends Smalltalk, creating a domain-specific language by adding a new vocabulary of language elements ...” - Adele Goldberg DSLs
Meta-Programming● Understanding The Type System
– Groovy has an optionally dynamic strongly typed type system– What does that mean ?
● Dynamic Method Invocation and GroovyObject– invokeMethod, get/setProperty
– What is a MetaClass ?● Meta Object Protocol
– def methodMissing(String name, args) { }– def propertyMissing(String name)– def propertyMissing(String name, value)
● Overloading Operators
GORM as a DSL// These Class Definitions Generate SQL and Database Tables
class Book {String titleDate releaseDateAuthor author
}
class Author {String name
}
// These Operation Generate SQL and Database Queries
def book = Book.findByTitle("The Stand")book = Book.findByTitleLike("Harry Pot%")book = Book.findByReleaseDateBetween( firstDate, secondDate )book = Book.findByReleaseDateGreaterThan( someDate )
book = Book.findByTitleLikeOrReleaseDateLessThan( "%Something%", someDate )
GORM as a DSLclass GORM {
def dynamicMethods = [ ]
def methodMissing(String name, args) {def method = dynamicMethods.find { it.match(name) }if(method) { GORM.metaClass."$name" = { Object[] varArgs →
method.invoke(delegate, name, varArgs)}return method.invoke(delegate,name, args)
}// GORM actually writes or creates a new method hereelse throw new MissingMethodException(
name, delegate, args )}
}
DSL: Tool Kit● Meta-Methods
– String.metaClass.swapCase= {delegate.collect{ c →
c in 'A'..'Z'? c.toLowerCase() :c.toUpperCase()}.join()}
assert "Foo".swapCase()== "fOO"
Number.metaClass.multiply= { Grid g g.multiple(delegate) }→
● MOP– propertyMissing, methodMissing, respondsTo,
hasProperty, listMethods, selectMethod
● Categories, ObjectBuilder, MetaBuilder
Function Eval / Visualizationnew SwingBuilder().frame(title:'Quant Modeller', size : [1100,600], // .. parameters here not shown {
tableLayout() {tr {
td { widget(gui) } td { widget(editor, border : titledBorder('Model Script',
preferredSize :[900,600])) } td {
panel(border : titledBorder('Model Calibration'), preferredSize :[200,600]) {tableLayout(cellpadding :15) { tr {
td { label(text : 'Params:') }td { button(text : 'Add', \* Edited Out *\ }
tr { td(colspan : 2, align : 'center') {
button(text : 'Execute', actionPerformed : {
shell.setVariable 'f', [:]shell.eval1uate(editor.text)def f = shell.getVariable('f')def obj = [ f : f] as Model3dgui.model = objgui.redraw()
} ) } } }
New AST Transforms
• @Singleton — okay, not really a pattern :-)
• @Immutable, @Lazy, @Delegate
• @Newify
• More DSL and Meta-Magic! @Category and @Mixin
• @PackageScope
• Swing’s @Bindable and @Vetoable
• Grape’s @Grab
The Evil Java Singleton
public class Evil {public static final Evil instance = new Evil();
private Evil() {}static Evil getInstance() { return instance; }
}
• In Groovy now:–@Singleton class Evil {}
• A lazy version also:–@Singleton(lazy = true) class Evil {}
Credit: Guillaume Laforge, “To Infinity and Beyond” & GDK 1.7 Release notes
DSLs → They Start Out As APIs// closure maddness// Before refactoring
def phrase = "The quick brown fox jumps over the lazy dog"
def result = phrase.toLowerCase().toList().findAll{ it in "aeiou".toList() }. // like WHERE …groupBy{ it }. // like GROUP BY …findAll{ it.value.size() > 1 }. // like HAVING …sort{ it.key}.reverse(). // like ORDER BY …collect{ "$it.key:${it.value.size()}" }.join(", ")
println result
Credit: Paul King, “Groovy DSL”
DSL: APIs That Get Refactored// Refactored helper closuresdef lowercaseLetters= phrase.toLowerCase()def vowels = { it in "aeiou".toList() }def occursMoreThanOnce= { it.value.size() > 1 }def byReverseKey= { a, b -> b.key<=> a.key}def self = { it }def entriesAsPrettyString= { "$it.key:${it.value.size()}" }def withCommaDelimiter= ", "
// Refactored main closure println lowercaseLetters.
findAll(vowels). groupBy(self).
findAll(occursMoreThanOnce).sort(byReverseKey).collect(entriesAsPrettyString).
join(withCommaDelimiter)Credit: Paul King, “Groovy DSL”
Putting All Together I● Drop Parenthesis
● Named Parameters
● Objects Graphs → Tree of 'Nouns' and 'Verbs'
● Human Readable / Machine Executable
● Specific Syntax
Putting All Together IIorder to buy 200.shares of GOOG {
limitPrice 500 allOrNone false at the value of { qty * unitPrice - 100 }
}
take 2.pills of chloroquinine after 6.hours
def "length of Spock's & his friends' names"() { expect: name.size() == length where:
name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6
} Credit: Paul King, “Groovy DSL”
Config SlurperLog4j {
appender.stdout = "org.apache.log4j.ConsoleAppender"appender."stdout.layout"="org.apache.log4j.PatternLayout"rootLogger="error,stdout"
logger {org.springframework="info,stdout"
}
additivity {org.springframework=false
}}
databaseURL : //
development {databaseURL : //
}
testing {databaseURL : //
}
Performance ● Dynamic Typing is Expensive (But Worth It)
– Hot Spot Opportunity
● Dynamic Method Invocation is expensive (But Getting Better)
– Da Vinci Project and invokeDynamic bytecode– Hot Spot Opportunity
● BigDecimal & Collections → Insanely Expensive For Serious Math
– People need to better learn to use IEEE floating point types–
Performance II ● Getting Too Groovy ???
– Abusing .each { tuple } → // for loops versus anon closures
● Simplicity --> faster– Groovy reduces your dependency on Java Enterprise– Gpath vs Xpath (Gpath almost always wins)– Use Visual VM: Groovy Memory Use Can Be Mysterious
● Too Many Collections ? – Is The Idea to 'Cache' The World As A Map
● Use Groovy-Aware Java for Faster Code– People Need to learn IEEE Floating Point Types
Actors
class Player extends AbstractPooledActor {String namedef random = new Random()
void act() {loop {
react {// player replies with a random movereply Move.values()[
random.nextInt(Move.values().length)]
}}
}}
Scaling the VM: Immutable Data Message + Services
Are Java & C# Becoming Groovy ?● Google Guava (formerly Collections)● Lots of new Libraries for FP, Enhanced
Collections, Tuples, Monads, etc...● Doug Lea's Fork-Join + Predicate Libraries● Project Lambda : Closures for Java (JDK 8 ?)● Da Vinci Project : invokeDynamic (JDK 7 ?)● Is Gpath LINQ for Java ? (Now ?)● Universal Low Level VM● Will Groovy or Groovy++ Continue ?
A few bad months for Java
● JavaTM By Oracle , ● The Java Community Process, Doug Lea● The Java Community Process, Apache● JavaOne, The Conference● Oracle vs Google● Oracle vs Apache● IBM and Apache● OpenJDK
Where is Goovy Going ?● Some Important JDK Improvements will be in
JDK 7 (as planned): invokeDynamic
● More explicitly FP approach to concurrency and state
– AST MP / GoF Patterns– @Immutable / @Singleton ( lazy : true) – GPars Actor-like syntax
● Groovy Server for Fast Scripting– Difficult on a cluster
Where is Goovy Going ?● Closures with Memory def cl = {a, b →
sleep(3000) // create a noticeable pause a + b
}def mem = cl.memoize()
// memorize() variants are // memorizeAtMost(n), memorizeAtLeast(n), // memorizeBetween(m, n)
● Groovy++– Not a fully compatible fork– An interesting argument – But no.
Groovy Gotchas!● Subclassing Map, Unexpected Results
● Map.class● the dot operator,
getAt(...), getXXX(), [ name ] literals
● Private isn't Private ● Inconvenient when you really want to hide implementation data
● Unexpected Weird Effects – Combining Static and Dynamic Numeric Types Across Java Code
and Groovy Code ● Be careful and deliberate with typing
– Method selection surprises ● identical static closures and static functions do not respond to
the same way Invoked dynamically
Best Practices● Explicitly Signatures and Types Where You Can
● Better GroovyDoc documentation● Faster method selection● More shareable (with Java / Scala / Ioke / etc..) Libraries
● Write Groovy Aware (GroovySupport) Java● This is where to build your high-speed / high-performance code
● Use ConfigSlurper to externalize environment / context data
● Use CliBuilder and Groovy Server if it fits your production model
● Add Performance Tests To Your Suite / Learn to use Visual VM– Testing is more important with dynamic and functional programming– Small tests – small in terms of code coverage – on big data sets
● Best place to find surprising performance enhancements– Automating 'Just Got Lucky' Into Your Development Cycle
Acknowledgments & Suggestions
● Derik Kognig, Author Groovy In Action ● Peter Bell, Systems Forge● Paul King, Asset Australia ● Guillaume Laforge, Groovy Project Manager● PLEAC, Project Euler, Groovy Zone
Contact Info● [email protected] ● [email protected]● www.twitter.com/Jonathan.Felch
– Rare but regular contributions via Blog, GitHub, SlideShare
● 3 – 5 contributions a year, typically unfinished half-baked ideas● Really targeted at a dozen or so people that I have worked with
in the past or occasionally collaborate