Er jang: Erlang on the JVM - GOTO Conferencegotocon.com/dl/goto-cph-2011/slides/... · Erjang: Goals •Learn Erlang ... Java Virtual Machine Your favorite OS ERJANG Your Erlang Program

Erjang: Erlang on the JVM

Kresten Krab ThorupTrifork

@drkrab

Thursday, May 12, 2011

Erjang: Goals

• Learn Erlang

• Discover “Actor Programming”

• Meet great people


What is Erjang?

• Erjang is an execution engine for BEAM byte code, written in Java.

• JIT-Compiles BEAM to JVM byte code.

• Runtime uses shared heap model.

• BIFs and drivers are written in Java.


Your favorite OS

BEAM Emulator

Your Erlang Program

OTP FrameworkBEAM

BIFs


Java Virtual Machine

Your favorite OS

ERJANG

Your Erlang Program

OTP FrameworkBEAM

JVM

BIFs


Let’s see it in action...


Erlang ≠ Erjang

Shared heap

¬ Native drivers

¬ NIF


Why Java?

• JVM is everywhere (from mobile to AS/400)

• JVM has many libraries / integrations.

• JVM has 500+ man-years of engineering

• JVM is fast (for Java-ish programs).

• ... and I know JVM pretty well, ...


These are experimenting...

StreamBase (CEP)

Running Erlang-based FSMs on event streams

ErlangIDE (Eclipse)

Run compiler, type analysis, etc.


Erjang - Challenges• Tail-recursion

• Interfacing to Java

• Ultra Light-Weight Processes

• Real-time Behavior

• Arbitrary Precision Numbers

• Erlang Drivers

• JVM is safe, urgh!


JIT Compiler

BEAM Reader

Type Analysis

JVM Codegen

ring.beam

ring-2a149ada.jar


JIT Compiler

BEAM Reader

Type Analysis

JVM Codegen

ring.beam

ring-2a149ada.jar

erlang:load_module error_handler:undefined_function

ClassLoader.loadClass(“erjang.m.ring”)

*


Erlang ⇒ JVM

• Module ⇒ Class (+support in a “.jar”)

• Function ⇒ Static Method + “EFun” object

• Value ⇒ Object Instance

ETuple, EPair, EFun, ESmall, EBig, ...


Erlang ⇒ JVM

-module(bar).

process([H | T], T2) -> process(T, foo(H, T2));

process([], T2) -> T2.

foo(H, T) -> lists:reverse(H ++ T).


The BEAM Code {function, process, {nargs,2}}. {label,264}. {test,is_nonempty_list,{else,265},[{x,0}]}. {get_list,{x,0},{x,0},{y,0}}. {call,2,foo}. {move,{x,0},{x,1}}. {move,{y,0},{x,0}}. {call_last,2,process,1}. {label,265}. {test,is_nil,{else,263},[{x,0}]}. {move,{x,1},{x,0}}. return. {label,263}. {func_info,{atom,appmon_bar},{atom,process},2}.


public static EObject process___2(EProc eproc, EObject arg1, EObject arg2){ ECons cons; ENil nil; tail: if((cons = arg1.test_nonempty_list()) != null) { // extract list EObject hd = cons.head(); EObject tl = cons.tail(); // call foo/2 EObject tmp = foo___2$call(eproc, hd, arg2); // self-tail recursion arg1 = tl; arg2 = tmp; goto tail; } else if ((nil = arg1.test_nil()) != null) { return arg2; } throw ERT.func_info(am_bar, am_process, 2);}


DEMO

• c(foo).

• c(foo).

• foo:main().


Erlang ⇒ JVM

-module(bar).

process([H | T], T2) -> process(T, foo(H, T2));

process([], T2) -> T2.



public static EObject foo__2(EProc p, EObject H, EObject T){ EObject r = foo__2$body(p,H,T); while (r == TAIL_MARKER) { r = p.tail.go(); } return r;}



public static EObject foo__2$body(EProc p, EObject H, EObject T){ // Tmp = erlang:’++’(H,T) EObject tmp = erlang_append__2.invoke(p,H,T);

// return lists:reverse(Tmp) p.tail = lists__reverse_1; p.arg1 = tmp; return TAIL_MARKER;}



package erjang.m.bar;class bar extends ECompiledModule {

@Import(module=“lists”, fun=“reverse”, arity=1) static EFun1 lists__reverse__1 = null;

@Import(module=“erlang”, fun=“++”, arity=2) static EFun2 erlang__append__2 = null;

...

}



Interface to Java

• Calling to JAVA from ERJANG

• BIFs: special calling convention

• Plain Java: Type conversions

• Calling ERJANG from JAVA


Example BIF

// foo:bar(...) native Function

package erjang.m.foo;class foo extends ENative {

@BIF public static EObject bar(EProc proc, EObject arg1, arg2, ...) {

}}


Example BIF @BIF public static EObject spawn_link(EProc proc, EObject mod, EObject fun, EObject args) throws Pausable { EAtom m = mod.testAtom(); EAtom f = fun.testAtom(); ESeq a = args.testSeq();

if (m==null||f==null||a==null) throw ERT.badarg(mod, fun, args); EProc p2 = new EProc(proc.group_leader(), m, f, a); p2.link_to(proc); ERT.run(p2); return p2.self_handle(); }


demo() -> Map = 'java.util.HashMap':new(), Map:put('x', "4"), Map:put(1, 'foo'), print(Map).

print([]) -> ok;print([{Key,Val}|Tail]) -> io:format("key=~p, value=~p~n", [Key,Val]), print(Tail).


! ! new Thread() {! ! ! {! ! ! ! setDaemon(true);! ! ! ! start();! ! ! }

! ! ! public void run() {! ! ! ! String[] ARGS = {! ! ! ! ! ! "-progname", "ej",! ! ! ! ! ! "-home", System.getProperty("user.home"),! ! ! ! ! ! "-root", "/Users/krab/Projects/OTP_R13B04",! ! ! ! ! ! "+A", "2",! ! ! ! ! ! "+S", "1",! ! ! ! ! ! "+e", "5.7.5",! ! ! ! ! ! "-s", "rpc", "erjang_started"! ! ! ! ! };

! ! ! ! try {! ! ! ! ! erjang.Main.main(ARGS);! ! ! ! } catch (Exception e) {! ! ! ! ! e.printStackTrace();! ! ! ! }! ! ! };! ! };


! ! //! ! // The -s rpc erjang_started argument // makes the loader call rpc:erjang_started/0! ! // which is the trigger that will release // 'wait_for_erjang_started'! ! //! ! System.err.println("did launch Erjang, ... waiting");! ! RPC.wait_for_erjang_started(60*1000L);! !

! ! //! ! // Call erlang:display( ["Hello, Joe!~n", []] )! ! // ! ! EString hello_str = EString.fromString("Hello, Joe!~n");! ! ESeq format_args = EList.make( hello_str, ERT.NIL ); ! !! ! RPC.call(am_erlang, am_display, (EObject)format_args );


Light-Weight Processes

• Threads don’t cut it.

• Erjang uses Kilim, a JVM byte code rewriter

• Allows scheduling processes onto threads.


KilimKilim: Isolation-Typed Actors for Java 105

!"#$%&'!%()*%)+

",,-&"&%.$#)

/'&%()-.%

*%"!0-.%1

2$-1"&2-,)*%)+

345&#",$6-#0

/'&%()-.%

7212089%":%#

;":")

%<&%#,"1",,-&"&2-,$

Fig. 1. javac output post-processed by Kilim weaver

network and disk) [4] and service-oriented workflows. With a view to immedi-ate industrial adoption, we impose the following additional requirements: (a) nochanges to Java syntax or to the JVM, (b) lightweight actors1 (c) fast messaging(d) no assumptions made about a message receiver’s location and implementa-tion language (e) widespread support for debugging, logging and persistence.

1.2 The Kilim Solution

This paper introduces Kilim2, an actor framework for Java that contains a byte-code post-processor (“weaver”, see Fig. 1) and a run-time library. We list belowsome important features as well as the design points:

Ultra-lightweight threads. Kilim’s weaver transforms methods identified byan @pausable annotation into continuation passing style (CPS) to providecooperatively-scheduled lightweight threads with automatic stack manage-ment and trampolined call stack [3, 20]. These actor threads are quick tocontext-switch and do not need pre-allocated private heaps. The annotationis similar in spirit to checked exceptions in that all callers and overridingmethods must be marked @pausable as well.

Messages as a special category. For the reasons outlined above, we treatmessage types as philosophically distinct from, and much simpler than otherJava objects. Messages are:

– Unencapsulated values without identity (like their on-the-wire coun-terparts, XML, C++ structs, ML datatypes and Scala’s case classes).The public structure permits pattern-matching, structure transforma-tion, delegation and flexible auditing at message exchange points; theseare much harder to achieve in the presence of encapsulation.

– Not internally aliased. A message object may be pointed to by at mostone other message object (and then only by one field or array element of

1 For example, threads are too heavyweight to assign per HTTP connection or percomponent in composable communication protocol state machines.

2 Kilims are flexible, lightweight Turkish flat rugs woven with fine threads.

http://www.malhar.net/sriram/kilim/




Kilim Rewritingint execute() throws Pausable { msg = mbox.get(); return msg.size();}


Kilim Rewritingint execute() throws Pausable { throw KilimError();}int execute(Fiber f) throws Pausable { msg = mbox.get(f); return msg.size();}


Kilim Rewritingint execute(Fiber f) throws Pausable { f.down(); msg = mbox.get(f); switch(f.up()) { case PAUSING|NO_STATE: f.setState(new State(locals)); case PAUSING|HAS_STATE: return 0; case RUNNING|HAS_STATE: locals = f.getState(); case RUNNING|NO_STATE: } return msg.size();}


Kilim Rewritingvoid execute(Fiber f) throws Pausable { switch (f.pc) { case 0: // default f.down(); mbox.get(f); switch(f.up()) { case PAUSING|NO_STATE: f.setState(new State(...locals...)); case PAUSING|HAS_STATE: return 0; case NORMAL|HAS_STATE: locals = f.getState(); case NORMAL|NO_STATE: } case 1: // default return msg.size(); }}


The ring!




Test Setup erjang01.hosted.netic.dk

• All run as JUnit tests (wrappers generated)

• Comparison runs Module:test() Erjang vs BEAM

• Running TRIQ tests comparison/local

• Run Erlang test_server based tests (single-node mode)

• Plain old JUnit tests


erl - boot time


erl - boot memory


EStones


Java 8

•MethodHandle•Tail calls•Coroutines

Would not be surprised to see Erjang 10x speedup

@drkrab


Erjang

•It’s real•It’s fast

• ... still lots to do.

@drkrab



Documents

Er jang: Erlang on the JVM - GOTO Conferencegotocon.com/dl/goto-cph-2011/slides/... · Erjang: Goals •Learn Erlang ... Java Virtual Machine Your favorite OS ERJANG Your Erlang Program