ZetaVM, a Platform for Programming Language Innovation · Love of Programming • As a kid, was extremely curious about machines • Was very into electronics, but cost was a barrier

ZetaVM, a Platform for Programming Language Innovation

Maxime Chevalier Boisvert @ PolyConf 2017

Love of Programming• As a kid, was extremely curious about machines

• Was very into electronics, but cost was a barrier

• Started playing with code at 9-10, using QBasic on a 386

• Started to program seriously at age 15, learned C++

• I love programming for its own sake, as a creative outlet

• If you program enough, you run into limitations

• Want more expressive language to express ideas

• Wanted to understand how programming languages are made

• Went on to get a PhD in compiler design

• I can make a mean compiler, but still not sure what my ideal language is like

What is ZetaVM?• Virtual Machine: virtual machine for dynamic languages

• Primary goals:

• Make language creation and experimentation more accessible

• Explore a few risky VM design ideas, shake things up

• Serve as a proof of concept, inspire other systems

• I would like Zeta to:

• Try a different take on computing platforms

• Become fun playground for programming language exploration

Notable Features

• Text-based image file format

• Built-in support for dynamic typing

• First-class bytecode

• Optimizing interpreter (JITterpreter)

• Multi-language support (language packages)

Basic Types• Basic data types supported by ZetaVM:

• boolean

• int32, float32

• object: dynamically growable objects (JS-like)

• array: dynamically extensible arrays (JS/Python-like)

• string: immutable UTF-8 strings

• Everything else is built from these basic types

• Why dynamic typing?

Textual Image Format• Smalltalk: concept of an image, store heap to disk

• The Zeta Image Format (ZIM)

• Zeta’s native program and package format

• Resembles JSON, but allows circular references

• Textual description of a graph of objects

• Can describe both code and plain data

# Functions are just objects containing data which # describes the function, its arguments, local variables # and the bytecode that defines its behavior. sum_fn = { name: "sum", params: ['x', 'y'], num_locals: 3, entry: { name: "sum_entry", instrs: [ { op: "get_local", idx: 0 }, { op: "get_local", idx: 1 }, { op: "add_i32" }, { op: "ret" } ] } };

# Packages define object containing exported definitions { sum: @sum_fn };











First-Class Bytecode• A bit of Lisp, code is data

• Zeta functions are objects, but not in the JS sense

• In JS, functions pretend to be objects

• In Zeta, functions are made of objects, arrays, etc.

• Can generate bytecode on the fly

• Can access the code for existing functions

• Currying expressible through bytecode manipulation

/** Do currying, functional-style partial evaluation of a function with respect to its last argument. */ var curry = function (fun, argVal) { var numParams = fun.params.length;

assert ( numParams > 0, "cannot curry function with no arguments" );

var newEntry = { instrs: [ { op:'push', val:argVal }, { op:'set_local', idx:(numParams - 1) }, { op:'jump', to:fun.entry } ] };

var newParams = []; for (var i = 0; i < numParams - 1; i += 1) newParams:push(fun.params[i]);

var newFun = { params: newParams, num_locals: fun.num_locals, entry: newEntry };

return newFun; };






return newFun; };






return newFun; };






return newFun; };






return newFun; };

var peval = import "std/peval/0"; var curry = peval.curry;

/// Produces a sine wave of a given frequency var sine = function (freq) { var f = function (time, freq) { return math.sin(time * freq * 2 * math.PI); };

return curry(f, freq); };

// Pluck sound, as a function of time var pluck = function (freq) { return mul( saw(freq/2.0f), ADEnv(0.005f, 0.1f) ); };

var peval = import "std/peval/0"; var curry = peval.curry;

/// Produces a sine wave of a given frequency var sine = function (freq) { var f = function (time, freq) { return math.sin(time * freq * 2 * math.PI); };

return curry(f, freq); };

// Pluck sound, as a function of time var pluck = function (freq) { return mul( saw(freq/2.0f), ADEnv(0.005f, 0.1f) ); };

JITterpreter• Zeta has an interpreter, does not generate machine code yet

• Problem: interpreting object bytecode directly is dog slow

• Lots of pointer indirection, object property lookups, validation

• Zeta’s solution: lazily translate object-based bytecode

• Produce flat and compact internal representation

• Opportunity to perform various optimizations

• Zeta’s interpreter does some amount of JIT compilation

• New interpreter is about 25x faster

• Fast enough to do (basic) real-time audio generation

• Nearing CPython’s speed, will get much faster

Plush Language• Inspired by JavaScript and Lua

• Two main goals:

• Bootstrap and test the platform

• Serve as a language implementation example

• Simple semantics

• No closures, no getters & setters

• Will intentionally remain simple

• Two implementations

• cplush, produces ZIM files

• Plush language package

Multi-Language Support

• Zeta supports multiple languages natively by delegating parsing to language packages

• You can add support for your language by creating a package which parses source code into Zeta byte code

• Source files begin with a “language line” which tells Zeta which language package (parser) to load

• The cplush compiler compiles the self-hosted Plush parser into a ZIM package

#language "lang/plush/0"

var fib = function (n) { if (n < 2) return n;

return fib(n-1) + fib(n-2); };

var r = fib(7);

print(r);

assert (r == 13);

#language "lang/plush/0"

var fib = function (n) { if (n < 2) return n;

return fib(n-1) + fib(n-2); };

var r = fib(7);

print(r);

assert (r == 13);

Looking Forward

Package Manager• Zeta’s package manager will be integrated with the VM

• Packages downloaded automatically

• Language packages accessible through the PM

• Can make your language usable instantly

• Packages will be versioned and immutable

• Can’t suddenly change under you

The Code Rot Problem• Ambitious goal: tackle the problem of code rot

• IMO: one of the biggest problems in modern software development

• Code breaks constantly, even if it doesn’t change

• Huge cause of reliability issues and time wasted

• This is somehow accepted as normal

• Problem: foundations constantly shifting under your feet

Tackling Code Rot• Could we design programs so they will run in 20, 30, 50 years? How?

• There is a class of systems where this is possible

• Can’t guarantee that the outside world won’t change

• Can design systems to reduce probability of breakage

• Zeta’s approach:

• Intentional minimalism: less surface area, less corner cases

• Simple, low-level APIs: basics are unlikely to change

• Avoiding undefined behaviors: pick reliability over performance

• Immutable, versioned package system: functional approach

Functional Graphics• Zeta will take a functional approach to 2D & 3D graphics

• Zeta will provide a parallel map/forall operator for graphics

• Parallelizable through SPMD execution model

• Same code can run on both CPU and GPU

• Back to the 80s: no polygons, plot individual pixels

• Simple, minimalistic, unchanging API: just one function

• Inspiration: signed distance fields, demoscene

• Contrast:

• Java: expose a billion UI widgets

• Python: leave graphics to external packages

Progress so far• VM with simple optimizing interpreter

• Basic file I/O, audio and graphics APIs

• Beginnings of a standard library (math, random)

• C++ Plush compiler implementation (cplush)

• Self-hosted Plush implementation (compiled with cplush)

• Espresso compiler implementation (Python)

Roadmap• Serialization, complete Plush bootstrap

• Garbage collector

• Package manager

• Incremental inlining (in the interpreter)

• JIT for x86-64

• SPMD programming support

Conclusion• ZetaVM is a young, open source VM with a different take on

computing platforms

• ZetaVM’s aims to allow you to:

• Create a working language with very little code

• Instantly publish your new language

• Explore new language ideas easily

• Eventually: ecosystem with a wide variety of languages

• Looking for contributors to help build this system

Check out github.com/zetavm Follow @Love2Code on twitter

And a warm thank you to all contributors. This project would not be possible without you:

Marco Concetto Rudilosso, Tommy Ettinger, Kartik Agaram, Luca Barbato, Meador Inge, Dirk Gadsden, krypt-n, and more

http://github.com/zetavm

Documents

ZetaVM, a Platform for Programming Language Innovation · Love of Programming • As a kid, was extremely curious about machines • Was very into electronics, but cost was a barrier