An Extensible Compiler Framework for Java · Ł Polyglot is a framework for writing compiler...

PolyglotAn Extensible Compiler Framework for Java

Nathaniel NystromMichael R. Clarkson

Andrew C. Myers

Cornell University

Language extension

� Language designers often create extensions to existing languages� e.g., C++, PolyJ, GJ, Pizza, AspectJ, Jif,

ArchJava, ESCJava, Polyphonic C#, ...

� Want to reuse existing compiler infrastructure as much as possible

� Polyglot is a framework for writing compiler extensions for Java

Requirements� Language extension

� Modify both syntax and semantics of the base language

� Not necessarily backward compatible� Goals:

� Easy to build and maintain extensions� Extensibility should be scalable

� No code duplication

� Compilers for language extensions should be open to further extension

Rejected approaches� In-place modification

� Macro languages� Limited to syntax extensions� Semantic checks after macro expansion

basecompiler

bug fixes &upgrades

basecompiler

copy &

modify

extensioncompiler

copy &

modify(again)

extensioncompiler

bug fixes &upgrades (again)

Polyglot

� Base compiler is a complete Java front end� 25K lines of Java

� Name resolution, inner class support, type checking, exception checking, uninitialized variable analysis, unreachable code analysis, ...

� Can reuse and extend through inheritance

Scalable extensibility

� Most compiler passes are sparse:

AST Nodes

Passes

Changes to the compiler should be proportional to changes in the language.

constant foldingexception checking

type checkingname resolution

=e.fxif+

AST Nodes

Passes

=e.fxif+

AST Nodes

Passes

=e.fxif+

AST Nodes

Passes

=e.fxif+

AST Nodes

Passes

=e.fxif+

Non-scalable approaches

Visitors

pass as ASTnode method(�naive OO�)

Polyglot

Easy to add or modifyPasses AST nodes

Javasource

Javatarget

Javaparser

Codegenerator

AST rewritingpasses

Base Polyglot compiler

Polyglot architecture

Extsource

Javatarget

Extparser

Codegenerator

AST rewritingpasses

sourceJava

target

parserCode

generatorAST rewriting

passes

Architecture details

� Parser written using PPG� Adds grammar inheritance to Java CUP

� AST nodes constructed using a node factory� Decouples node types from implementation

� AST rewriting passes:� Each pass lazily creates a new AST� From naive OO: traverse AST invoking a method

at each node� From visitors: AST traversal factored out

Example: PAO

� Primitive types as subclasses of Object� Changes type system, relaxes Java syntax� Implementation: insert boxing and unboxing

code where needed

HashMap m;m.put(�two�, 2);int v = (int) m.get(�two�);

HashMap m;m.put(�two�, new Integer(2));int v = ((Integer) m.get(�two�)).intValue();

PAO implementation� Modify parser and type-checking pass to

permit e instanceof int� Parser changes with PPG:

include �java.cup�

drop { rel_expr ::= rel_expr INSTANCEOF ref_type }

extend rel_expr ::= rel_expr:a INSTANCEOF type:b

{: RESULT = node_factory.Instanceof(a, b); :}

� Add one new pass to insert boxing and unboxing code

Implementing a new pass� Want to extend Node interface with rewrite() method

� Default implementation: identity translation� Specialized implementations: boxing and unboxing

� Mixin extensibility: extensions to a base class should be inherited by subclasses

codeGen()typeCheck()

condthenelse

lhsrhs

If Add

rewrite()codeGen()

typeCheck()

rewrite()

condthenelse

rewrite()

lhsrhs

Inheritance is inadequate

condthenelse

lhsrhs

If Add rewrite()codeGen()

typeCheck()

rewrite()

condthenelse

rewrite()

lhsrhs

PaoNode

PaoIf PaoAdd

Inheritance is inadequate

typeCheck()codeGen()

typeCheck()codeGen()typeCheck()

codeGen()typeCheck()codeGen()typeCheck()

codeGen()typeCheck()codeGen()

codeGen()

rewrite()codeGen()

typeCheck()

rewrite()

PaoNode

rewrite()

Extension objects

Use composition to mixin methods and fields into AST node classes

extcondthenelse

extlhsrhs

rewrite()ext

PaoExt

PAO extension objects; installed into all nodes

by node factory

Extension objects

Extension objects have their own ext field to leave extension open

extcondthenelse

extlhsrhs

rewrite()ext

PaoExt

ext_type_infotypeCheck()

Method invocation� A method may be implemented in the node or in

any one of several extension objects.

� Extension should call node.ext.ext.typeCheck()

� Base compiler should call: node.typeCheck()

� Cannot hardcode the calls

rewrite()ext

PaoExt

ext_type_infotypeCheck()

Delegate objects� Each node & extension object has a del field� Delegate object implements same interface as node or ext

� Directs call to appropriate method implementation� Ex: node.del.typeCheck()

� Ex: node.ext.del.rewrite()

� Run-time overhead < 2%

delext

Nodedel

rewrite()ext

PaoExt

ext_type_info

typeCheck()ext

codeGen()typeCheck() { node.ext.ext.typeCheck() }

{ node.codeGen() }

JavaDel

Scalable extensibility� To add a new pass:

� Use an extension object to mixin default implementation of the pass for the Node base class

� Use extension objects to mixin specialized implementations as needed

� To change the implementation of an existing pass� Use delegate object to redirect to method providing

new implementation

� To create an AST node type:� Create a new subclass of Node� Or, mixin new fields to existing node using an extension

object

Polyglot family tree

Polyglot base (Java)

parameterizedtypes

Coffer PolyJ Jif

Jif/split

JMatch covariantreturn

Results

� Can build small extensions in hours or days� 10% of base code is interfaces and factories

78129KJif65108KJMatch6099KJif/split

23.2Kparameterized types

4879KPolyJ

100% of Base

132Kjavac 1.11.6Kcovariant return

6.1KPAO24KCoffer

166KPolyglot base (Java)# TokensExtension

Related work

� Other extensible compilers� e.g., CoSy, SUIF� e.g., JastAdd, JaCo

� Macros� e.g., EPP, Java Syntax Extender, Jakarta� e.g., Maya

� Visitors� e.g., staggered visitors, extensible visitors

Conclusions

� Several Java extensions have been implemented with Polyglot

� Programmer effort scales well with size of difference with Java

� Extension objects and delegate objects provide scalable extensibility

� Download from:http://www.cs.cornell.edu/projects/polyglot

AcknowledgmentsJMatchBrandon Bray

Jif, Jif/splitLantian Zheng

Jif, Jif/splitSteve Zdancewic

JLtoolsDan Spoonhower

JLtoolsNaveen Sastry

JMatchJed Liu

JLtools, PolyJAleksey Kliger

JMatchMatt Harren

Jif, Jif/split, covariant returnSteve Chong

PPGMichael Brukman

http://www.cs.cornell.edu/projects/polyglot

Questions?

An Extensible Compiler Framework for Java · Ł Polyglot is a framework for writing compiler...

Documents

CON4265 McManus Plugging Into the Java Compiler

Seminar: Compiler-Konstruktion Realtime Java 1. Übersicht Echtzeitsysteme Warum Java? Java´s Echtzeitfähigkeit-Entwicklung Java´s Probleme und Lösungsansätze

Java Virtual Machine - Kirkwood Community · PDF filejava virtual machine Source code (java or other HLL) compiler/linker java compiler class file ... –Compiled Java program makes

Java Virtual Machine Guide - Oracle2 Compiler Control Compiler Control provides a way to control Java Virtual Machine (JVM) compilation through compiler directive options. The level

Web-Anwendungen mit Java - dekanat.cs.uni-dortmund.dedekanat.cs.uni-dortmund.de/Veranstaltungen/SS2002/skript/skript... · Web-Anwendungen mit Java 2 Übersicht Ł Java im Server

1.1 Introduction to Java - KFUPMConventional Program IOICOIOOOIOCCOOICOÍ n 1 1 101000010000100 Java Program 1110100100010000010 1000 Compiler SPARC Compiler Macintosh Compiler Java

Structure of Java Compiler

1 Introduction to Java and Applet. 2 Download Java Compiler (1)

LPR Corso A - di.unipi.itricci/Lezione2.pdfLPR Corso A a.a. 2005/2006 Lezione n. 2 Ł Gestione indirizzi IP Ł La classe JAVA InetAddress Ł Thread Pooling

A Retargetable Optimizing Java-to-C Compiler for … · “A Retargetable Optimizing Java-to-C Compiler for Embedded ... was developed at Bell Laboratories in ... type-polymorphism

Compiler-Enforced Immutability for the Java Language

Overview of the IBM Java Just-in-Time Compiler

The Java HotSpot Virtual Machine Client Compiler ...reins.altervista.org/java/javaOne2001_2696.pdf · 2696, The Java HotSpot™ Virtual Machine Client Compiler: Technology and 3 Application

USING THE OOSIML/JAVA COMPILER With Eclipse

JAVA—Bitwise. Compiling/Executing a Java Application Source Code Java Compiler ByteCode > Java Interpreter Machine Code >

compiler java

Compiler Design: Theory, Tools, And Examples. Java Edition [HQ]

Principles of Compiler Design - CSE - IIT Kanpur · PDF filePrinciples of Compiler Design ... –Java compiler: Java to interpretable bytecode ... Linker Programmer Source Program

Compiler Construction Java World

Modern Compiler Implementation in Java - …appel/modern/basic/java/extract.pdf · Preliminary edition of Modern Compiler Implementation in Java. ... 10.1 Solution of dataﬂow equations