Eric Lafortune - The Jack and Jill build system

The Jack and Jillbuild system

Eric Lafortune

Developer of ProGuard and DexGuard

CTO at GuardSquarewww.guardsquare.com

The Jack and Jill build system

Inside the build systemand the runtime

Practical implications and results

Jack and Jill

Java Android Compiler Kit

Jack Intermediate Library Linker

Jack and Jill

● Now default in build process of AOSP

● Optional in Gradle build process:

android { defaultConfig { useJack true ... } ...}

android { defaultConfig { useJack true ... } ...}

com.android.tools.build:gradle:2.0.0

android { defaultConfig { jackOptions { enabled true jackInProcess true } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 }

... } ...}


... } ...}

Jack and Jillcom.android.tools.build:gradle:2.1.0 alpha1


... } ...}


... } ...}

Jack and Jill

...:generateReleaseSources:jillReleasePackagedLibraries:jillReleaseRuntimeLibraries:processReleaseJavaRes UP-TO-DATE:transformResourcesWithMergeJavaResForRelease:compileReleaseJavaWithJack:compileReleaseNdk UP-TO-DATE:compileReleaseSources...

...:generateReleaseSources:jillReleasePackagedLibraries:jillReleaseRuntimeLibraries:processReleaseJavaRes UP-TO-DATE:transformResourcesWithMergeJavaResForRelease:compileReleaseJavaWithJack:compileReleaseNdk UP-TO-DATE:compileReleaseSources...

com.android.tools.build:gradle:2.1.0 alpha1

Why?

Technical

Legal ?

Strategical ?

Android build process

Java bytecode(.class)

Libraries(.class)

Dx

JavacSource code

(.java)

Libraries(.class)

XML resources(.xml)

Assets Assets

CompiledXML resources

(.xml)Aapt

Dalvik bytecode(classes.dex)

Androidruntime stubs

(.class)

Transform API


Processor

Libraries(.class)

TransformedJava bytecode

(.class)Dx

Javac

Assets


(.xml)Aapt


Source code(.java)

Libraries(.class)

XML resources(.xml)

Assets

Runtime stubs(.class)

ProGuard


ProGuard

Libraries(.class)

OptimizedJava bytecode

(.class)Dx

Javac

Assets


(.xml)Aapt


ProGuardconfig(.txt)

ProGuardmapping

(.txt)

Source code(.java)

Libraries(.class)

XML resources(.xml)

Assets


ProGuard


ProGuard

Libraries(.class)

OptimizedJava bytecode

(.class)Dx

Javac

Assets


(.xml)Aapt



ProGuardmapping

(.txt)

Source code(.java)

Libraries(.class)

XML resources(.xml)

Assets


Jack and Jill

Libraries(.jayce)

Jack

Assets


(.xml)Aapt



ProGuardmapping

(.txt)

Source code(.java)

Libraries(.class)

XML resources(.xml)

Assets


Jill

Jill Runtime(.jayce)

Code representations

for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ...

iconst 0istore v0iload v0bipush 8ificmpge +12...iinc v0, 1goto -12


.class .dex

.java

Javac

Dx

const/4 v0, #0const/16 v1, #8if-ge v0, v1, 8 ...add-int/lit8 v0, v0, #1goto -9






.class .dex

.java

Javac

Efficient to store and executeInconvenient to process

Dx



LOCAL int n

FOR_STATEMENT:- EXPRESSION_STATEMENT: - ASG_OPERATION: - LOCAL_REF i - INT_LITERAL 0- LT_OPERATION: - LOCAL_REF i - INT_LITERAL 8- POSTFIX_INC_OPERATION: - LOCAL_REF i- ...

LOCAL int n

FOR_STATEMENT:- EXPRESSION_STATEMENT: - ASG_OPERATION: - LOCAL_REF i - INT_LITERAL 0- LT_OPERATION: - LOCAL_REF i - INT_LITERAL 8- POSTFIX_INC_OPERATION: - LOCAL_REF i- ...





.class .dex

.java

Javac

.jayce

Jack



Jill

Jack

Implications of Jack and Jill

● Java annotation processors – supported in JackE.g. Dagger, Butterknife

● Bytecode processors – only supported via JillE.g. JaCoCo, Retrolambda

● Other JVM languages – only supported via JillE.g. Scala, Groovy, Dart

● Instant Run – not supported yet

Implications of Jack and Jill

Libraries(.jayce)

Jack

Assets


(.xml)Aapt


Javac annotation processorsE.g. Dagger, Butterknife

Source code(.scala,...)

Libraries(.class)

XML resources(.xml)

Assets

Runtime(.class)

Jill

Jill Runtime(.jayce)

Bytecode processors

Java bytecode(.class)Scalac

Application(.jayce)Jill

Internals

Jack: Java frontendEclipse ecj library

Jill: Java frontendASM4 library

Java 8: closures.....button.setOnClickListener(view -> .....);.....

.....button.setOnClickListener(view -> .....);.....

.....button.setOnClickListener(new MyActivity$LambdaImpl());.....

.....button.setOnClickListener(new MyActivity$LambdaImpl());.....

public class MyActivity$LambdaImpl {

public void onClick(View view) { ..... }}

public class MyActivity$LambdaImpl {

public void onClick(View view) { ..... }}

Jack

For all versionsof Android

Device

More changes!

RunDalvik bytecode(classes.dex)

Source code(.java)

Libraries(.class)


Build Dalvik bytecode(classes.oat)

AndroidruntimeJack & Jill

OpenJDK


Source code(.java)

Libraries(.class)


Android +OpenJDKruntime

Android +OpenJDK

runtime stubs(.class)

OpenJDK

Device

OpenJDK

Java 8: default methods

● In your own code:

● In the Android runtime:

public interface java.util.List<E> extends java.util.Collection<E> { public abstract int size(); ... public default void sort(java.util.Comparator<? super E>) { ..... }}

public interface java.util.List<E> extends java.util.Collection<E> { public abstract int size(); ... public default void sort(java.util.Comparator<? super E>) { ..... }}

public interface Foo { public abstract void someMethod();

public void someDefaultMethod() { .... }}

public interface Foo { public abstract void someMethod();

public void someDefaultMethod() { .... }}

Only onAndroid N

New instructions

Dalvik bytecode


Source code(.java)

Libraries(.class)


Android +OpenJDKruntime

Android +OpenJDK

runtime stubs(.class)

New instructions

Device

Java 8: closures

.....

● put-field-lambda● get-field-lambda● put-static-lambda● get-static-lambda● aget-lambda● aput-lambda● move-lambda● cmp-lambda● return-lambda

.....

● put-field-lambda● get-field-lambda● put-static-lambda● get-static-lambda● aget-lambda● aput-lambda● move-lambda● cmp-lambda● return-lambda Only on

Android N+

New Dalvik instructions:

Build performance 44s – Javac + dx (clean build)

1m9s – Javac + ProGuard optimize + dx (clean build)

24s – Javac + DexGuard (clean build)

2m36s – Javac + DexGuard optimize (clean build)

1m17s – Jack (clean build)

47s – Jack (clear all dex files)

15s – Javac + dx (clear all class files) 15s – Javac + dx (clear all class files)

5s – Javac + dx (clear one class file) 5s – Javac + dx (clear one class file)

2s – Javac + dx (code unchanged) 2s – Javac + dx (code unchanged)

40s – Jack (clear one dex file) 40s – Jack (clear one dex file)

3s – Jack (code unchanged) 3s – Jack (code unchanged)

Google I/O app

(AMD A10-7800, Ubuntu 14.04, Java 8, Android Gradle 2.1.0 alpha1, Gradle daemon)

49s – Jack minify (clear all dex files) 49s – Jack minify (clear all dex files)

21s – Javac + ProGuard + dx (clean build) 21s – Javac + ProGuard + dx (clean build)

Application runtime performance

50,000 – Javac + dx

63,800 – Javac + ProGuard optimize + dx

82,300 – Javac + DexGuard optimize

54,500 – Jack

Caffeinemark benchmark

(Nexus 5, Android 6.0, ART)

+9%

+28%

+65%

(higher is better)

Application size

3.1M – Javac + dx

1.9M – Javac + ProGuard optimize + dx

1.8M – Javac + DexGuard optimize

1.9M – Jack + minify

Google I/O app

-39%

-38%

-44%

(smaller is better)

DexGuard


DexGuard

Libraries(.class)

Javac

Optimizedassets

OptimizedXML resources

(.xml)Aapt

OptimizedDalvik bytecode

(classes.dex)


ProGuardmapping

(.txt)

Source code(.java)

Libraries(.class)

XML resources(.xml)

Assets


(.class)


(.xml)


(.class)

Assets

Native code(.so)Clang

Source code(.c / .cpp)

Optimizednative code

(.so)

Conclusions

● Huge changes● Clean implementation

Internally

Conclusions


● Different build process● New intermediary representation

Internally

For languages and tools

Conclusions


● More efficient builds● Java 8

● Different build process● New intermediary representation

Internally

For languages and tools

For app developers

Further reading

● “Experimental New Android Tool Chain - Jack and Jill”http://tools.android.com/tech-docs/jackandjill

● “Java 8 Language Features”https://developer.android.com/preview/j8-jack.html

● “Jack (Java Android Compiler Kit)”https://source.android.com/source/jack.html

● AOSP repositoryhttps://android.googlesource.com/toolchain/jack/

● “State of the ART”http://androidbackstage.blogspot.com/2016/03/episode-45-state-of-art.html

Questions?

Size

Java bytecode

Performance

Jayce bytecode

Dalvik bytecode

Jack & Jill

ProGuard

DexGuard

Software

Eric Lafortune - The Jack and Jill build system