In Vogue Dynamic

In Vogue Dynamic

Alexander Shopov <[email protected]>

[ash@edge ~]$ whoamiBy day: Software Engineer at Cisco {,Nov. 28}By night: OSS contributorCoordinator of Bulgarian Gnome TPGit speaks Bulgarian Contacts:

E-mail: [email protected] Jabber: [email protected] LinkedIn: http://www.linkedin.com/in/alshopov SlideShare: http://www.slideshare.net/al_shopovGitHub: https://github.com/alshopovWeb: Just search “al_shopov”

mailto:[email protected]

xmpp:[email protected]

http://www.linkedin.com/in/alshopov

http://www.slideshare.net/al_shopov

https://github.com/alshopov

http://lmgtfy.com/?q=al_shopov

Please Learn and Share

License: Creative Commons Attribution 4.0 InternationalCC-BY v4.0

http://creativecommons.org/licenses/by/4.0/

Disclaimer

My opinions, knowledge and experience!Not my employer's.

Menu for Today

● Numbers● Letters● Sentences● Poetry

Numbers

Hundred

100

Hundred Is a Wonderful Number

100 = (1 + 2 + 3 + 4)²

Hundred Is a Wonderful Hectonumber

100 = (1 + 2 + 3 + 4)² = 1³ + 2³ + 3³ + 4³


100 = (1 + 2 + 3 + 4)² = 1³ + 2³ + 3³ + 4³Νικόμ

αχος

was he

re!


100 = (1 + 2 + 3 + 4)² = 1³ + 2³ + 3³ + 4³Νικόμ

αχος

was he

re!A Very BoldComic Sans

Relief

Triple Hundred Is Even More Magical

100100100

Triple Hundred by Any Other Base Will Smell as Sweet...

●100100100₂●444₈●292₁₀

Our Last Numerological Slide…

292

The One and Only – JSR 292

● JSR 292 specified a new bytecode instruction: invokedynamic– First and only in all the history of Java platform and

technologies.

● There have been retired instructions– jsr, jsr_w – cannot appear in class file version 51.0 (JDK 7.0) and above;

– Together with ret were used for the finally block in Java < 1.5;

– Mainly used for obfuscation.

Letters

Bytecode? Who Cares? And It Is Not for Java…

● Well – it is also for Java, especially ≥ 8.0;● And it is also for dynamic languages;● And it is also for static languages;● And it is also for way too many other things;● But – yeah, the lecture is about bytecode, so

this is last chance to leave (run to the hills)…

Bytecodes Are the Letters of

JVM

Bytecode Crash Course in 1-2-3-4

Thr

ead

A

F0

F1

F2

F3

F4

Thr

ead

B

F0

F1

F2

Thr

ead

CF0

F1 Thr

ead

D

F0

F1

F2

F3


JVM (heap)

F0

Cla

ss

Pool of constants

0 1 2 3 4 5 6 …

Local variables

Stack

Method codePC

Class

Bytecode Crash Course in 1-2-3-4● 206 byte sized, typed instructions;● Bytecodes can encode information in themselves, retrieve

things from class bytestream, constant pool, local variables array;

● Perform computations only on stack;● Store temporary results in local variables array (non-visible

after method call);● Store more permanent results in object/class fields (visible

after method call);● Enter frame by method call;● Exit frame by a return from with a possible result (top thing on

stack), or by throwing top of stack;● JVM can also throw and unroll frames;


● You print the bytecode with the javap command:

javap -l -v -p -constants -s -cp CLASSPATH CLASSES

For More Info

● Check my other presentation: Lifting The Veil Reading Java Bytecode During Lunchtime

http://www.slideshare.net/al_shopov/lifting-the-veil-reading-java-bytecode-during-lunchtime

http://www.slideshare.net/al_shopov/lifting-the-veil-reading-java-bytecode-during-lunchtime

Most Bytecodes Are Very Dull

● They are lowercase letters – they continue or are a part of a single computational sequence:– The operate on a single stack;

– Inside a single frame;

● They are part of a single method call.

Sentences

The Uppercase Bytecodes

● Are the 5 bytecodes that invoke methods (start sentences) in some way.– invokestatic (0xb8)

– invokevirtual (0xb6)

– invokeinterface (0xb9)

– invokespecial (0xb7)

– invokedynamic (0xba)

The Punctuations

● Just for completeness – there are only 2 ways to end a sentence (method call):– Normal – returning by:

● areturn, dreturn, freturn, ireturn, lreturn, return;

– Abrupt – exception:● Either we throw via athrow● Or the JVM throws itself

Let's Check Simple Sentences

package org.kambanaria.standardinvoke;

public class Examples { public static int I_STATIC() { int a = Integer.parseInt("7"); return a; }}

Decompile…

Classfile /Java/StandardInvoke/target/classes/org/kambanaria/standardinvoke/Examples.class Last modified 2014-11-5; size 477 bytes MD5 checksum 2b499f0d33b9e2ed33181349f767bb95 Compiled from "Examples.java"public class org.kambanaria.standardinvoke.Examples SourceFile: "Examples.java" minor version: 0 major version: 51 flags: ACC_PUBLIC, ACC_SUPERConstant pool: #1 = Methodref #5.#19 // java/lang/Object."<init>":()V #2 = String #20 // 7 #3 = Methodref #21.#22 // java/lang/Integer.parseInt:(Ljava/lang/String;)I #4 = Class #23 // org/kambanaria/standardinvoke/Examples #5 = Class #24 // java/lang/Object #6 = Utf8 <init> #7 = Utf8 ()V #8 = Utf8 Code #9 = Utf8 LineNumberTable #10 = Utf8 LocalVariableTable #11 = Utf8 this #12 = Utf8 Lorg/kambanaria/standardinvoke/Examples; #13 = Utf8 I_STATIC #14 = Utf8 ()I #15 = Utf8 a #16 = Utf8 I #17 = Utf8 SourceFile #18 = Utf8 Examples.java #19 = NameAndType #6:#7 // "<init>":()V #20 = Utf8 7 #21 = Class #25 // java/lang/Integer #22 = NameAndType #26:#27 // parseInt:(Ljava/lang/String;)I #23 = Utf8 org/kambanaria/standardinvoke/Examples #24 = Utf8 java/lang/Object #25 = Utf8 java/lang/Integer #26 = Utf8 parseInt #27 = Utf8 (Ljava/lang/String;)I

{ public org.kambanaria.standardinvoke.Examples(); Signature: ()V flags: ACC_PUBLIC LineNumberTable: line 3: 0 LocalVariableTable: Start Length Slot Name Signature 0 5 0 this Lorg/kambanaria/standardinvoke/Examples; Code: stack=1, locals=1, args_size=1 0: aload_0 1: invokespecial #1 // Method java/lang/Object."<init>":()V 4: return LineNumberTable: line 3: 0 LocalVariableTable: Start Length Slot Name Signature 0 5 0 this Lorg/kambanaria/standardinvoke/Examples;

public static int I_STATIC(); Signature: ()I flags: ACC_PUBLIC, ACC_STATIC LineNumberTable: line 6: 0 line 7: 6 LocalVariableTable: Start Length Slot Name Signature 6 2 0 a I Code: stack=1, locals=1, args_size=0 0: ldc #2 // String 7 2: invokestatic #3 // Method java/lang/Integer.parseInt:(Ljava/lang/String;)I 5: istore_0 6: iload_0 7: ireturn LineNumberTable: line 6: 0 line 7: 6 LocalVariableTable: Start Length Slot Name Signature 6 2 0 a I}

But It Is Actually Simple…




Stats About Class

Constant Pool

Default Constructor

I_STATIC method

But It Is Actually Very Simple…




Stats About Class

Constant Pool

Default Constructor

I_STATIC code

I_STATIC stats 1

I_STATIC stats 2

The Whole Assembler

Code: stack=1, locals=1, args_size=0 0: ldc #2 2: invokestatic #3 5: istore_0 6: iload_0 7: ireturn

Physical Layout

Code: stack=1, locals=1, args_size=0 0: ldc #2 2: invokestatic #3 5: istore_0 6: iload_0 7: ireturn od -x --endian=big ./Examples.class 02 b8 00 03 3b 1a

0 1 2 3 4 5 6

ac

7

12

ldc Is 0x12, 1 Byte Reference to Constant Pool


0 1 2 3 4 5 6

ac

7

12

#2 in Constant Pool:String 7

invokestatic Is 0xb8, 2 Bytes Reference (00<<8+03==03)


0 1 2 3 4 5 6

ac

7

12

#3 in Constant Pool points:Method

java/lang/Integer.parseInt:(Ljava/lang/String;)I

istore_0 Is 0x3b, iload_0 Is 0x1a, ireturn Is 0xac


0 1 2 3 4 5 6

ac

7

12

Let's Check Another Sentence

public static void I_VIRTUAL() throws InterruptedException { "7".wait(3, 7); }

As Before – Push Arguments Onto Stack, Invoke Name + Signature

public static void I_VIRTUAL()Code: 0: ldc #2 // String 7 2: ldc2_w #4 // long 3l 5: bipush 7 7: invokevirtual #6 // Method java/lang/Object.wait:(JI)V10: return

Let's Check the Rest of Old Invoke Instructions

public static int I_INTERFACE(){ return "7".compareTo("8"); } public static Object I_SPECIAL(){ return new Integer(5); } // also private methods, super

As Before – Push Arguments Onto Stack, Invoke Name + Signature

I_INTERFACE();Code:0: ldc #2 // "7"2: ldc #7 // "8"4: invokevirtual #8// Method // String.compareTo7: ireturn

I_SPECIAL();Code:0: new #9//class j/l/Integer3: dup4: iconst_55: invokespecial #10//Method Integer.//"<init>":(I)V8: areturn

Fundamentally Traditional ∀Invokes:

● Are scoped Java-ish (class or interface);● Have a Java-ish receiver (single or none);● And definitely do Java-ish dispatch;● Everything else – you have to simulate.

Now Here Comes Java 8, Functional Interfaces, Lambdas

public static void notEmpty(Collection<String> all) { all.removeIf(String::isEmpty);}

Which is Syntactic Sugar for an Anonymous Inner Class

public static void notEmptyOld(Collection<String> all) { all.removeIf(new Predicate<String>() { @Override public boolean test(String t) { return t.isEmpty(); } });}

Which is Syntactic Sugar For An Anonymous Inner Class


New ObjectPer Invocation

Which is Syntactic Sugar For An Anonymous Inner Class


New ObjectPer Invocation

A Whole NewClass

Improve by Caching

public static void nonEmptyOldNeat(Collection<String> all) { all.removeIf(nonEmptyP);}

private static Predicate<String>nonEmptyP = new Predicate<String>() { @Override public boolean test(String t) { return t.isEmpty(); }};

Improve by Caching

public static void nonEmptyOldNeat(Collection<String> all) { all.removeIf(nonEmptyP);}

private static Predicate<String>nonEmptyP = new Predicate<String>() { @Override public boolean test(String t) { return t.isEmpty(); }};

A Whole NewClass

Let's Disassemble nonEmpty

nonEmpty Code: 0: aload_0 //Load ref to col on stack 1: invokedynamic #2, 0 // Here's Waldo // Magic! 6: invokeinterface #3, 2 // Collection.removeIf 11: pop // pop boolean from stack 12: return // we are ready}

Let's Disassemble nonEmpty

1: invokedynamic #2, 0// That is ba 00 02 00 00 when dumped// #2 in constant pool is: InvokeDynamic #0:test:()Ljava/util/function/Predicate;// void -> Predicate, has index 0, OK // Where is String.isEmpty?

At the End of the Class

BootstrapMethods:0: #25 invokestatic j/l/invoke/LambdaMetafactory.metafactory:( Ljava/lang/invoke/MethodHandles$Lookup; Ljava/lang/String; Ljava/lang/invoke/MethodType; Ljava/lang/invoke/MethodType; Ljava/lang/invoke/MethodHandle; Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;

Method arguments: #26 (Ljava/lang/Object;)Z #27 invokevirtual j/l/String.isEmpty:()Z #28 (Ljava/lang/String;)Z




Index 0




Fat Factory – Gives CallSite, not

Predicate




Here Is the Test

Let's Try a More Realistic nonEmpty

public static void nonEmpty(Collection<String> all) { all.removeIf(s -> s == null || s.trim().isEmpty());}

The nonEmpty Method Stays the Same

nonEmpty Code: 0: aload_0 //Load ref to col on stack 1: invokedynamic #2, 0 // Here's Waldo // Magic! 6: invokeinterface #3, 2 // Collection.removeIf 11: pop // pop boolean from stack 12: return // we are ready}

We Get a Private, Static, Synthetic Method lambda$nonEmpty$0

lambda$nonEmpty$0 // (Ljava/lang/String;)Z Code: 0: aload_0 1: ifnull 14 4: aload_0 5: invokevirtual #4 // String.trim 8: invokevirtual #5 // String.isEmpty 11: ifeq 18 14: iconst_1 15: goto 19 18: iconst_0 19: ireturn

Which in Java Terms is

private static void lambda$nonEmpty$0(String s) { return s == null || s.trim().isEmpty();}

The Bootstap Glue Becomes


Method arguments: #24 (Ljava/lang/Object;)Z #25 invokestatic org/kambanaria/invokedynamic/ Examples.lambda$nonEmpty$0:(Ljava/lang/String;)Z #26 (Ljava/lang/String;)Z

Seems We Are Winning

● Lambdas get to be called with invokedynamic● Which refers some glue code (bootstrap

method)● Which refers either:

– Some existing method (String::isEmpty);

– A new custom method (lambda$nonEmpty$0).

● No need to reuse a static instance;● No new inner class!

Another Win for the Fundamental Theorem of Software Engineering

● David Wheeler● But what is that theorem?

The Traditional Invoke Instructions

● Have a precise Java-ish meaning;● They point to some executable code by:

– Name;

– Signature.● Somewhat eased by boxing/varargs.

– What they actually point to is a string in the constant pool that gives name + signature.

● Given that – JVM does a predefined resolution.

JVM

Old Invokes

invoke

type

Name+

signature

JVM

Methodcode

Methodcode

Methodcode

Decision by JVM

JVM

Invokedynamic – We Get to Decide(This Is not the Whole Picture)

invoke

dynamic

BootstrapMethod→CallSite

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

Decision by Us Decision by Us

Decision by Us

The Fundamental Theorem of Software Engineering

All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections.

David Wheeler

Poetry

Prose ↔ Poetry

● Using invokedynamic we can tell the JVM that a particular phrase (callsite) points to a particular code – has a particular meaning;– This can be very dull, repetitive and prosaic;

● In poetry – you can talk about one thing but mean another;– And even when things are ambiguous or just not

the right types, number or order – it works fine;

– And it is always in vogue;

– And as rapping has shown – it can be fast!

How Does Invokedynamic Exactly Work

● A dynamic call site is every lexical occurrence of invokedynamic;– Thus EVERY occurrence can behave differently;

● The instruction is followed by 4 bytes, only the fist 2 of them are currently used, the rest must be 0;

● The 1st 2 bytes point to an index in the constant that contains a call site specifier that leads to a j.l.invoke.MethodHandle that serves as a bootstrap method;

● The bootstrap method is invoked and returns a j.l.invoke.CallSite, this is bound permanently to this call site;

● The CallSite object provides a MethodHandle via getTarget() method which provides the exact behavior.

● The result of getTarget() CAN change!

JVM

Invokedynamic

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

JVM

Invokedynamic

invoke

dynamic

MethodHandle

Bootstrap

JVM

Methodcode

Methodcode

Methodcode

JVM

Invokedynamic

invoke

dynamic

MethodHandle

Bootstrap

JVM

Methodcode

Methodcode

Methodcode

CallSite

JVM

Invokedynamic

invoke

dynamic

MethodHandle

Bootstrap

JVM

Methodcode

Methodcode

Methodcode

CallSite

JVMC

allS

ite

Invokedynamic

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

JVMC

allS

ite

Invokedynamic

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

MethodHandle

MethodHandle

JVMC

allS

ite

Invokedynamic

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

MethodHandle

MethodHandle

JVMC

allS

ite

Invokedynamic

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

MethodHandle

MethodHandle

MethodHandles vs. Reflection

● Reflections allows you to look at yourself and discover things, they are closer to Java language constructs;

● MethodHadles assume you know what you are looking for – no exploration, but they are closer to the way JVM functions – they are fast.– Using them is like shaving in the dark – you have to

know where the razor is, how much foam to squeeze, what you are shaving but you can do it fast since your hands know their way – they have memory of their own.

But Wait – Types Must Match!

● You cannot pass some number of arguments to a method that expects another number– Except varargs

● You cannot pass different types of arguments– Except boxing/unboxing

● You cannot reorder;● You cannot group;● You cannot aggregate.

You Cannot Put a Square Peg In a Round Whole

But You Actually Can

Reshape the Peg to Be No Longer Square But Round and It Fits

JVMC

allS

ite

Method Handles Know Types

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

MethodHandle

JVMC

allS

ite

But You Can Convert/Reshape/ Rechisel the JVM Way

invoke

dynamic

JVM

Methodcode

Methodcode

Methodcode

MethodHandle

MethodHandle

MethodHandle

MethodHandle

java.lang.invoke In a Nutshell

● CallSite (Constant/Mutable/Volatile) – produces MethodHandle

● MethodType – represents types● MethodHandle – reference to method, (static) field, array

member access;● MethodHandles – combining or producing handles;● MethodHandles.Lookup – get a MethodHandle referring to

actual method, filed, array member;● MethodHandleProxies – proxies – interface-handle;● SwitchPoint – fast, one-way if-then-else for handles;

Example – Curry

public class Computer { public static int sum (int a, int b){ return a + b; }}

Example – Curry

Computer.sum(2, 7); // => 9// What we can seeMethodHandles.Lookup lookUp = MethodHandles.lookup();MethodType sumType = MethodType.methodType( // result, 1 arg, , 2 arg int.class, int.class, int.class);MethodHandle sumHandle = lookUp.findStatic( Computer.class, "sum", sumType);sumHandle.invoke(2, 7); // => 9MethodHandle add2Handle = MethodHandles. insertArguments(sumHandle, 0, 2);add2Handle.invokeWithArguments(7); // => 9

Example – Switch Receiver and Arg

"tom".concat("cat"); // => "tomcat""cat".concat("tom"); // => "cattom"

Example – Switch Receiver and Argument

MethodType catType = MethodType.methodType(// result, argument String.class, String.class);MethodHandle catHandle = lookUp.findVirtual( String.class, "concat",catType);catHandle.invoke("tom", "cat"); // "tomcat"

MethodType tacType = MethodType.methodType(// result, receiver, argument String.class, String.class, String.class);MethodHandle tacHandle = MethodHandles. permuteArguments(catHandle, tacType, 1, 0);tacHandle.invoke("tom", "cat"); // "cattom"

Rhymes And Allusions

● Rhymes get from repetition of sounds – and you get to make MethodHandles from other MethodHadles all the way turtles go;

● You can adapt and convert types – thus you can use a MethotHandle with one type for something else – with a different type. You can write one stanza but use it for many contexts;

● JVM actually understands this algebra of handles and types and can optimize it.

If You Are Not So Much Into Poetry

Shameless Plug for Geertjan & NetBeans




Further Resources

● Nutter, Ch. (headius – JRuby guy)– Invokedynamic: Changing the JVM Forever

– Invokedynamic in 45 minutes– Indy Deep Dive

● Worthington, J.: Using invokedynamic to teach the JVM a new language

● Rose, J.: Bytecodes meet combinators: invokedynamic on the JVM

● Ponge, J: Demystifying invokedynamic, Java Magazine 2013, January-February: Part 1, May-June: Part 2

● Forax, R.: jsr292-cookbook

https://www.youtube.com/watch?v=BOOXa06y96A

http://www.slideshare.net/CharlesNutter/invokedynamic-in-45-minutes

https://github.com/headius/indy_deep_dive

http://www.infoq.com/presentations/invokedynamic

http://cr.openjdk.java.net/~jrose/pres/200910-VMIL.pdf

http://www.oraclejavamagazine-digital.com/javamagazine/20130102?folio=49

http://www.oraclejavamagazine-digital.com/javamagazine/20130506?folio=41

https://code.google.com/p/jsr292-cookbook/

Images

● rosipaw: 146/365 square peg into a round holeCC BY-NC-SA 2.0

● Eric: Lego Porn CC BY-NC-SA 2.0

https://www.flickr.com/photos/rosipaw/4643095630

https://www.flickr.com/photos/ejpphoto/2314610838