© 2013 IBM Corporation
Ryan A. Sciampacone – Managed Runtime Architect
25 September 2013
An Introduction to
Packed Objects
© 2013 IBM Corporation
Important Disclaimers
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.
IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.
IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
2
© 2013 IBM Corporation
Your Speaker: Ryan A. Sciampacone
• Runtime Architect @ IBM (JTC)• Interpreters• Garbage Collection• Platform interfacing / optimization• Hardware exploitation
(E) [email protected](T) @rsciampacone
• Visit the IBM Booth #5112 to meet other IBM Developers
3
© 2013 IBM Corporation
PROBLEM SPACEWhat are we trying to solve here?
4
© 2013 IBM Corporation
Problem? What problem?
• JNI just isn’t a great way to marshal data
• Locality in Java can matter (e.g., JEP 142)
• Existing native and data placement stories aren’t very good
• In many cases, legacy systems exist – the interop is just terrible
• So we want something that integrates well with the Java language and helps us…
5
© 2013 IBM Corporation
Native Access
Fighting the Java/Native interface
6
intint
Java Native
…
int[] d
anObject
intint
int…
Object header
Object field / data
© 2013 IBM Corporation
Everything is an Object
7
Hash
Array
Entry
(object)
(object)table
key
value
Object header
Object field / data
© 2013 IBM Corporation
Everything is an Object
8
Hash
Array
Entry
(object)
(object)table
key
value
Object header
Object field / data
© 2013 IBM Corporation
Everything is an Object
9
Hash
Array
Entry
(object)
(object)table
key
value
Object header
Object field / data
© 2013 IBM Corporation
Everything is an Object
10
Hash
Array
Entry
(object)
(object)table
key
value
Object header
Object field / data
© 2013 IBM Corporation
Object Internals
• Field ordering has performance implications
• JVM can potentially reorder your fields for you
11
….“Hot” fields
Object header
Object field / data
© 2013 IBM Corporation
Establishing Goals
• On heap / off heap seamless referencing of data
• Ability to do away with headers
• Ability to bring related objects close together
12
struct Address {char[4] addr;short port;
}struct Header {
struct Address src;struct Address dst;
}
struct Header
portaddr
Address src
Address dstportaddr
Packed Objects!
© 2013 IBM Corporation
THE BASICSFrom the ground up, let’s start with…
13
© 2013 IBM Corporation
Packed Objects: Under the covers
14
int yint x
aPointObject header
Object field / data
© 2013 IBM Corporation
Packed Objects: Under the covers
15
int yint x
aPoint
offsettarget
aPackedPoint
int yint x
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: Under the covers
16
int yint x
aPoint
offsettarget
aPackedPoint
int yint x
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice
17
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice
18
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
aPackedLine
offsettarget
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice
19
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
aPackedLine
offsettarget
Object header
Object field / data
aPackedPoint s
aPackedPoint e
© 2013 IBM Corporation
Packed Objects: In Practice
20
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
aPackedLine
offsettarget
Object header
Object field / data
@Packedfinal class PackedPoint extends PackedObject {
int x;int y;
}
@Packedfinal class PackedLine extends PackedObject {
PackedPoint s;PackedPoint e;
}
aPackedPoint s
aPackedPoint e
© 2013 IBM Corporation
Packed Objects: In Practice
21
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
aPackedLine
offsettarget
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice
22
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
aPackedLine
offsettarget
Object header
Object field / data
aPackedLine.e
© 2013 IBM Corporation
Packed Objects: In Practice
23
int yint x
aPoint
int yint x
aPointPoint ePoint s
aLine
int yint x
int yint x
offsettarget
aPackedPoint
aPackedLine
offsettarget
aPackedLine.e
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice with Arrays
24
int yint x
int yint x
offsettarget
aPackedPoint
anArrayOfPackedPoints
offsettarget
int yint x
int yint x
….
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice with Native Access
25
int yint x
Java Native
struct Point {int x;int y;
}
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice with Native Access
26
int yint x
offsettarget
aPackedPoint
Java Native
@Packedfinal class PackedPointextends PackedObject {
int x;int y;
}
struct Point {int x;int y;
}
Object header
Object field / data
© 2013 IBM Corporation
Packed Objects: In Practice with Native Access
27
int yint x
offsettarget
aPackedPoint
Java Native
Ø
@Packedfinal class PackedPointextends PackedObject {
int x;int y;
}
struct Point {int x;int y;
}
Object header
Object field / data
© 2013 IBM Corporation
ADVANTAGESSo what do we get?
28
© 2013 IBM Corporation
Lets Build Something in C!
• Nested substructures
• Compact
• Alignment
29
struct Address {char[4] addr;short port;
}struct Header {
struct Address src;struct Address dst;
}
struct Header
portaddr
Address src
Address dstportaddr
© 2013 IBM Corporation
Let’s Build the Same “Something” in Java!
• Headers
• Locality
• Alignment
30
class Address {byte[] addr;short port;
}
class Header {Address src;Address dst;
}
Address
portaddr
addrHeader
Address dstAddress src
Address
portaddr
addr
byte[]
byte[]
© 2013 IBM Corporation
Java code: What’s it look like under the covers?
• From a code point of view, this isn’t terrible…
31
Bytecodes:aload1getfield Header.dest LAddress;getfield Address.addr [Biconst0baloadbipush 192ificmpeq ...
JIT (32 bit):mov EBX, dword ptr -4[ECX] // load temp1mov EBX, dword ptr 8[EBX] // load destmov EBX, dword ptr 4[EBX] // load addrmovsx EDI, byte ptr 8[EBX] // array[0]cmp EDI, 192
© 2013 IBM Corporation
What if we did this with Packed Objects?
32
@Packedfinal class Address extends PackedObject {
PackedByte[[4]] addr;short port;
}
@Packedfinal class PacketHeader extends PackedObject{
Address src;Address dest;
}
portaddr Address src
Address dstport
addr
offsettarget
aPackedHeader
• The Java code is pretty clean… and a pretty good result!
© 2013 IBM Corporation
Ok, what about the code under the covers?
• Bytecodes don’t change… JIT code is pretty good too!
33
Bytecodes:aload1getfield Header.dest LAddress;getfield Address.addr [Biconst0baloadbipush 192ificmpeq ...
JIT (32 bit):mov EBX, dword ptr -4[ECX] // load temp1mov EAX, dword ptr 4[EBX] // load targetmov EDX, dword ptr 8[EBX] // load offsetlea EBX, dword ptr [EAX + EDX]movsx EDI, byte ptr 0[EBX] // array[0]cmp EDI, 192
© 2013 IBM Corporation
What about native access?
How do we implement this normally?
34
Java Native
…
anObject
portaddr
portaddr
© 2013 IBM Corporation
JNI implementation
• Usual “stash pointers in long types” tricks
• JNI costs tend to be high
35
© 2013 IBM Corporation
JNI implementation
• Usual “stash pointers in long types” tricks
• JNI costs tend to be high
36
© 2013 IBM Corporation
Unsafe implementation
• You shouldn’t be here
• Keeping your indices straight is never fun
37
© 2013 IBM Corporation
DirectByteBuffer implementation
• No extra JNI to write (this is good)
• Still playing the indices game
38
© 2013 IBM Corporation
PackedObject answer
• Looks like natural Java code
• Foregoes JNI
• Same type capable of on-heap representation
39
portaddr
portaddr
offsettarget
aPackedHeader
Ø
© 2013 IBM Corporation
POSSIBILITIESWith a little bit of imagination…
40
© 2013 IBM Corporation
Let’s look at transferring data
41
Heap
A
B
C
© 2013 IBM Corporation
Let’s look at transferring data
42
RemoteTransfer
Heap
A
B
C
© 2013 IBM Corporation
Let’s look at transferring data
43
Heap
A
B
C
RemoteTransfer
© 2013 IBM Corporation
Let’s look at transferring data
44
Heap
A
B
C
RemoteTransfer
© 2013 IBM Corporation
PackedObjects could help…
45
Heap
A
B
C
RemoteTransfer
© 2013 IBM Corporation
PackedObjects could help…
46
B
C
A
Packed
Heap
A
B
C
RemoteTransfer
© 2013 IBM Corporation
Heap
A
B
C
PackedObjects could help…
47
RemoteTransfer
B
C
A
Packed
B
C
A
Packed
© 2013 IBM Corporation
Making the data transfer easier…
48
RemoteTransfer
Heap
© 2013 IBM Corporation
Making the data transfer easier…
49
Specialized Heap Area
RemoteTransfer
Heap
© 2013 IBM Corporation
Making the data transfer easier…
50
B
AB
C
A
C
RemoteTransfer
Heap
Specialized Heap Area
© 2013 IBM Corporation
Making the data transfer easier…
51
RemoteTransfer
Heap
Specialized Heap Area
B
AB
C
A
C
© 2013 IBM Corporation
Making the data transfer seamless
52
RemoteTransfer
Heap
Specialized Heap Area
© 2013 IBM Corporation53
B
C
A
B
C
A
B
C
A
RemoteTransfer
Heap
Specialized Heap Area
Making the data transfer seamless
© 2013 IBM Corporation54
Packed
offsettarget
Packed
offsettarget
RemoteTransfer
Heap
B
C
A
B
C
A
B
C
A
Specialized Heap Area
Making the data transfer seamless
© 2013 IBM Corporation
IN PRACTICECase study
55
© 2013 IBM Corporation
Scenario and Measurement
• PackedObjects and jVerbs• Benefits of combining two technologies
• Scenario• Remote Procedure Call
• Low latency RPC• Zero-copy / zero-serialization
• PackedCache• PackedObjects in a remote cache
• Work done by IBM Zurich Research Lab (Stuedi, Metzler, Trivedi)
56
© 2013 IBM Corporation
Motivation: PackedObject transfer using jVerbs
• PackedObject used in off heap scenario
• Transfer native data directly using jVerbs
• Zero-copy / Zero-marshalling or serialization (both ends)
57
© 2013 IBM Corporation
Background on traditional RPC
58
© 2013 IBM Corporation
RPC using PackedObjects and jVerbs
59
© 2013 IBM Corporation
RPC Performance: Single Object
• Simple PackedObject containing int, float and long
• Signature: PackedObject foo(PackedObject obj)
60
RP
C L
aten
cy
© 2013 IBM Corporation
RPC Performance: Packed Array
• Simple PackedObject containing int, float and long
• Signature PackedObject[] foo(PackedObject[] array)
61
RP
C L
aten
cy
Packed Array size
© 2013 IBM Corporation
On Demand Object Access
• Avoid copying / serialization / CPU utilization
• RDMA direct data placement
62
© 2013 IBM Corporation
PackedObject Cache Performance
• Robust with regard to the data size
63
RP
C L
aten
cy
Packed Array size
© 2013 IBM Corporation
CHALLENGESHow is this different in the context of Java?
64
© 2013 IBM Corporation
Identity Crisis
What does aPackedLine == anotherPackedLine mean?
The data is what really matters
int yint x
offsettarget
aPackedLine
Java Native
Ø
int yint x
offsettarget
anotherPackedLine
Ø
© 2013 IBM Corporation
Synchronization
int yint x
offsettarget
aPackedLine
Java Native
Ø
int yint x
offsettarget
anotherPackedLine
Ø
© 2013 IBM Corporation
Synchronization
int yint x
offsettarget
aPackedLine
Java Native
Ø
int yint x
offsettarget
anotherPackedLine
Ø
Thread T1:synchronized (aPackedLine) {
aPackedLine.e.x = 10;
}
Thread T2:synchronized (anotherPackedLine) {
anotherPackedLine.e.x = 13;
}
aPackedLine.e.x = 10
anotherPackedLine.e.x = 13
© 2013 IBM Corporation
Finalization
68
int yint x
int yint x
offsettarget
aPackedPoint aPackedLine
offsettarget
© 2013 IBM Corporation
Finalization
69
int yint x
int yint x
offsettarget
aPackedPoint aPackedLine
offsettarget
© 2013 IBM Corporation
Finalization
70
int yint x
int yint x
offsettarget
aPackedPoint aPackedLine
offsettarget
Finalization ?
© 2013 IBM Corporation
Finalization
71
int yint x
int yint x
offsettarget
aPackedPoint aPackedLine
offsettarget
Finalization ?
Finalization ?
© 2013 IBM Corporation
Finalization
int yint x
offsettarget
aPackedLine
Java Native
Ø
int yint x
offsettarget
anotherPackedLine
Ø
Finalization ?
© 2013 IBM Corporation
Nested Data Structures
73
aPackedLine.s = aPackedPoint;
int yint x
offsettarget
aPackedLine
int yint x int y
int xoffsettarget
aPackedPoint
PackedPoint s
PackedPoint e
© 2013 IBM Corporation
Nested Data Structures
• Base types do share the same assignment operator
• Helps convey aPackedLine == anotherPackedLine as meaningless
74
aPackedLine.s := aPackedPoint;
int yint x
offsettarget
aPackedLine
int yint x int y
int xoffsettarget
aPackedPoint
PackedPoint s
PackedPoint e
© 2013 IBM Corporation
Field Initialization
No no-argument
constructor
Implicitly instantiates
PackedPoint objects
for s & e fields
@Packed
final class PackedPoint extends PackedObject {
int x;
int y;
PackedPoint(int x, int y) { … }
}
@Packed
final class PackedLine extends PackedObject {
PackedPoint s;
PackedPoint e;
PackedLine(int sx, int sy, int ex, int ey) { … }
}
© 2013 IBM Corporation
Field Initialization@Packed
final class PackedPoint extends PackedObject {
int x;
int y;
void init(int x, int y) {
this.x = x;
this.y = y;
}
PackedPoint(int x, int y) { … }
}
@Packed
final class PackedLine extends PackedObject {
PackedPoint s;
PackedPoint e;
PackedLine(int sx, int sy, int ex, int ey) {
s.init(sx, sy);
e.init(ex, ey);
}
}
© 2013 IBM Corporation
ADVANCEDTaking it to the next level…
77
© 2013 IBM Corporation
Modeling Native Data Pointers
78
*right*left
aBinaryTreeNode*right*left
*right*left
…
…
…
…
<data>
<data>
<data>
Java
offsettarget Ø
Native
struct BinaryTreeNode {
struct BinaryTreeNode* left;
struct BinaryTreeNode* right;
// data
}
© 2013 IBM Corporation
Modeling Native Data Pointers
79
*right*left
aBinaryTreeNode*right*left
*right*left
…
…
…
…
<data>
<data>
<data>
Java
offsettarget Ø
Native
aBinaryTreeNode.right
? struct BinaryTreeNode {
struct BinaryTreeNode* left;
struct BinaryTreeNode* right;
// data
}
© 2013 IBM Corporation
Modeling Native Data Pointers
80
*right*left
aBinaryTreeNode*right*left
*right*left
…
…
…
…
<data>
<data>
<data>
Java
offsettarget Ø
Native
aBinaryTreeNode.right
32b or 64b?
? struct BinaryTreeNode {
struct BinaryTreeNode* left;
struct BinaryTreeNode* right;
// data
}
© 2013 IBM Corporation
Modeling Native Data Pointers
• Annotation to mark a field as a native pointer (rather than a Java one)
• Enhance getfield / putfield to recognize
• Restrict for security reasons
• Unmanaged pointers (no GC involvement)
81
@Packed
class BinaryTreeNode extends PackedObject {
@NativePointer BinaryTreeNode left;
@NativePointer BinaryTreeNode right;
// data
}
© 2013 IBM Corporation
Modeling Native Data Pointers
82
Java Native
offsettarget Ø
aBinaryTreeNode.right
aBinaryTreeNode
*right*left
*right*left
…
<data>
<data>
…
…
© 2013 IBM Corporation
Modeling Native Data Pointers
83
Java Native
offsettarget
offsettarget Ø
Ø
aBinaryTreeNode.right
aBinaryTreeNode
*right*left
*right*left
…
<data>
<data>
…
…
© 2013 IBM Corporation
Alignment
84
@Packedfinal class Address extends PackedObject {
PackedByte[[4]] addr;short port;
}
@Packedfinal class PacketHeader extends PackedObject {
Address src;Address dest;
}
portaddr
portaddr
offsettarget
aPackedHeader
© 2013 IBM Corporation
Alignment
• Which is the correct default behavior?
• How do you get the alternate if that’s what you want?
85
portaddr
portaddr
addr
@Packedfinal class Address extends PackedObject {
PackedByte[[4]] addr;short port;
}
@Packedfinal class PacketHeader extends PackedObject {
Address src;Address dest;
}
portaddr
portaddr
offsettarget
aPackedHeader
offsettarget
aPackedHeader
© 2013 IBM Corporation
Alignment
86
class A {
int i;
short s;
short padding; // align
long l;
}
class A {
int i;
short s;
@Align long l;
}
© 2013 IBM Corporation
Alignment
• Padding isn’t quite right in the context of nested structures…
87
class A {
int i;
short s;
short padding; // align
long l;
}
class A {
int i;
short s;
@Align long l;
}
@Packedfinal class Address extends PackedObject {
PackedByte[[4]] addr;short port;
}
@Packedfinal class PacketHeader extends PackedObject {
@Align Address src;@Align Address dest;
}
© 2013 IBM Corporation
Endian
• Provide a field annotation @BigEndian (and @LittleEndian)
88
@Packedfinal class SimpleValue extends PackedObject {
int value;}
Java Native
offsettarget Ø
aSimpleValue
DE 37 C4 FE
© 2013 IBM Corporation
PARTING REMARKSAnd finally…
89
© 2013 IBM Corporation
What next?
• Not an IBM Special!• Working with Oracle on a JSR (Java9+)
• Existing Betas now available• Visit ibm.com/java for downloads• Full disclosure: Helper methods / classes in place of proposed syntax
• Your feedback is not only welcome, it’s required!
90
© 2013 IBM Corporation91
© 2013 IBM Corporation
Copyright and Trademarks
© IBM Corporation 2013. All Rights Reserved.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., and registered in many jurisdictions worldwide.
Other product and service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web – see the IBM “Copyright and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml
92