Java Profiling

ZeroProductionIncidents.Wordpress.Com

Java Profiling

1

ProductionIncidents

• Introduction– What is Profiling– Different Approaches in Java

• Source Code Instrumentation• JPDA

– JVMTI– JDWP– JDI

• Byte Code Instrumentation– Instrumentation Mechanisms– BCI Transformations– Static Byte Code Instrumentation– Dynamic Byte Code Instrumentation– java.lang.instrument

• Java Management Extensions (JMX)• Obstacles to Profiling

Topics

ProductionIncidents

Introduction

• It is a form of Dynamic Program analysis.• Monitor or Capture the information as the program executes.• Profiling can be used to capture as well as measure the key metrics to understand the behavior of

the program.• The analysis can be used to optimize the behavior of the program.

• Once you've captured performance data, you have a wide variety of options to make the data available. You can– Display to the standard output– Broadcast the information– Write to a log file– Save into the database

What is Profiling

ProductionIncidents

Different Approaches in Java

Basic Different approaches to Capture the state of JVM• Modification of JVM• Instrument source files• Java Platform Debug Architecture (JPDA)• Java Management

– java. lang.instrument– java.Management.Extentions

ProductionIncidents

INSTRUMENTSOURCE FILES

ProductionIncidents

Instrument Source Files

Manually add logging code for important methods. Execution time is printed to the log.

• Advantages– Instrumentation code is easy to create and understand– Easy to debug

• Disadvantages– Its totally a manual process of writing the code, recompiling & redeploying after

determining what needs to be measured.– Source is complex to parse (inner classes, generics)– Source format changes often– Complex to use

ProductionIncidents

Instrument Source Files

class T {

public void foo() { } public static void main(String[] args) {

T t = new T(); for (int i=1; i<=10; i++) { t.foo(); }

} }

You can take the simple java program as given below:

Instrumented version that will track the number of method calls and print the data out upon completion.

class T {

public static int count_foo = 0; public static int count_main = 0; public void foo() { count_foo++; } public static void main(String[] args) {

user_main(args); System.out.println("foo: "+count_foo); System.out.println("main: "+count_main);

} public static void user_main(String[] args) {

count_main++; T t = new T(); for (int i=1; i<=10; i++) { t.foo(); }

} }

can be Instrumented

ProductionIncidents

JAVA PLATFORM DEBUG ARCHITECTURE

ProductionIncidents

JPDA - IntroductionThe Java Platform Debugger Architecture (JPDA) provides the infrastructure needed to build end-user debugger applications for the Java Platform.

JPDA is not an application nor a debugging tool but a set of well-designed and implemented interfaces and protocols.

The communication is connection oriented - one side acts as a server, listening for a connection. Another side acts as a client and connects to the server.

ClassClass

JVM

JVMTI

Back End

Debuggee

Front End

Debugger UI

Debugger

JDI

JDWP

JVMTI

JVM

Debugger/TracerJDI

JDWPConnector

Architecture is based on three interfaces:• JVMTI• JDWP• JDI

Architecture can be viewed as three Components:• Debugee• Communication Channel• Debugger

ProductionIncidents

Three layers in JPDA - Components

VM

Back-end

Components

Debugee

Communication Channel

Debugger

Interfaces

JVM TI -- JVM Tool Interface

JDWP – Java Debug Wire Protocol

JDI – Java Debug Interface

The debuggee is the process being debugged. It consists of • application being debugged• VM running the application • back-end of the debuggee

Debuggee

Front - end

UI

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces



JDI – Java Debug InterfaceFront - end

UI

Java Virtual Machine (VM)• Refers to the VM running the Application being debugged.

• VM implements the Java Virtual Machine Debug Interface (JVM TI).

Back-end• Back-end helps in communicating requests from the debugger front-end to the debuggee.

• It also helps in communicating the responses to the requests, including events.

• The back-end communicates with the debuggee VM using the JVM Tool Interface (JVM TI).

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




It is the link between the front and back ends of the debugger. It can be thought of as consisting of two mechanisms:

• Connectors• Transport

Connector - A connector is an object that is the means by which a connection is established between the front and back-ends. Transport. A transport is the underlying mechanism used to move bits between the front-end and the back-end. The format and semantics of the serialized bit-stream flowing over the channel is specified by the JDWP.


Front - end

UI

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




A connector is a JDI abstraction that is used in establishing a connection between a debugger application (written to the JDI) and a target VM. Different JDI implementations are free to provide different connector implementations to match the transports and VMs they support.

There can be three types of connectors:• listening: The front-end listens for an incoming connection from the back-end.• attaching: The front-end attaches to an already running back-end.• launching: The front-end actually launches the Java process that will run the debuggee code and the back-end.

The JDI reference implementations provides several connectors which map to the available transport types and the modes of connection (launching, listening, and attaching).


Front - end

UI

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




Transport:Sun ships two transport implementations with the reference implementation: Socket & Shared Memory.

Socket• With the socket transport, the debugger application and the target VM can reside either on the same machine

or on different machines. • The socket transport is identified through a unique string, dt_socket. This name can be used to select the socket

transport when invoking the target VM. • socket transport addresses have the format "<name>:<port>" where <name> is the host name and <port> is the

socket port number at which it attaches or listens. Shared Memory• The shared memory transport uses a shared memory region to exchange JDWP packets between the debugger

application and the target VM. • With the shared memory transport, the debugger application and target VM must reside on the same machine. • The shared memory transport is identified through a unique string, dt_shmem.


Front - end

UI

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




Front-endThe debugger front-end implements the high-level Java Debug Interface (JDI). The front-end uses the information from the low-level Java Debug Wire Protocol (JDWP).

User Interface (UI)There is no specific standard mentioned for user interface. It is upto the tools that provide the implementation.

Front - end

UI

ProductionIncidents

Three layers in JPDA - Interfaces

VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

A native interface implemented by the VM. It defines interfaces VM must provide for debugging. It includes:• Requests for Information (ex, current heap usage)• Actions (ex, set break point)• Notifications (ex, when break point has been hit)

JVM TI is a two-way interface. • A client of JVMTI can be notified of interesting occurrences through events. • JVM TI can query and control the application through many functions

JVM TI was introduced in J2SE 5.0 and replaced JVMDI and JVM Profiling Interface (JVMPI). JVMDI was removed in Java SE 6 and JVMPI will be removed in Java SE 7.

JVMTI --- Java Virtual Machine Tool Interface

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

Agents can be written in any native language that supports C language calling conventions and C or C++ definitions. The function, event, data type, and constant definitions needed for using JVM TI are defined in the include file jvmti.h.

Agents run in the same process with and communicate directly with the VM executing the application being examined. This communication is through a native interface (JVM TI). The native in-process interface allows maximal control with minimal intrusion on the part of a tool.

The JVMTI specification supports the use of multiple simultaneous JVMTI agents. Each agent has its own JVMTI environment i.e., JVM TI state is separate for each agent - changes to one environment do not affect the others.

JVMTI --- Agents

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

Agent startupThe VM starts each agent by invoking a start-up function. If the agent is started in the OnLoad phase the function Agent_OnLoad will be invoked. If the agent is started in the live phase the function Agent_OnAttach will be invoked. Exactly one call to a start-up function is made per agent.

Agent shutdownThis function will be called by the VM when the library is about to be unloaded.

JNIEXPORT void JNICALL Agent_OnUnload(JavaVM *vm)Loading Agent-agentlib:<agent-lib-name>=<options> agent-lib-name is the name of the library to load. Filename and the location is dependent on the OS platform. For ex, -agentlib:myagent, VM will load myagent.dll file from PATH under windows. myagent.so from LD_LIBRARY_PATH for solaris. Options can also be passed to the agent.-agentpath:<path-to-agent>=<options> path-to-agent will have the absolute path to load the library. For ex, -agentpath:d:\myagent\MyAgent.dll will load the MyAgent .dll library.

JVMTI --- Agents

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

Sample Functions:Get Thread State Get the state of a thread.

jvmtiError GetThreadState(jvmtiEnv* env, jthread thread, jint* thread_state_ptr)

Get All Threads Get all live threadsjvmtiError GetAllThreads(jvmtiEnv* env, jint* threads_count_ptr, jthread** threads_ptr)

Iterate Over HeapjvmtiError IterateOverHeap(jvmtiEnv* env, jvmtiHeapObjectFilter object_filter, jvmtiHeapObjectCallback

heap_object_callback, const void* user_data)

JVMTI --- Agents

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

Defines the format of information and requests transferred between the debuggee process and the debugger front-end.

It does not define the transport mechanism (socket, serial line, shared memory, ...).

The specification of the protocol allows the debuggee and debugger front-end to run under separate VM implementations and/or on separate platforms.


ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

JDWP --- Configure InterfaceFor JDK5 & above

-agentlib:jdwp=<sub-options>For Prior version of JDK5

-Xdebug and -Xrunjdwp:<sub-options>

<sub-options>:• transport - is a method of communication between a debugger and the virtual machine that is being

debugged.• Address - when establishing a connection, transport addresses is used to identify the end-point of the

connection.• server - if the server property is 'y', the application will listen for a debugger to attach; otherwise, it

will attach to the debugger at the specified address.• suspend - if suspend is 'n', the application will start immediately and will not wait for a debugger to

attach to it. If 'y', the application will be suspended until you attach to it.

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

JDWP --- Configure InterfaceExamples:

For JSE 5 or above, we can use the following to attach to remote debudder.-agentlib:jdwp=transport=dt_socket,address=localhost:7007,server=y,suspend=y

For JDK 1.4 or below, we use the following options:-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7007

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

JDWP --- Configure InterfaceExamples:-agentlib:jdwp=transport=dt_socket,server=y,address=7007 Listen for a socket connection on port 7007. Suspend this VM before main class loads. Once the debugger application connects, it can send a JDWP command to resume the VM.

-agentlib:jdwp=transport=dt_socket,server=y,address=mymachine:7007,timeout=5000 Listen for a socket connection on port 7007 on mymachine. Terminate if the debugger does not attach within 5 seconds. Suspend this VM before main class loads. Once the debugger application connects, it can send a JDWP command to resume the VM.

-agentlib:jdwp=transport=dt_shmem,server=y,suspend=n Choose an available shared memory transport address and print it to stdout. Listen for a shared memory connection at that address. Allow the VM to begin executing before the debugger application attaches.

-agentlib:jdwp=transport=dt_shmem,address=mysharedmemory Attach to a running debugger application via shared memory at transport address "mysharedmemory". Suspend this VM before the main class loads.

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

A 100% Java interface implemented by the front-end.

Defines information and requests at a user code level.

While debugger implementers could directly use the Java Debug Wire Protocol (JDWP) or Java Virtual Machine Debug Interface (JVM TI), this interface greatly facilitates the integration of debugging capabilities into development environments.

Since it is a Java interface implemented by the front-end, the command-line parameters depends on the Application.


ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

In order to use the JPDA API, you need to add the JDK tools.jar to the classpath. It can be found in the JDK lib directory.com.sun.jdi package holds most of the classes required for writing our own JDI.

High level steps of writing our own JDI includes:• Acquiring the VirtualMachine object instance.• Getting the Connector object from VirtualMachine object.• We can use VirtualMachine’s EventRequestManager to instruct it to notify us of events. The VirtualMachine

EventQueue is then used to process the generated events.

Based on our requirements, we can use the relevant functions and get the necessary output.


ProductionIncidents

Three layers in JPDA

VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

• The Java Virtual Machine Tools Interface (JVMTI) defines the services a VM must provide for debugging.

• The Java Debug Wire Protocol (JDWP)defines the format of information and requests transferred between the process being debugged and the debugger front end, which implements the Java Debug Interface (JDI).

• The Java Debug Interface (JDI) defines information and requests at the user code level

Three Interfaces - Summary

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

• Theoretically JPDA could have only one interface, the Java Debug Wire Protocol (JDWP). This would allow cross-platform remote debugging.

• Writing directly to JDWP however is painstaking, information sent across the wire must be read and written precisely.

• The Java Debug Interface (JDI) is provided to make interfacing to JDWP easier. • Not only does JDI format data to be sent across the wire and parse incoming data, but it provides queuing, caching,

connection initiation and many other services. And all this functionality is available from an easy to use Java programming language interface.

• In an analogous way, JVM TI insulates Java virtual machine implementors from the intricacies of the debuggee side of JDWP.

• A debugger developer may hook into JPDA at any layer

Why Three Layers

ProductionIncidents


VM

Back-end

Components

Debugee


Debugger

Interfaces




UI

If you are writing a debugger (or debugger like tool), the easy answer is, use JDI.

You might want to use JDWP if your front-end tool is not written in the Java programming language.

You might want to use JVM TI if your tool has very specialized functionality, not available in JDWP/JDI, that can only be performed in the debuggee process and you have a lot of time on your hands.

If you are writing a Java virtual machine the easy answer is, implement JVM TI.

Three Interaces – Which Interface layer to use

29 ProductionIncidents

JPDA_OPTS="-agentlib:jdwp=transport=$JPDA_TRANSPORT,address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND"

Sun JVM hotspot has the inbuilt JPDA implementation. We can enable the default JPDA provided by hotspot JVM by setting up the relevant startup parameters. For example, the following has been set for Tomcat server.

Out of Box support for JPDA & JDI

The Java Debugger, jdb, is a simple command-line debugger for Java classes. It is a demonstration of the JPDA that provides inspection and debugging of a local or remote Java Virtual Machine

As shown in the above image, we are attaching to the process listening on port 7007. After attaching to the JVM, we can run the commands. “threads” command, provides the current threads and their status.

ProductionIncidents

BYTE CODE INSTRUMENTATION

ProductionIncidents

ByteCode Instrumentation

• Java source is compiled to get the bytecode. JVM executes the byte code.

• Bytecode Instrumentation is the ability to alter the bytecode instructions in the target program.

• Alterations can be additive (with no modification of application state or behavior) or adding new functionality.

• Bytecode Instrumentation can be used

– For Monitoring application Behavior( trace program execution, gather profiling data, monitor memory usage etc.)

– New functionality can be added

• For monitoring the behavior, typical alterations include adding "events" to the code of a method like at the end or beginning of the method, instantiating an object, etc.

• Instrumentation can run entirely in Java programming language code or can call into the native agent.

ProductionIncidents

Byte Code Instrumentation Support in Java

Java provides Instrumentation using– JVMTI– Java.lang.instrument

• The JVMTI offers a mechanism to instrument classes as they are being loaded. Unfortunately, the JVMTI requires instrumentation-processes to be implemented in native code, contradicting the Java motto ‘write once, run anywhere’.

• JDK 1.5 has introduced a mechanism, Java language instrumentation agents (package java.lang.instrument), to instrument classes as they are being loaded.

ProductionIncidents

Byte Code Instrumentation Mechanisms

• Static translation– Class files on the disk are translated– Doesn’t work with class file that are not available– Somewhat more difficult to use

• Class loader– New class loader delegates to the original class loader to get the original class file– Target program class loaders require special support

• Pre-agent– Instrumentation code is passed each class as it is loaded– New classes cannot be created

ProductionIncidents

Byte Code Instrumentation TransformationsTransformation Target

Whole ProgramSelective

ClassMethod

Transformation TimeAhead of time – or StaticAt run-time

Load TimeArbitrary time

Requires Class Loader interaction (pre 5.0)RedefineClasses() in 5.0

Transformation TypesReplacementInsertionDeletion

ProductionIncidents

Static Byte Code Instrumentation

Static bytecode instrumentation inserts all instrumentation code before the program under instrumentation starts execution.

Advantages of this approach:• It causes less runtime overhead, as all classes are instrumented before the program is executed.• It can leverage any high-level bytecode engineering library without any risk of perturbing

measurements;• As static instrumentation is completed before the program under instrumentation starts execution,

the overhead caused by the instrumentation process is not an important issue.

The major drawback of static instrumentation is that dynamically generated or loaded code is not instrumented. Scope for missing the instrumentation of classes as there is no crosscheck on the instrumented classes.

ProductionIncidents

Dynamic Byte Code Instrumentation

Dynamic Bytecode instrumentation is a technique that allows to modify the bytecode of Java classes when they are loaded by the Java virtual machine.

• Dynamic instrumentation provides the ability to alter the Java virtual machine bytecode instructions, which compose the target program, when classes are loaded at runtime.

• By exploiting a hook provided by the JVM since version 1.5, it is possible to register a “Java Agent” that inspects each bytecode instruction and introduces method calls to report interesting events.

• This approach allows a very high transparency level to end users (it is triggered by a single option in the JVM invocation command), has a very low latency between the event and its recording, but introduces some overhead.

• Core Java classes cannot be instrumented at runtime, although it is possible to instrument them before running the JVM.

ProductionIncidents

Static vs Dynamic Byte Code Instrumentation

Static Byte Code Instrumentation

Dynamic Byte Code Instrumentation

Instrumentation Time Before the Program Start At run time

Overhead Less overhead during Startup

Introduces extra overhead during program startup and may be perturbing measurements

Accuracy of Classes Instrumentation

Chances of missing the instrumentation of classes

Ensures that all classes will be instrumented

ProductionIncidents

Java.lang.instrument -Introduction

The java.lang.management package provides the interface for monitoring and managing the JVM. It provides access to information such as:

– Classes loaded– Threads running– Memory consumption– Garbage Collection statistics– etc

In order to use the API, a special agent class must be implemented and registered with the JVM at runtime.

The -javaagent: is available from JDK 5. Agent is enabled by adding the following options as part of startup parameters.

-javaagent:<jarpath>[=<options>]

Enabling the Agent:

ProductionIncidents

1. Implement the java.lang.instrument.ClassFileTransformer interface: public interface ClassFileTransformer { public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException; }

2. Create a "premain" method. This method is called before the application's main() method and looks like this:

public class Main { public static void premain(String args, Instrumentation inst) { ... } }

3. In the agent JAR file, include a manifest entry identifying the class containing the premain() method: Manifest-Version: 1.0 Premain-Class: sample.verboseclass.Main

Java.lang.instrument – Creating an Agent

ProductionIncidents

Several frameworks are available for performing bytecode manipulation. Some of them include:

– Apache’s BCEL– ObjectWeb’s ASM– Javaassist

Byte Code Manipulation Framework

ProductionIncidents

Java Management Extensions

• JMX also defines the API for management of Java applications.• We can enable the JMX agent for:

– Local monitoring, for a client management application running on the local system. – Remote monitoring, for a client management application running on a remote system.

http://zeroproductionincidents.wordpress.com/2013/07/20/serviceability-in-hotspot/

http://zeroproductionincidents.wordpress.com/2013/08/05/enabling-jmx-for-java-application/

http://zeroproductionincidents.wordpress.com/2013/08/15/dynamic-attach/

Additional Information can be obtained from:

http://zeroproductionincidents.wordpress.com/2013/07/20/serviceability-in-hotspot/

http://zeroproductionincidents.wordpress.com/2013/08/05/enabling-jmx-for-java-application/

http://zeroproductionincidents.wordpress.com/2013/08/15/dynamic-attach/

ProductionIncidents

Obstacles to Profiling

The two biggest obstacles to profiling have been – runtime overhead and – interpretation of the results.

ProductionIncidents

Royalty Free• NetBeans Profiler• Eclipse Test & Performance Tools Platform (TPTP)

Commercial Products• CA Wiley Introscope• Borland Optimizeit Enterprise Suite 6.0• YourKit Java Profiler 4.0.11• JProbe Suite 5.2.3• AppPerfect DevSuite 5.5• EJ-Technologies JProfiler 4.0.1• J-Sprint

Profiling Tools

ProductionIncidents

THANKYOU

Technology

Java Profiling