Web sphere application server performance tuning workshop

WebSphere Application Server Performance Tuning Toolkit

Workshop with Daytrader

Statement: draft

Author : Sui Peng Fei ([email protected])

Version: 1.0

Date: Jan, 16th, 2012

Table of Contents

Getting Start..................................................................................................................................3

Performance Tuning Toolkit Overview.......................................................................................................3

Prepare for the workshop..........................................................................................................................4

Set up WebSphere Application Server Topology............................................................................4

Download the WebSphere Application Server Performance Tuning Toolkit(PTT)..........................4

Download the Apache JMeter.......................................................................................................5

Install the Daytrader Application...............................................................................................................5

Have a brief stress testing using the JMeter Tool.......................................................................................8

Scenario1: “Synchronization Blocking” Problem..........................................................................11

Determine the “Synchronization Blocking” problem...............................................................................14

Generate a thread dump.............................................................................................................14

Launch the ISA 5.0 Tech Preview..................................................................................................15

Use ISA V5 to analyze the “Synchronization Blocking” problem..................................................15

Scenario2: “Dead Lock” problem.................................................................................................19

Monitoring the “Dead Lock” problem.....................................................................................................20

Determine the “Dead Lock” problem......................................................................................................21

Generate a thread dump.............................................................................................................21

Use ISA V5 to analyze the “Dead Lock” problem..........................................................................22

Scenario3: “High CPU Usage” problem.......................................................................................25

Trigger the “High CPU Usage” problem...................................................................................................26

Monitoring the “High CPU Usage” problem............................................................................................26

Determine the “High CPU Usage” problem.............................................................................................28

Scenario4: “Connection Leak” problem.......................................................................................30

Application Changes to Simulate “Connection Leak” Situation................................................................30

Trigger the “Connection Leak” problem..................................................................................................31

Monitoring the “Connection Leak” problem...........................................................................................31

Determine the “Connection Leak” problem............................................................................................33

Scenario5: “Memory Leak” problem...........................................................................................36

Application Changes to Simulate “Memory Leak” Situation....................................................................36

Trigger the “Memory Leak” problem.......................................................................................................36

Monitoring the “Memory Leak” problem................................................................................................36

Determine the “Memory Leak” problem.................................................................................................37

Other optional functions you can go through..............................................................................42

Tuning the performance related parameters...........................................................................................42

Adjust rules from rule editor if needed...................................................................................................43

Generate report.......................................................................................................................................44

Getting Start

Performance Tuning Toolkit Overview

The WebSphere® Application Server Performance Tuning Toolkit (PTT) is an Eclipse-based intelligent tool designed to help users tune the performance of WebSphere Application Server using data analysis, and statistical inference technology. It is designed to help users locate bottlenecks and tune their applications appropriately.

Compared with other performance tuning tools, the main character of PTT is light weight and easy to use. You need not do any install or configuration work except providing the IP and soap port of your Dmgr.

Below is the function list of the toolkit:Functions Components

Performance Monitoring

Dashboard (key status of all servers) Monitor page (chart of components of one server) Data page (detailed page of components)

Performance Tuning “Tuning Parameters” view (tune common server parameters)

Scripts view (tune other parameters)Health Check Rule engine (process data)

Rule editor (edit rules)Operations Dump agent (generate thread dump and heap

dump) Trace agent (enable trace) Connection agent (extract the connection pool

contents)Generate Report Report engine (generate, export and print report)

In order to help users learn how to use this toolkit, it provides a modified Daytrader benchmark and a set of scripts to illustrate existing common customer issues and solutions, thus greatly reducing the user’s learning curve. Through execution of the workshop, you will have a good understanding of the impact of typical bad code, and know more about symptoms of certain problems thereby speeding the process of problem resolution.

The built-in Daytrader application

DayTrader is a open source benchmark built on a core set of Java EE technologies that includes Java Servlets and JavaServer Pages (JSPs) for the presentation layer and Java database connectivity (JDBC), Java Message Service (JMS), Enterprise JavaBeans (EJBs) and Message-Driven Beans (MDBs) for the back-end business logic and persistence layer. The following diagram provides a high-level overview of the application architecture.

Figure 1: Application architecture

The built-in Daytrader application will simulate improper application design or some error coding scenario which will involve "Performance" issues.

The simulated "sick" situations include:1. Causal factors lead to synchronization blocking or lock competition, and the threads have to spend more time on waiting for the lock.2. Causal factors lead to a deadlock and block other threads which need to gain the lock owned by deadlock threads. 3. Causal factors lead to heavy JDBC connection consuming, which will cause lots of connection waiting threads.4. High CPU usage cause by some CPU consuming threads.5. Out of memory problems caused by memory leak.

Prepare for the workshop

Set up WebSphere Application Server Topology

Install the WebSphere Application Server 7.0 or later and create a cluster with at least 2 members (Download the trial version from DeveloperWorks if needed). Create a proxy server as the dispatcher.

Download the WebSphere Application Server Performance Tuning

http://www.ibm.com/developerworks/cn/downloads/ws/was/index.html

Toolkit(PTT)

Download link (Internal):

http://cattail.boulder.ibm.com/cattail/#[email protected]/files/2F87C2500E943DDC86A2F0EC093F23B6

Download the Apache JMeter

Download link: http://jmeter.apache.org/download_jmeter.cgi

Download the ISA 5.0 Tech Preview from IBM website.

Download link: http://www-01.ibm.com/support/docview.wss?uid=swg27023689

Install the Daytrader Application

Unzip the PTT zip file and double click the “PerfTuningToolkit.exe” to launch the PTT workbench

Figure 2: PTT workbench

Click the “Add host” button in the toolbar of “Host” view to add a new host. Just input the IP address and soap port of Dmgr. You need also provide the user name and password if the global security of WAS is enabled.

http://www-01.ibm.com/support/docview.wss?uid=swg27023689

http://jmeter.apache.org/download_jmeter.cgi



Figure 3: Add the host info

Click the “Help” -> “Case Study” from the workbench menu bar to import the sample application and script files to the “scripts” view.

Open the settings.py in the “Sample” folder and change the cluster name to the cluster you created in previous steps. You also need to set the security settings if the WAS security is enable.

Figure 4: Import the assets and modify settings

Select the host of your Dmgr in the host view, and right click the installDaytrader.py in the sample folder of the “scripts” view. Select “run script” menu in the context

menu to run the installDaytrader.py script against your cell. The script will install the modified Daytrader Application to the cluster you specified.

Figure 5: Install the Daytrader Application

Figure 6: Check the results of installation

Important: (Re)start the cluster and proxy server, and manually check the installed application from url http://<proxy server host name>:<port>/daytrader/scenario

Figure 7: Verify the installation of Daytrader application

Note: the Daytrader application will use an embedded Derby data base, it is not a valid design for a normal clustered application, but is very suitable for a demo purpose.

You can also use DB2 as a central db for performance related testing, but you need to specified the db2 settings in the settings.py and also need to created table and populate data according to the instruction on the “configuration” tab of Daytrader applications.

Have a brief stress testing using the JMeter Tool

Launch the JMeter tool and open the daytrader.jmx file in the <PTThome>/workspace/Scripts/sample folder.

Change the hostname and port number (default is 80) to the host name and port number of your proxy server.

Figure 8: Open the daytrader.jmx and modify the host name and port

Figure 9: Start the stress

Double click the host in the “Hosts” view of PTT workbench to connect to the cell and start monitoring. It is recommended monitoring at least 10 minutes to make sure no error will occur during normal stress testing.

Figure 10: Monitor your cell

Note: The dashboard will monitor all the key status of servers (include Application Servers and Proxy Servers) in the latest monitoring interval (monitoring interval is multiple of the data collection interval which was set in the preference panel)

The color of performance data will turn red if some abnormal events were detected in current monitoring interval, it only represent the current status. There will also be a warning mark in the second column as long as any error ever occurs and it will leave unless you clear it. So the warning marks can represent history abnormal events.

Figure 10: Errors in the dashboard

Scenario1: “Synchronization Blocking” Problem

Application Changes to Simulate “Synchronization Blocking”

Situation

In this scenario, the application will continuously call static synchronized method, in which the thread will sleep several seconds.

Sample code changes in the TradeAppServlet

//For Synchronization Blockingif(TradeConfig.isThreadHang()){

syncMethod();

}

public static synchronized void syncMethod(){

try {

Thread.sleep(TradeConfig.getHangTime());

} catch (InterruptedException e) {

e.printStackTrace();

}

}

Trigger the “Synchronization Blocking” problem

Important: make sure that the stress is running and the PTT is monitoring before you trigger the “Synchronization Blocking” problem

Click the “Server” tab of the home page (http://hostname :port/daytrader ). The error reproduction options will be shown (see figure11). In order to trigger the synchronization blocking problems, select the “Synchronization Blocking” checkbox in the “Server:” tab and update the runtime configuration. If you find that the hang time is not long enough to trigger the thread hung problem, you can increase this value accordingly.

.

Figure11. Error reproduction options

Monitor the “Synchronization Blocking” problem

When a synchronization blocking problem occurs, you will see the red data and warning marks in the dashboard as below.

Figure12. Dashboard monitoring

Double click the server panel to switch to the monitoring page. In the servlet response time section, you can see that the average response time keeps increasing.

Figure13. Server chart monitoring

Click the point to switch to the data page to check the servlet that is blocking.

Figure14. Check the detailed data

Besides, some servlet alerts will appear in the alert section.

Figure15. The alert panel

Click the alert point to switch to the data page, which show that the time limit of servlet average response time is exceeded.

Figure 16: Alert monitoring data

Determine the “Synchronization Blocking” problem

Generate a thread dump

In order to determine the “Synchronization Blocking” problem, we need to generate a “Thread Dump” of the sick server from the content menu of the “Topology” view.

Figure 17: Take a thread dump of the sick server

Launch the ISA 5.0 Tech Preview

Perform the following steps to launch ISA 5.0 Tech Preview.1) Unzip the zip file to your file system. 2) Start WAS CE server with ISA5/start_isa.bat3) Access ISA 5.0 with URL: http://localhost:8080/isa5/

Figure 18: GUI of ISA 5.0 Tech Preview

Use ISA V5 to analyze the “Synchronization Blocking” problemAdd a new test case for the “Synchronization Blocking” problem.

Figure 19: Add new case

http://localhost:8080/isa5/

Figure 20: Input case infoAdd the thread dump files generated to the test case. By default, they are generated in the profile root folder of the sick server.

Figure 21: Add thread dump to the caseNext, analyze the thread dump file using the “Thread and Monitor Dump Analyzer” tool of ISA v5 and view the analysis results.

Figure 22: Analyze the thread dump

Figure 23: Analysis finished

Figure 24: View the analysis resultsYou need to check the detail info of monitors and threads to find the root cause.

Figure 21: Analysis results summary

Figure 22: “Monitor Dump” analysis summary

Figure 23: “Monitor Dump” analysis detailed infoFrom the detailed page we can find the blocked threads and the status of lock owner, so we can improve our application accordingly.

Scenario2: “Dead Lock” problem

Application Changes to Simulate “Dead Lock” Situation

In this scenario, a dead lock problem will be triggered and all the other threads will be hung for ever, as they can not get the synchronization lock which is owned by the deadlock threads.

Below is the code changes in the TradeAppServlet

//For Dead Lockif(TradeConfig.isDeadLock()){

deadLockMethod();

}

public void deadLockMethod(){

try {

synchronized (lock1) { // lock1 is defined in the "Methods

and Static Variables" tab

Thread.sleep(5000);

synchronized (lock2) {

}

}



Thread.sleep(5000);


}

}

} catch (Exception e) {


}

}

Trigger the “Dead Lock” problem

Important: make sure that the stress is running and the PTT is monitoring before you trigger the “Dead Lock” problem

In the “Server” tab of Daytrader home page, select the “Deadlock” checkbox and click the “Update Configuration” button. A deadlock will be triggered. It will not only block the deadlock lock threads, but also block other threads which have to wait for the lock owned by the deadlock threads.

Figure24: Error reproduction options

Monitoring the “Dead Lock” problem

When a dead lock problem occurs, you will see the red data and warning marks in the dashboard.

Figure25: Dashboard monitoring

Double click the server panel to switch to the monitoring page, and you can see that some servlet alerts are shown in the alert section.

Figure26: Server monitoring

Click the alert point in the chart to check the alert content in the detailed data page.

Figure27: Alert detailed content

Determine the “Dead Lock” problem

Generate a thread dump

In order to determine the “Dead Lock” problem, we need to generate a “Thread Dump” of the sick server from the content menu of the “Topology” view.

Figure 28: Take a thread dump of the sick server

Use ISA V5 to analyze the “Dead Lock” problemAdd a new test case for the “Dead Lock” problem.

Figure 29: Add new case

Figure 30: Add thread dump to the case

Figure 31: Analyze the thread dump

Figure 32: View the analysis results



Figure 35: “Monitor Dump” analysis detailed infoFrom the detailed page we can find the deadlock threads and their status, so we can fix the problem accordingly. A best practice on this kind of problem is getting the locks in the same order. So here is the improved code:public void deadLockMethod(){

try {



Thread.sleep(5000);

synchronized (lock2) { // lock2 is defined in the

"Methods and Static Variables" tab

}

}


Thread.sleep(5000);


}

}



}

}

Scenario3: “High CPU Usage” problem

Application Changes to Simulate “High CPU Usage” Situation

In this scenario, a CPUConsumer thread will be started which will invoke an empty loop method. You can increase the thread number if the CPU usage is not high enough.

Sample code changes in the TradeAppServlet

//For High CPU Usage if(TradeConfig.isHighCpuUsage()){

if(cpuThreadCount<TradeConfig.getThreadNum()){

try {

cpuThreadCount++;

CPUConsumer cc = new CPUConsumer();

cc.start();



}

}

}

class CPUConsumer extends Thread{

public void run() {

try{

while(TradeConfig.isHighCpuUsage()){

}

cpuThreadCount--;

}catch(Exception e){


}

}

}

Trigger the “High CPU Usage” problem

In the “Server” tab of Daytrader home page, select the “High CPU Usage” checkbox and click the “Update Configuration” button. A loop thread will be started to consume the CPU resources. If you find that the CPU usage is not high enough to trigger the “HighCPU” health policy, you can increase the thread number accordingly


Monitoring the “High CPU Usage” problem

When a high CPU usage problem occurs, you will see the red data and warning marks in the dashboard.

Figure37: Dashboard monitoring

Double click the server panel to switch to the monitoring page, and you can see that the CPU usage is very high.

Figure38: Server CPU monitoring

Some runtime alerts are shown in the alert section.

Figure39: Alert panel

Click the alert point to check the alert content in the detailed data page

Figure39: Alert detailed content

Determine the “High CPU Usage” problem

In order to determine the High CPU Usage problem, we need to collect the CPU usage info from the command line. But the command differs from OS to OS; here we will take the Linux for example. Change the node name and server name of the sick server and run the shell file with the following content and content.

node="node1"

server="TradeDC_node1"

echo Start get cpuInof on `date` >> cpuInfo.txt

echo

===================================================================

================================= >> cpuInfo.txt

pid=`ps -ef |grep java | grep com.ibm.ws.runtime.WsServer | grep -w "$node $server" |

awk '{print $2}'`

echo "$pid" >> cpuInfo.txt

if [ -n "$pid" ]

then

top -b -n1 -H -p $pid >> cpuInfo.txt

kill -3 $pid

fi

echo Finish custom action on `date` >> cpuInfo.txt

Open the cpuInfo.txt file generated by the “collectCPUInfo” custom action. You will find the system CPU info as below:Start custom action on Mon Feb 20 23:20:01 CST 2012

==================================================================================================

32094

top - 18:20:20 up 6 days, 8:25, 4 users, load average: 1.48, 1.38, 1.31

Tasks: 180 total, 1 running, 179 sleeping, 0 stopped, 0 zombie

Cpu(s): 6.6%us, 0.5%sy, 0.0%ni, 92.4%id, 0.1%wa, 0.0%hi, 0.4%si, 0.0%st

Mem: 4152360k total, 4029544k used, 122816k free, 162536k buffers

Swap: 2064376k total, 0k used, 2064376k free, 2350792k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

623 root 20 0 915m 494m 58m R 82.8 12.2 123:00.06 java

32255 root 20 0 915m 494m 58m S 2.0 12.2 0:14.72 java

32094 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java

32096 root 20 0 915m 494m 58m S 0.0 12.2 0:18.93 java

32097 root 20 0 915m 494m 58m S 0.0 12.2 0:02.77 java

32098 root 20 0 915m 494m 58m S 0.0 12.2 0:26.89 java

32101 root 20 0 915m 494m 58m S 0.0 12.2 0:24.46 java

32102 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java

32103 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java

32104 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java

32106 root 20 0 915m 494m 58m S 0.0 12.2 0:00.06 java

32107 root 20 0 915m 494m 58m S 0.0 12.2 0:00.06 java

32109 root 20 0 915m 494m 58m S 0.0 12.2 0:00.48 java

32115 root 20 0 915m 494m 58m S 0.0 12.2 0:00.00 java

32116 root 20 0 915m 494m 58m S 0.0 12.2 0:00.38 java

32118 root 20 0 915m 494m 58m S 0.0 12.2 0:00.02 java

32121 root 20 0 915m 494m 58m S 0.0 12.2 0:02.24 java

List 1: Content of cpuInfo.txtFrom these info, we can find that the thread “623” consume 82.8% of CPU resource, which is the root cause of high CPU usage problem.As the thread dump will use “Hexadecimal” as the thread id, we need to convert the thread pid from 623 (Decimal) to 0x26F (Hexadecimal) firstly, and then search it in the thread dump file, you will find the root cause is the CPUConsumer.run method in TradeAppServlet. Below is the stack trace:3XMTHREADINFO "Thread-95" J9VMThread:0x0C0EFE00, j9thread_t:0x8D15AFA8, java/lang/Thread:0x9B2C2FC0, state:CW,

prio=5

3XMTHREADINFO1 (native thread ID:0x26F, native priority:0x5, native policy:UNKNOWN)

3XMTHREADINFO2 (native stack address range from:0x02B87000, to:0x02BC8000, size:0x41000)

3XMTHREADINFO3 Java callstack:

4XESTACKTRACE at

org/apache/geronimo/samples/daytrader/web/TradeAppServlet$CPUConsumer.run(TradeAppServlet.java:120)

3XMTHREADINFO3 Native callstack:

4XENATIVESTACK (0x00CA1752 [libj9prt24.so+0xb752])

4XENATIVESTACK (0x00CACF60 [libj9prt24.so+0x16f60])

4XENATIVESTACK (0x00CA17E5 [libj9prt24.so+0xb7e5])



4XENATIVESTACK (0x00CACF60 [libj9prt24.so+0x16f60])

4XENATIVESTACK (0x00CA15F8 [libj9prt24.so+0xb5f8])

4XENATIVESTACK (0x00C9D500 [libj9prt24.so+0x7500])

4XENATIVESTACK (0x0099740C)

List 2: Content of thread dumpFrom the content of cpuInfo.txt and the thread dumps, we can find the threads consuming lots of CPU resources and their stack trace, so we will be able to fix the problem accordingly.

Scenario4: “Connection Leak” problem

Application Changes to Simulate “Connection Leak” Situation

In this scenario, some requests will invoke a ConnectionConsumer thread in which it will get a connection from the connection pool and never close it. You can adjust the “Leak Frequency” to control the leak rate.

Code changes in the TradeAppServlet

//For Connection Consuming if(TradeConfig.isConnectionLeak()){

if(triggerByFrequency(TradeConfig.getLeakFrequency())){

ConnectionConsumer cc = new ConnectionConsumer();

cc.start();

}

}

class ConnectionConsumer extends Thread{

public void run() {

try{

javax.naming.InitialContext ctx = new

javax.naming.InitialContext();

javax.sql.DataSource ds = (javax.sql.DataSource)

ctx.lookup("jdbc/TradeDataSource");

java.sql.Connection con = ds.getConnection();

}catch(Exception e){


}

}

}

Trigger the “Connection Leak” problem

In the “Server” tab of Daytrader home page, select the “Jdbc Connection Leak” checkbox and click the “Update Configuration” button. It will leak the connection of TradeDataSource at the specified frequency, and you can also modify the “Leak Frequency” accordingly.


Monitoring the “Connection Leak” problem

When a connection leak problem occurs, you will see the red data and warning marks in the dashboard.

Figure41: Monitoring the dashboard

Double click the server panel to switch to the monitoring page, and you can see several connection pool and servlet alerts in the alert section.

Figure42: Alert Panel

Go to the detailed data page to check the alert content.

Figure43: Detailed alert content

Go to connection pool detailed page to check the c

Figure44: Detailed connection pool data

Some other problems will be triggered after a while.


Determine the “Connection Leak” problem

In order to get the jdbc connection allocation stack trace, we need to enable the “ConnLeakLogic” trace. In the “topology” view, right click the “sick” server and select the “Enable Trace…” from the context menu.

In the popup dialog box, select the “ConnLeakLogic” trace and click “Ok”, button to enable the trace.

Figure46: Enable the “ConnLeakLogic” trace

After several minutes, right click the “sick” server again and select the “Show connection pool contents” from the context menu.

Figure47: Show connection pool contents

In the connection pool content window, select the “TradeDataSource”, you will be able to check all the connections status, including the allocation time, used time, used thread number and allocation stack trace.

Note: If you did not enable the “ConnLeakLogic” trace, the allocation stack trace will not be able to shown here. Besides, we only be able to get the alloction stack trace info for connections that allocated after we enable the “ConnLeakLogic” trace, other

connections’ allocation stack trace is not recorded.

Figure48: Connection pool contents

From the connection pool content info, we will be able to see that lots of connections are used for more then 200 seconds by some threads, but all of them can not be found in the thread dump, which means that the threads are completed but the connections are not released, so it is a connection leak problem. Besides, nearly all of the leaked connections were allocated by the following stack trace, so we can fix the problem in our code accordingly.

Scenario5: “Memory Leak” problem

Application Changes to Simulate “Memory Leak” Situation

In this scenario, some requests will put some byte[] to the static ArrayList which will never be released. You can adjust the “Leak Size” to control the leak rate.

Code changes in the TradeAppServlet

//For Memory Leakpublic static ArrayList leakContainer = new ArrayList();

if(TradeConfig.isMemeoryLeak()){

leakContainer.add(new byte[TradeConfig.getLeakSize()]);

}

Trigger the “Memory Leak” problem

In the “Server” tab of Daytrader home page, select the “Memory Leak” checkbox and click the “Update Configuration” button. It will leak the heap memory at the specified rate.


Monitoring the “Memory Leak” problem

When a connection leak problem occurs, you will see the red data and warning marks in the dashboard.


Double click the server panel to switch to the monitoring page, and you can see runtime alerts in the alert section.

Figure51: Alert Panel

Click the alert point to check the alert content in the detailed data page.

Figure52: Detailed Alert Contents

Determine the “Memory Leak” problem

In order to determine the memory leak problem, we need to generate a heap dump. In the “topology” view, right click the “sick” server and select the “Generate heap dump” from the context menu. The heap dump file will be generated in the profile home by default.

Figure53: Generate a heap dump

Then add the heap dump into the "Case" in the ISA v5.

Figure54: Add heapdump file to you case

In "Tools" panel, Launch "Memory Analyzer" for JVM heap dump.

Figure55: Analyze the heap dump

Click "Launch", then Click "Browse" to select a dump file. Submit the task.

Figure56: Start to analyze

The task is running as a background task. A pop-up window appears to indicate the output directory:

Figure57: Confirm dialog

The execution history and status are shown in the bottom of "memory analyzer'" page.

Figure58: Status panel

Memory analyzer takes several minutes to analyze the dump file. Please be patient when the task is running background.

Check the output directory for the heapdump analysis and open the report file "heapdump.*_analyzer.html".

Figure59: Check the analysis results

Select the "Leak suspects" reports for details. In this sample, "problem suspect 1" consumed 237.4MB memory.

Figure60: Leak suspects

Expand the detail analysis for problem suspect 1, you will find the memory issue is introduced by "ArrayList" objects in TradeAppServlet, which is the code changes we did in section Application Changes to Sils -lmulate "Memory Leak" Situation.

Figure61: Detailed info of leak suspects

For more information about Memory Analyzer, visit http://www.ibm.com/developerworks/java/jdk/tools/index.html

Other optional functions you can go through

Tuning the performance related parameters

There are 2 ways to change the settings of application server in the PTT. The first one is via the “Tuning Parameters” view.

http://www.ibm.com/developerworks/java/jdk/tools/index.html

Figure62: Tune the perfomance parameters

The second way is executing tuning scripts in the scripts view. Some useful tuning scripts have been added to the toolkit and you can make some modification base on them.

Figure63: Run wsadmin scripts

Adjust rules from rule editor if needed

The WebSphere Application Server Performance Tuning Toolkit (PTT) allows users to define

their own rules according to the indicator of their system. For example, set a proper threshold for

response time of each servlet. All the customized rules for detecting performance decline and

locating the bottleneck are write in the rule file.

The existing rules:

· If the used heap reaches the 85% of maximum heap size, it will generate a runtime alert

· If the used thread pool reaches 90% of the max pool size, it will generate a threadpool alert

· If the average CPU usage reaches 90%, it will generate a runtime alert

· If some servlet errors occur, it will generate a servlet alert

· If some JDBC connection timeout occur, it will generate a connection alert

· If there are more then 1000 prepared statement discarded, it will generate a connection alert

· If there are some thread waiting for connection, it will generate a connection alert

· If some JCA connection error occur, it will generate a JCA alert

· If there is no room for new session, it will generate a session alert

· If some hung threads are declared, it will generate a threadpool alert

· If some transactions rolled back, it will generate a transaction alert

· If some transactions timeout, it will generate a transaction alert

· If some proxy request failed, it will generate a proxy alert

· If a servlet performance decline detect it will generate a servlet alter

· If a jdbc performance problem detect, it will generate a connection pool alert

All these rules are common rules, and you may need to modify them according to the characters of your own system. For example, we can change the threshold of servlet response time according to the performance data we collected to make the performance problem detection rule more precise.

Figure64: Modify the rules

Generate report

You can also generate a report to get a clear view of your system.

Figure65: Generate a report

More Functions will be added later ……

Technology

Web sphere application server performance tuning workshop