Optimizing Your WebSphere Applications with Optim and pureQuery

Curt Cotner

IBM Fellow, CTO for IBM Database Servers

Development Tools for Your Java Database Applications

1. Select table

Java Data Access in 5 Simple Steps

2. Name bean & select styles 3. Generate test code

4. Map table to bean 5. Select template SQL CRUD

4

What happened?Generate code from table

The code is generated for Bean (aka Java Object) representing the table

Interface with associated SQL or Sample Application with Inline SQL

Implementation of Interface

Optionally a JUnit test case.

5

Generated Test Cases for Your SQL

• Jump start your migrations!

6

SQL analysisSQL executionSQL validation

Developing with pureQueryUnleash SQL from Java

SQL content assist

7

SQL content assist SQL analysisSQL execution

Developing with pureQueryUnleash SQL from Java

SQL validation

8

SQL validationSQL content assist SQL analysis

Developing with pureQueryUnleash SQL from Java

SQL execution

Run SQL with parameters at design time in the Java program WITHOUT writing a test application

9

SQL executionSQL validationSQL content assist

Developing with pureQueryUnleash SQL from Java

SQL analysis

View explain plans for SQL inside you Java programs

10

pureQuery OutlineSpeed up problem isolation for developers – even when using frameworks

• Capture application-SQL-data object correlation (with or without the source code)

• Trace SQL statements to Java source code for faster problem isolation

• Enhance impact analysis by identifying specific application code impacted by a database changes

• Answer “Where used” questions like “Where is this column used within the application?”

• Use with modern Java frameworks e.g. Hibernate, Spring, iBatis, OpenJPA

11

pureQuery Outline

pureQuery Outline View’s 3 categorizations

• How do you look at the relationship between SQL and Java?

Optim Development Studio

• Problem determination and isolation

– with pureQuery outline go to the source of the problematic SQL

• Improve Hibernate data access calls

– Re-write HQL– Use better

performing native SQL

• With performance metrics Identify the hot spots in your hibernate application

Source code correlation

Database object correlation

Identify and change the HQL

Analyzing Performance

Analyze Use of Sensitive Data

See queries accessing sensitive data

Optim Development Studio

Icon identifies sensitive access

Filter to see use of sensitive data

View only SQL accessing

sensitive data

Filter SQL by action

IBM Data Studio

pureQuery For DBAs

and Applicatio

n Developer

s (v1.2) - Part 1

1515

pureQuery ToolsSQL templates and customizations

Create your own SQL templates

Use templates to write SQL that is frequently reused

Use tabs to change the variable names after inserting SQL statement from the template through SQL context assist

Use newly created merge template in your java code using SQL assist

1616

Java/SQL Development on SteroidsWhy the 10x Productivity Improvement

Optimizing Your WebSphere Applications

DB2 ConnectRecommended Deployment Options

Desktop PCsDesktop PCs

Application serversApplication servers

Web application serversWeb application servers

Several options: Personal Edition or DB2 Connect Server or file server

Several options: Personal Edition or DB2 Connect Server or file server

Co-locate DB2 Connecton the application server

Co-locate DB2 Connecton the application server

Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops

Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops

Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring

Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring

Open Source Persistence

Engine

JPA Persistence Engine

JDBC API

Hibernate, iBATIS, EclipseLink,...

JPA for WebSphere, Apache OpenJPA

pureQueryAPI

Data Studio pureQueryPlain JDBC

Data Web Services, Project Zero, sMash

On-ramps to pureQuery

JCC driver

JDBC

.Net Applications

.Net applications

pureQuery

JPA APIOpen Source

Persistence API Web API

ADO .Net

DB2, Informix, and Oracle now…more coming

Client OptimizationImprove Java data access performance– without changing code

• Captures SQL for Java applications– Custom-developed, framework-based, or packaged applications

• Bind the SQL for static execution without changing a line of code– New bind tooling included

• Delivers static SQL execution value to existing DB2 applications– Making response time predictable and stable by locking in the SQL access path

pre-execution, rather than re-computing at access time– Limiting user access to tables by granting execute privileges on the query

packages rather than access privileges on the table– Aiding forecasting accuracy and capacity planning by capturing additional

workload information based on package statistics – Drive down CPU cycles to increase overall capability

• Choose between dynamic or static execution at deployment time, rather than development time

Optim pureQuery Runtime for z/OS

• In-house testing shows double-digit reduction in CPU costs over dynamic JDBC

•

• IRWW – an OLTP workload, Type 4 driver• Cache hit ratio between 70 and 85%• 15% - 25% reduction on CPU per txn over dynamic JDBC

274

360420 446

485524

0

100

200

300

400

500

No

rmalized

Th

rou

gh

pu

t (I

TR

)

Normalized Throughput by API for JDBC Type 4 Driver

-35%

-14%

6%15%

25%

-50%

% in

cre

ase/r

ed

ucti

on

in

CP

U p

er

tran

sn

co

mp

are

d t

o J

DB

C

% increase/reduction in CPU per transaction compared to JDBC using Type 4 driver

Throughput Increase with .NET• Same IRWW OLTP application used for the Java tests but in .NET

• Application access DB2 for z/OS via Windows Application Server (IIS)

• Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio

pureQuery -- More Visibility, Productivity, and Control of Application SQL

• Capture SQL• Share, review, and optimize SQL• Revise/optimize SQL and validate equivalency

without changing the application• Bind for static execution to lock in service level

or run dynamically • Restrict SQL to eliminate SQL injection

Capture Review Optimize Revise Restrict

Visualize execution metrics

Execute, tune, share, trace, explore SQL

Replace SQL without changing the application

Position in Database Explorer

Visualize application SQL

Have You Heard of SQL Injection?

pureQuery – Stripping Literals from SQL

JDBC app

INSERT INTO T1 VALUES(‘ABC’,2,’DEF’) INSERT INTO T1

VALUES(:h1,:h2,:h3)

pureQueryRuntimeconversion

• pureQuery can identify statements that use no parameter markers, and strip the literals out at runtime• significant performance gains:

• less CPU cost at PREPARE• better use of dynamic statement cache

What Is Heterogeneous Batching?

Data Server

Table 2, operation 2

Table 1, operation 1

Table 1, operation 2

Table 1, operation 3

Table 2, operation 1

Table 3, operation 1

Heterogenous Batching – multiple operations across different tables all execute as one batch

Table 1, operation 4

JDBC Batching vs pureQuery Batching• JDBC batching used by

Hibernate Batcher is currently limited

– Cannot batch entities that map to multiple tables

• Primary and Secondary tables.• Inheritance Join and Table per

class strategies– Cannot batch different operations

against same table• Field level updates• Insert, update

– Cannot batch different entities• pureQuery heterogeneous

batching plug-in– Can batch entities that map to

multiple tables– Can batch different operations

against the same table– Can batch different entities into a

single batch– Combines insert, deletes, updates

into single batch

The advantage of Heterogenous Batching

050

100150200250300350400450500

0 20 40 60 80 100

# of operations per transaction

Elap

sed

Tim

e (m

Sec)

HeteroBatching

No Batching

JDBC Batching

* Preliminary findings based on validation with a test designed to demonstrate heterogeneous batching differences. This is not intended to be a formal benchmark.

OpenJPA and Hibernate --SQL Query Generation

JPA Query

Select emp_obj(), dept_obj()

SQL

Select * from EMP WHERE …Select * from DEPT WHERE …

JPAquery transform

• Hibernate and OpenJPA often rewrite queries• No database statistics are used – entirely heuristic!!!• Can often result in poorly performing queries

Monitoring of Java Applications

Optim Database Performance Manager future directions -- Associate SQL with Java SourceHeat

ChartDashboard

Alerts

SLAs

In-flight analysis

Database: Accounting

Statement text schema E2E elapsed occurrences sort time phys. I/O

SELECT TIME FROM UNIVERSE SAP3 132.13 1323 123.32 1.303

SELECT SALARY FROM PAYMENT … SYSIBM 323.4 221 11.3 32.1

DELETE FROM ACCOUNT WHERE AID = 3…

PROC 23.3 435 32322.3 32.1

TOP 3 currently running SQL Statements

TOP by DS elapsed DS CPU time Physical I/O Sort time

- +

- +

- +

SELECT TIME FROM UNIVERSEStmt text Analyze

Application DS user ID KARN Client IP addr / hostname TPKARN.de.ibm.com Client user ID KARN Client workstation name TPKARN Client application name Jawaw.exe Client accounting N/A application name Online banking application contact hkarn@de.bm.com package West.OLBank class Account method Transfer() source line 314

Time distribution

Force applicationStop SQL

sorting

Resource usageQuery cost estimates 18.456Buffer Pools Data – hit ratio (%) 43.4% Data – physical reads / min 4323 Index – hit ratio (%) 54.2% Index – physical reads / min 3214

Statement information X

DS Proc

USER CPU SYSTEM CPU

Unacc wait

DS sorting

Statement elapsed timeCurrent 132.13 sec last day 239.40 seclast week 15.60 sec

Toughest issue for Web applications – Problem diagnosis and resolution

Web BrowserUsers Web

Server

Application Server

DB2Server

Business Logic

Data A

ccess Logic

Persistence Layer

DB

2 Java Driver

JDBCPackage

EJB

Query Language

What’s so Great About DB2 Accounting for CICS Apps?

z/OS LPARCICS AOR1Txn1 - Pgm1 - Pgm2

CICS AOR2TxnA - PgmX - PgmY

DB2PROD

CICS AOR3Txn1 - Pgm1 - Pgm2

App CPU PLANTxn1 2.1 TN1PLNTxnA 8.3 TNAPLN

DB2 Accounting for CICS apps allows you to study performance data from many perspectives:• By transaction (PLAN name)• By program (package level accounting)• By address space (AOR name)• By end user ID (CICS thread reuse)

This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.

JDBC Performance Reporting and Problem

Determination – Before pureQuery

Application ServerDB2 or IDS

A1

A2

A5

A3

A6

A4

USER1

USER1

USER1

User CPU PACKAGEUSER1 2.1 JDBCUSER1 8.3 JDBCUSER1 22.0 JDBC

What is visible to the DBA? - IP address of WAS app server - Connection pooling userid for WAS - app is running JDBC or CLI

What is not known by the DBA? - which app is running? - which developer wrote the app? - what other SQL does this app issue? - when was the app last changed? - how has CPU changed over time? - etc.

Data A

ccess Logic

Persistence Layer

DB

2 Java Driver

EJB

Query Language

What’s so Great About Optim pureQuery Accounting for WebSphere Applications?

z/OS LPAR

CICS AOR2TxnA (PLANA) - PgmX - PgmY

App CPU TxnA 2.1 TxnB 8.3

Data Studio and pureQuery provide the same granularity for reporting WebSphere’s DB2 resources that we have with CICS:• By transaction (Set Client Application name )• By class name (program - package level accounting)• By address space (IP address)• By end user ID (DB2 trusted context and DB2 Roles)

This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.

Unix or Windows

WAS 21.22.3.4TxnA (Set Client App=TxnA) - ClassX - ClassY

Simplifying Problem Determination Scenario

Application Developer

• Available for each db access

–SQL text generated

–Access path

–Cost estimates

–Estimated response time

–Elapsed & CPU time

–Data transfer (getpages)

–Tuning advice

Database Administrator

• Available for each SQL

–Application name

–Java class name

–Java method name

–Java object name

–Source code line number

–Source code context

–J-LinQ transaction name

–Last compile timestamp

Java

Profiling

pureQuery

DRDA Extentions

Using pureQuery to Foster Collaboration and Produce Enterprise-ready Apps

Application Server

Catalog data for SQL

ApplicationMeta data

DB2 or IDSProd

A4

A1

A1

A6

A6

A2

A2

A3

A3

A4

A4

A5

A5

A1

A4

A5

PerformanceData

Warehouse

Application Developer Database

Administrator

A1

A6A2 A3

A4 A5

Quickly compare unit test

perf results to production

Use pureQuery app metadataas a way to communicate in terms familiar to both DBA and developer

ApplicationMeta data

DB2 or IDSDev System

A1

A6

A2 A3

A4 A5A1

A4

A5

Customer Job Roles – A Barrier to a “Holistic View”

Application Server

DBServer

Data A

ccess Logic

Persistence Layer

DB

Java Driver

JDBCPackage

EJB

Query Language

WebS

phereC

onnectionP

ool

Busines

sLogic

13

54

2

ApplicationDeveloper

SystemProgrammer DBA

NetworkAdmin

DB2 Performance Expert and Extended Insight• Enables early and rapid problem detection to

prevent impact to production systems– Identify resource shortages across CPU, memory,

and file systems, locking conflicts and deadlocks, and data skew

• Increase ability to meet service level agreements

– Provides optimization and tuning recommendations

• Supports trend analysis and growth planning– Maintains and analyzes performance warehouse

• Supports a variety of database workloads – Includes online transaction processing, data

warehouse, and enterprise content management– Monitors Workload Management environment

• Visualizes the end-to-end response time – See where Java database workloads,

transactions, and SQL requests are spending their time across the database client, application server, and network

Provides in-depth database monitoring, problem isolation, and trend analysis for DB2 for

Linux, UNIX, and Windows databases. Add Extended

Insight Feature for visibility into where Java database

workloads, transactions, and SQL requests are spending their

time

Scenario

It seems that the first application server has a problem. Double-click to drill-down.

In this situation, all applications are equally affected, and the problem seems not to be in the data server.

Scenario - continued

Double-click to drill-down and display detail information.

Most of the time is spent for „WAS connection pool wait“ time.

Scenario – continued

5 second wait time indicates that the maximum number of allowed connections is not sufficient…

… which becomes also evident when comparing the parameters and metrics of this client with other clients.

Where to get IBM Data Studio ?

• IBM Data Studio –www.ibm.com/software/data/studio

• FAQs / Tutorials

• Downloads

• Forum / Blogs

• Join the IBM Data Studio user community