Tom Quinn, Principal Consultant, CA technologies St Louis CMG August 23, 2011 St Louis

Kayaks and Cruise Ships which would you rather turn around?Tom Quinn, Principal Consultant, CA technologies

St Louis CMG

August 23, 2011

St Louis

http://www.cmg.org/

2Tom Quinn Kayaks and Cruise Ships - Application Performance Management August 23, 2011

— various definitions of apm— management vs monitoring— understanding your applications— business volume metrics vs system volume metrics— application performance budget— application performance monitoring

agenda

St Louis

http://www.cmg.org/


- application performance management vs application performance monitoring- application performance management refers to the processes and

procedures used to ensure applications have and meet performance goals. application performance monitoring is a piece of this.

- application performance monitoring refers to the use of tools to gather metrics, real time status, identification. this can be done throughout the software development lifecycle (sdlc)

- performance falls under the ITIL discipline of Availability Management- In accordance with ITIL, the primary objective of the Availability Management

process is to define, plan, evaluate and improve all aspects of the availability of IT services, establishing and maintaining them in compliance with agreed availability targets. These aspects include the people, process and technology factors of availability.2

what is apm?

2Availability Management: http://www.ca.com/files/processmaps/availability-mgmt-process-map_222472.pdf

St Louis

http://www.cmg.org/

apm

tools

people

process

data

— apm: application performance management− the use of technology to

initiate, deploy, monitor, fix, update and/or optimize systems within an organization.1

− apm is more than end-user monitoring. it is a philosophy as much as a tool that spans across many areas.

− today’s applications are complex and span platforms and teams, it’s important to understand all the pieces.

1. APM Best Practices. Sydor. CA Press


St Louis

http://www.cmg.org/

5

levels of monitoring

End User Experienc

e

Business Transactio

ns

Architecture

Software Compone

nts

— monitoring occurs at various levels and by different audiences− end user experience

• what is your customer experiencing?

− business transactions• how are the individual business

transactions performing?

− architecture• how are the systems performing for

all things running on it?

− software• how are the individual components

that make up the business transactions performing?

Tom Quinn Kayaks and Cruise Ships - Application Performance Management August 23, 2011St Louis

http://www.cmg.org/


— A positive customer experience is your business lifeline.

— IT is critical to delivering a positive customerexperience to drive revenue and growth.

why do you need apm?

St Louis

http://www.cmg.org/


today’s applications are complex

WAN/WWW

End User

Router Firewall Switch Web Servers

Load Balancer

Portal

SAP

Siebel

Web Services

3rd Party Applications

Databases

MAINFRAME PSFT

NETWORKUSER

BACK END

FRONT END

MIDDLEWARE

App Server

CICS

Datacom

DB2

CTG

IMS

MQ

z/OS

today’s applications span platforms and teams, are self developed and vendor code.

St Louis

http://www.cmg.org/

who’s measuring the entire thing?


Web Server

App Server

MQ

DB2

z/OS

CICS

End-User App

05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00

99%

99%

99.9%

99.9%

99.999%

99.9%

?

% available

Unavailable or SlowAvailable, Performant

And no one is measuring the real customer experience…

St Louis

http://www.cmg.org/

how to start with apm

Tom Quinn Kayaks and Cruise Ships - Application Performance Management August 23, 2011

9

St Louis

http://www.cmg.org/


— business volume metrics (bvms) – the number of business transactions executed during a given time interval with peaks identified

— system volume metrics (svms) – the number of system transactions executed on behalf of a business transaction

— application performance budget – the resource constraints (response time, cpu, etc) for a given application and its components

— critical path – the application path that is most used and provides the most business value

— peak hour – identifying when system utilization is likely to spike based on higher business/transaction volume

terminology

St Louis

http://www.cmg.org/


APM

System Analysis

Design

Development

Implementation

Validation

Evaluation

sdlc and performance

— APM can be incorporated into all phases of the sdlc

— get performance involved early and often

— it is less expensive to fix problems early in the cycle – build a kayak! the later you discover these problems, the harder and more expensive they are to fix – the cruise ship.

— build performance requirements into your project plans

Performance gateway

St Louis

http://www.cmg.org/


—the business defines what the problem is, in essence, what the application is to do

—requirements are gathered−end user response time goals

—business transactions are determined−ex: Balance Inquiry

—business volume metrics are built

system analysis

St Louis

http://www.cmg.org/


—Build your application Business Volume Metrics (BVMs)−define the business transactions this application contains

• Balance Inquiry

−Estimate how many times per day each business transaction will be invoked• Balance Inquiry 150,000 times per day

−Estimate the transaction distribution and peak hour• Balance Inquiry 150,000 times per day

• 80% 7:00 am – 5:00 pm 120,000 or 12,000/hour

• Peak hour11:00 am – 12:00 pm 50% increase 18,000/hr (11,334/hour)

Business Volume Metrics (BVMs)

St Louis

http://www.cmg.org/


—Classify your business transactions based on response time goals and importance−Tier 1: 0.0 – 3.0 seconds response time

−Tier 2 0.0 - 5.0 seconds response time

−Tier 3 0.0 – 10 seconds response time

—This is intended to help you determine priorities in your development and within your system−Tier 1 transactions get priority in your system

Business Volume Metrics (BVMs)

St Louis

http://www.cmg.org/

15

design

— Describes desired features and operations in detail, including screen layouts, business rules, process diagrams, pseudocode and other documentation.

— performance management begins with application design. get involved early!

— have baselines of the proposed architectures available to help in your decision making− data sources: own environment

metrics, vendor metrics, industry experts

— build the Application Performance Budget


http://www.cmg.org/


—Application Performance Budgeting is an approach to ensure applications meet their response time objectives and proactively mitigate performance concerns at any phase – development, performance testing, production.

—Always updating and validating

—May assist in identifying performance bottlenecks and inefficiencies.

—A Performance Budget breaks down a transaction to allow for the developer to code to requirements and enable operations to maintain the application and continue meeting design specifications.

application performance budget

St Louis

http://www.cmg.org/

17

application design and the performance budget

— identify the required pieces of the application before a single line of code is written− data sources and locations

− security/authentication requirements

− hardware hops

— measure the infrastructure response times.

before a single line of code is written, you know the fastest this application will ever run!

Tom Quinn Kayaks and Cruise Ships - Application Performance Management August 23, 2011 St Louis

http://www.cmg.org/


Client Tier Network Tier Middle Tier Database Tier Transaction Class 1 0 – 3 second objectives

.3 seconds 1.9 seconds .5 seconds .25 seconds

Transaction Class 2 0 – 5 second objectives

.45 seconds 2.9 seconds .9 seconds .7 seconds

Transaction Class 3 0 – 10 second objectives

.75 seconds 4.5 seconds 1.75 seconds 2.75 seconds

build the application performance budget

-start with what you know. use existing metrics, industry standards, vendor metrics-start by allocating time across the tiers for each transaction class to add up to the maximum response time.

St Louis

http://www.cmg.org/


populate the application performance budget

Class component name Tier # of execs cpu response cumulative cpu cumulative response

Balance Inquiry 1 abcd client 2 0.002 0.09 0.004 0.18

defg middle 1 0.001 0.15 0.001 0.15

wxyz db 3 0.025 0.83 0.075 2.49

mnop middle 1 0.1 0.43 0.1 0.43

0 0

0 0

0 0

0 0

0 0

total 7 0.128 1.5 0.18 3.25

- application performance budgets can vary in information, look and feel, but the goal is the same – help develop well performing transaction that meet response time goals

St Louis

http://www.cmg.org/


—make application performance testing a requirement

—test at least your defined critical path

—gather numbers to populate and validate the application performance budget

—build the application monitoring in the test environment−build end to end dashboards

performance testing

St Louis

http://www.cmg.org/


—create a performance gateway where the performance team reviews the performance testing numbers and must sign off before the application can be implemented into your production environment.−this is the final opportunity to keep poor

performing applications from being implemented into production

−this also gives you an opportunity to acquire necessary capacity planning numbers

implementation

St Louis

http://www.cmg.org/


—once in production, capture performance data for these transactions and compare to your application performance budget and validate your model

—if large discrepancies, look at why?−test infrastructure is too different than production

−BVM’s are incorrect

−database inefficiencies

validation

St Louis

http://www.cmg.org/


—can improvements be made?−can I tune the infrastructure?

• more cics or ims regions

• classify and group workload– horses, rabbits and elephants

• caching

−can I tune the application?• break transactions into multiple pieces

• verify database indexes are the same in production as tested against

evaluation

St Louis

http://www.cmg.org/


— implement the application dashboards built during performance testing into the production environment

— compare performance test numbers to production numbers

— set up thresholds and alerts based upon these numbers

— publish performance goals for transactions− example: 95% of CICS transactions

must be 0.5 seconds or less

application performance monitoring

St Louis

http://www.cmg.org/


— the earlier you get involved, the earlier performance problems can be identified and corrected

— build in a performance gateway to approve or deny new and changed applications that will/will not perform

— maintain application performance budgets for future development

get engaged early and often

St Louis

http://www.cmg.org/


questions?

St Louis

http://www.cmg.org/

27

— Tom Quinn is a Principal Consultant, Mainframe Technical Sales at CA technologies responsible for mainframe performance monitoring products. Tom joined CA in 2009, bringing with him over 15 years of technical and leadership experience in Information Technology. Having worked for a large Mid-Western insurance company in mainframe system and application performance, Tom understands the technical challenges a large organization can face and utilizes his ITIL and ScrumMaster experience to discover and build customer solutions. You can reach him at [email protected] or [email protected].


mailto:[email protected]


http://www.cmg.org/


— Kayaks and Cruise Ships – which would you rather turn around? Today’s applications are complex, and competition is a click away. Performance needs to be involved in every step of the application lifecycle, ensuring you’re building a kayak that can turn easily when nearing troubled waters. If you build a cruise ship, you can turn it, but it’s costly and slow. In this session you will learn about the concepts and techniques available to implement Application Performance Management in your shop.

— Bulldozing the Bottleneck– the CA Solution : Do you know where your bottleneck is? How fast can you find the right one and then eliminate it? CA technologies’ Cross Enterprise APM solution helps you manage enterprise business applications, quickly pinpointing the problem and helping you resolve it. Work smarter by having the enterprise picture at your fingertips.

abstract

St Louis

http://www.cmg.org/

thank [email protected] or [email protected]

Tom Quinn Kayaks and Cruise Ships - Application Performance Management August 23, 2011 29

St Louis



http://www.cmg.org/

Documents

Tom Quinn, Principal Consultant, CA technologies St Louis CMG August 23, 2011 St Louis