Upload
rachel-niedzwiecki
View
155
Download
0
Embed Size (px)
Citation preview
#ibmiod
Proactive Performance Tuning: A Day in the Life of a DBA at ADP, Inc Session Number IDZ-1620
Rachel Niedzwiecki, ADP
Saghi Amirsoleymani, IBM
#ibmiod
Please note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM
benchmarks in a controlled environment. The actual throughput or performance
that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user’s job stream,
the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve results
similar to those stated here.
#ibmiod
Acknowledgements and Disclaimers:
© Copyright IBM Corporation 2012. All rights reserved.
– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
– Please update paragraph below for the particular product or family brand trademarks you mention such as WebSphere, DB2,
Maximo, Clearcase, Lotus, etc
IBM, the IBM logo, ibm.com, [IBM Brand, if trademarked], and [IBM Product, if trademarked] are trademarks or registered trademarks of
International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are
marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common
law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law
trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
f you have mentioned trademarks that are not from IBM, please update and add the following lines:
[Insert any special 3rd party trademark names/attributions here]
Other company, product, or service names may be trademarks or service marks of others.
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are
provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice
to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is
provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of,
or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the
effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the
applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may
have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these
materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific
sales, revenue growth or other results.
#ibmiod
Agenda
• Automatic Data Processing, Inc. - Who are we?
• DB2 Query Monitor (QM) – Overview
• Optim Query Workload Tuner (OQWT)- Overview
• How ADP uses these tools to help with both reactive and proactive performance resolutions.
• ADP Performance Success Stories using these tools.
• QUESTIONS?
3
#ibmiod
Automatic Data Processing, Inc.
Automatic Data Processing, Inc. (NASDAQ: ADP), • About $10 billion in revenues
• Approximately 570,000 clients
• Is one of the world's largest providers of business outsourcing solutions.
• Leveraging over 60 years of experience, ADP offers a wide range of human resource, payroll, tax and benefits administration solutions from a single source.
• ADP's easy-to-use solutions for employers provide superior value to companies of all types and sizes.
• ADP is also a leading provider of integrated computing solutions to auto, truck, motorcycle, marine, recreational vehicle, and heavy equipment dealers throughout the world.
http://www.adp.com
4
#ibmiod
DB2 Query Monitor v3.1
• SQL Monitoring Tool
o Not Thread or Subsystem monitor
• Low overhead SQL statement monitor
• Identify problem SQL activity
• Determine the tables and indexes used by SQL statement
#ibmiod
InfoSphere Optim Query Workload Tuner Requirements
Benefits
Get expert, actionable advice to
improve SQL workload
performance
Improved performance
Improved productivity
Reduced costs
Reduced risks
Optimize
Performance
Reduce skill
requirements and
increase productivity for
tuning
Analyze entire workloads
rather than query at a
time
Provide expert query
tuning advice that
analyzes tradeoffs
#ibmiod
Streamlined Analysis with expert advisors
• Statistics
o Get recommendations on the best statistics to capture to influence access path selection
• Access Path
o Get recommendations on
how to influence the
access path selection
• Query
o Get recommendations regarding how to rewrite the query for better efficiency
• Index
o Get recommendations on indexes changes that can reduce database scans
#ibmiod
How ADP Uses Query Monitor & Query Workload Tuner
• Reactive Performance issues: Top 25 CPU Usage Report : identifies poor performing jobs with high CPU usage or support calls for jobs running longer than expected.
• Proactive performance issues: SQL Reviews
8
#ibmiod
Top 25 CPU Usage
• ADP runs this report daily to identify our top 25 jobs with the highest CPU usage. These jobs are reviewed and each one starting with the worst and working our way down are evaluated using QM and OQWT.
• With QM we are able to quickly drill down and identify the problem SQL(s) for the specific programs run in the job.
• Using OQWT we are able to run the various advisors to help us quickly identify possible improvements.
9
#ibmiod
TOP 25 CPU Users Report
10
• We produce these reports daily using data that we store in DB2 in our PM History Database.
#ibmiod
Query Monitor: Identify the “culprit”
• We use Query Monitor to then look up the job that was identified in the TOP 25 report to identify the query(ies) and their respective program(s) that utilized the majority of the CPU.
11
#ibmiod
Query Monitor: Continued
12
#ibmiod
Query Monitor: List SQL statements for a given Job
13
#ibmiod
QM: Identify Poor SQL and its Program
14
#ibmiod
QM: Identifying the “problem” Query
15
#ibmiod
OQWT: Execute Advisors
16
#ibmiod
OQWT: Single Query: Continued
17
#ibmiod
OQWT: Single Query: Continued
18
#ibmiod
OQWT: Single Query Analysis results
19
Recommendations by severity
#ibmiod
OQWT: Stats Advisor Recommendation
20
At this point we can try running the Stats and redo the advisors
Generates
RUNSTATS control statements
Improve Statistics Quality and Collection
#ibmiod
OQWT: Access Path Advisor
21
Interrogate the plan table to discover access plan flags
#ibmiod
OQWT: Index Advisor
22
Estimated performance
improvement
Indexing Advice to Improve Query Efficiency
#ibmiod
OQWT: Tuning Report (shows all advisor recommendations)
23
You can save the
report to your PC
#ibmiod
TOP 25 CPU Report Success Stories (1)
• Identified a job that was first on the report for months. Sent summary report to developers and verified with them the expected performance improvement for implementing some of the advisor recommendations: Here are the results of the recommended index change
o DSN_STATEMNT_TABLE ESTIMATED COST CHANGE - OLD EST SVC UNITS 823,974
o DSN_STATEMNT_TABLE ESTIMATED COST CHANGE - OLD EST CPU 18,952
o DSN_STATEMNT_TABLE ESTIMATED COST CHANGE - NEW EST SVC UNITS 163,377
o DSN_STATEMNT_TABLE ESTIMATED COST CHANGE - NEW EST CPU 3,758
• 80% reduction in CPU cost
24
#ibmiod
TOP 25 CPU Report Success Stories (2)
• We have a job that runs all day Tuesday through Saturday and has peak processing every Wednesday and Thursday. Its accounting information shows that it spends almost all of the time in DB2 and all application CPU is same as DB2 CPU time.
• Through QM we quickly identified the most expensive SQL where we verified that it had a bad access path doing a tablespace scan for some of its query blocks.
• Then we generated OQWT advisor reports for this problem query. We implemented the recommendation of a new index on the table being scanned and performance of the query improved by as much as 70% : ( the query is a SELECT COUNT(*)…… DELETE FROM…….)
25
#ibmiod
What to do next …SQL Reviews
• At ADP we conduct “DART” (DB2 Application Review Template) reviews for new and modified SQL for our upcoming application releases.
• At these reviews Queries are reviewed along with their explain output. If an access path is not acceptable (TBL or IX scans, IX/MC 1 on a low cardinality column, etc.) then we provide an OQWT summary report with recommended improvements.
• In most cases the recommendation is a change to the query, in some cases it was a change or addition of an index.
26
#ibmiod
SQL Reviews: Example • This is taken from an OQWT Query Advisor report:
27
Highlights relevant components of the
query
Recommendation and rationale
#ibmiod
OQWT Summary report: Access Plan Summary
28
#ibmiod
SQL Review: Results
• After running several iterations of the Query with recommended changes, we put together the cost as shown in the OQWT summary reports to determine the best choice for the Query:
29
We verified that this is the best case by running two DSNTEP2 jobs, one in
the morning and one in the afternoon using the original query and the
modified query for option 2 and the results seen in QM for ET, CPU and
I/O were very close to the results shown in the summary reports for cost.
#ibmiod
Analyze DB2 10 Workload Success Story
30
Best practice for getting workload tuning advice
– Improve statistics quality
– Improve query design
– Improve database design (index & access path )
Speed up analysis, optimize design,
and balance resource usage
Workload : Multiple SQL statements defined by user
•The effort for tuning the whole application with good performance by evaluating every statement is overwhelming . Optimization decisions are based on tradeoffs
•Statistics – CPU costs vs. query savings •Indexing – query speed vs resource and transaction
•When application data grows, allows you to do proactive application health check periodically to find potential problems earlier before costly application outages
#ibmiod
Compare current access path with DB2 10
31
•Adding ‘OPTIMIZE FOR 1 ROW’ clause to the SQL statement for the DAT
Cursor
•creating a new index to match the predicates and here are the cost results:
#ibmiod
Improve Query Design
32
Recommendation and rationale
Guard against errors and
oversights:
Further constrain query,
increase index utilization, and
reduce data reads
#ibmiod
Using QM & OQWT: Big Savings
• One of our DBA’s who is primarily using these tools to research these poor performing jobs and recommend improvements stated the following with regard to how much time on average it takes to find these improvements (this is as recent as work done on 6/19/2012):
• “Using QM, PM and OQWT took one hour. Testing indices took 6 hours as the query was complex and took time to understand which query block is bad performer”.
• So in total, it takes on average less than a day for one DBA to identify improvements for poor performing queries. Identifying a solution has been reduced significantly for us.
33
#ibmiod
Summary
Meet SLAs Increase
Performance
Lower
Infrastructure
Costs
Minimize slowdowns which impact End User Productivity
Avoid Problems with proactive query tuning
Improved HW and DB Utilization & Cost Savings
Database Administrators do more with Less
Faster Time to Problem Resolution
KEY SOLUTION: "IBM InfoSphere Optim Query Workload Tuner for z/OS“
& "IBM DB2 Query Monitor for z/OS"
Lower
Operational
Costs
#ibmiod 35
#ibmiod
Resources
• Integrated Data Management Community
o http://www.ibm.com/developerworks/spaces/optim
• IBM Optim Query Workload Tuner web page
o https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-expert-zos/
• IBM DB2 Query Monitor for z/OS
o http://www-01.ibm.com/software/data/db2imstools/db2tools/db2querymon.html
• SQL Tuning, Not just for hardcore DBAs anymore
o http://www.ibmdmmagazinedigital.com/dmmagazine/
• Redbooks at www.redbooks.ibm.com
o DB2 9 for z/OS Performance Topics SG24-7473
o IBM DB2 9 for z/OS: New Tools for Query Optimization SG24-7421
Contact
Saghi Amirsoleymani, Optim Enablement Specialist, [email protected]
#ibmiod 37
Thank You for Joining Us today!
If you would take a moment to fill out the feedback form which will display on the next slide, it would be greatly appreciated. Your comments are very important to us.
Go to www.ibm.com/software/systemz/events/calendar to:
Replay this webcast
View previously broadcast webcasts
Register for upcoming webcasts
#ibmiod
Communities
• On-line communities, User Groups, Technical Forums, Blogs, Social networks, and more
o Find the community that interests you …
• Information Management bit.ly/InfoMgmtCommunity
• Business Analytics bit.ly/AnalyticsCommunity
• Enterprise Content Management bit.ly/ECMCommunity
• IBM Champions
o Recognizing individuals who have made the most outstanding contributions to Information Management, Business Analytics, and Enterprise Content Management communities
• ibm.com/champion
#ibmiod
Thank You!
Your Feedback is Important!
• Access SmartSite to complete your session surveys
o Any web or mobile browser at iodsmartsite.com
o Any SmartSite kiosk onsite
o Each completed session survey increases your chance to win an Apple TV with daily drawing sponsored by Alliance Tech