65
Oracle Database Performance Management Best Practices Workshop AIOUG Product Management Team Database Manageability

Oracle Database Performance Management Best Practices Workshop · Oracle Database Performance Management Best Practices Workshop ... methodology and associated best practices using

  • Upload
    dodan

  • View
    246

  • Download
    0

Embed Size (px)

Citation preview

Oracle Database Performance

Management Best Practices

Workshop

AIOUG

Product Management Team

Database Manageability

Page 2 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Table of Contents

Oracle DB Performance Management Lab .................................................................................................... 3 A. Configure SPA Quick Check ................................................................................................................................6 B. Find: Performance Problem Diagnosis .............................................................................................................. 10 C. Fix: Tune Regressed SQL .................................................................................................................................. 16 D. Validate: Validate Tuning Using SPA Quick Check on Production ...................................................................... 19 E. Concurrent Replay [Optional] ........................................................................................................................... 32

Appendix A. How to start EM 12c? ............................................................................................................. 62

Appendix B. How to Reset SPA Lab in Section B, C and D? .......................................................................... 62

Appendix C. Real Application Testing overview and Business Cases ............................................................ 64

Page 3 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Oracle DB Performance Management Lab

Objective: The objective of this lab is to provide exercises designed to showcase database performance management capabilities (toolset), methodology and associated best practices using Oracle Enterprise Manager Cloud Control 12c.

Students will learn to apply the “Find-Fix-Validate” performance tuning methodology to diagnose performance bottlenecks, tune SQL statements with poor performance in a safe way directly on production and make sure that the overall result of the tuning activity doesn’t cause new regression.

By improving performance we will be able to free enough resources on the server to perform a server consolidation. We will therefore validate that there are enough capacity on the server after consolidation.

Functional Coverage: In this lab you will go through new features in the following functional areas:

Database Performance management A. Configure SPA Quick Check

B. Find: Performance Problem Diagnosis

C. Fix: Tune Regressed SQL

D. Validate: Validating Tuning Using SPA Quick Check

E. Concurrent Replay

Page 4 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Lab environment setup Your environment has already been setup for the lab. Below are the details of the machine, VNC, and EM URLs.

Oracle Enterprise Manager URL:

https://XX.XXX.XX.XXX:7802/em

username: dbuser

password: oracle12

PDB

“prod12 _DW”

Credentials: Select Named Credential use “DB_SYS_GLOBAL

Host Named credential: HOST_ORACLE

VNC Viewer

Use VNC Viewer to connect to the machine allocated to your machine, localhost:1, password is

“oracle12”

Putty (Alternative to VNC Viewer)

Log on to your environment localhost:22 using Putty

Username: oracle

Passwd : oracle12

Page 5 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 1. Logon with VNC to your assigned machine (machine_ip-address:1)

Password: oracle12

2. Open a terminal: Right click in VNC window

3. Change directory to scripts, the scripts to start / stop databases are here, note you’re using

prod12db, but since the database is already started you can skip this step

cd lab/scripts

Page 6 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

A. Configure SPA Quick Check

Estimated Time to Complete Use Case: 15 minutes

Business Case

Changes in production should always be validated to make sure that the change doesn’t cause any regression. When refreshing Optimizer Statistics for example, we want to make sure that not just the overall performance becomes better but also that no SQL statement, especially business critical ones have regressed. It doesn’t help us to increase the overall performance by 50% if key order entry application degrades in performance as a result of refreshing statistics. To make sure that tuning exercises are completed successfully, we need to validate it by executing SQL statements that cover most of the workload. SPA Quick Check uses a predefined SQL Tuning Set and optimized testing configuration for use in a production environment. We are going to configure SPA Quick check with a pre collected SQL Tuning Set and use the most optimal method for executions. In this lab we will use an Oracle Database 12c Container database (prod12) with one pluggable database (dw).

1. Configure SPA Quick Check 1.1 From your laptop machine, open Firefox or Chrome browser and Log into Enterprise Manager Cloud

Control. Note you’re not using VNC brower but laptop browser.

1.2 Please use URL: https://localhost:7802/em

1.3 Login: dbuser/oracle12

Page 7 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

1.4 Click on Target -> Databases.

1.5 Expand the prod12 database and within the Pluggable databases , select the Pluggable Database

“prod12_dw”

Page 8 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 1.6 Select “Performance -> SQL -> SQL Performance Analyzer Setup”

1.7 Logon using named Credential “DB_SYS_GLOBAL”

Page 9 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

1.8 Select SQL Tuning Set (STS “PENDING_STATS_WKLD”). Click on to get the complete list of STS. Note

that you may find it has already been selected in your case which is fine.

This is the setup page for SPA Quick Check. It gives 3 options for Trial Method. “Optimal” is the default.

Optimal Trial Mode: This is the recommended mode. It finds SQLs with plan changes first by generating

plans, then test-execute every SQL statement with plan changes to validate performance changes.

Per-SQL Time Limit: Default is 300 seconds if the statement has not finished by then it will be terminated

and not validated.

Comparison Metric: Choose “Buffer Gets”. This metric will show us how much work that is done for the

statement by measuring logical reads. This metric is stable and will not be impacted if the SQL is executed

during peak or non-peak load.

This database is not configured with Resource Consumer Groups but this is a way to make sure that the

statement will have limited impact on the server. Be aware that this can cause a statement to be

terminated once it exceeds the defined Per-SQL Time Limit.

Save the changes you have made by clicking on “Save”.

We have now configured SPA Quick Check so it can be used in future exercises.

Page 10 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

B. Find: Performance Problem Diagnosis

Environment Details:

Database prod12 is a multitenant database and it contains only one PDB for the moment. The plan was to move the manufacturing calculation data warehouse (PDB name DW) first and later on move the order Entry system as well. DW has been in production for several months after the migration to the new Linux server. Performance was acceptable during the first month but has slowly degraded due to a combination of increasing data volume and more users.

Estimated Time to Complete Use Case: 17 minutes

Business Case

You have been asked to tune the database. But under no circumstance is it allowed to cause any performance

degradation since it might cause the manufacturing process to stop with a cost of over $1M a day.

In this exercise we are supposed to identify slow SQL statements so we can tune them.

2.1 You should already be logged on to Enterprise Manager. If you are not, please follow the instructions

detailed in earlier section of this workbook.

2.2 Go to Enterprise -> Job -> Library

2.3 Select “RUN_PRODUCTION_WORKLOAD”

Click Submit.

Page 11 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

2.4 Click Submit. (the second submit is a confirmation), note there is no need to select em12 target, it is

picked up automatically

The workload has now started. Just to make sure that the load has ramped up please wait about 1 minute. The job can be

monitored by right clicking on the job name and opening it in a new page.

Page 12 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 2.5 It is now time to see how the database behaves and if there are any SQL statements that indicates poor

performance

Click on Targets -> Databases

2.6 Select pluggable database prod12.oracle.com_DW in prod12.oracle.com

2.7 Wait for 5-7 minutes till the workload that you submitted ramps up so that the top SQL intended for

tuning in this lab appears on the ASH Analytics page. You can explore ASH Analytics feature meanwhile. Go

to Performance -> ASH Analytics to view the information on the page, but do not perform any actions until

after 5 minutes.

Page 13 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

2.8 Select a period of time during the first spike. (If the spike is not as distinct as it is in the picture please

select the first 5 minutes of the workload).

2.9 By default the ASH analytic page is displayed in Activity mode. The load map mode will give us more fine

grained information so let’s switch to this mode.

Click “Load Map”

2.10 We can see that most of the time is spent in “CPU + Wait for CPU”.

Page 14 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

2.11 Let’s change dimension and see which part of the application is using our resources.

Click on the drop down list with current value “Wait Class, Wait Event” and select “Module, Action”

2.12 Our resources are used by Action STAT. Let’s drill down even further to see which SQL statements are the

main contributors.

Click on the box for STAT

Page 15 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

2.13 Select “Wait Class, SQL ID”

2.14 We have now identified our queries that use most resources (You may see SQL IDs 99v2f8wz5am4x,

706vj3n6jvxzn and 807g9b91wb9s5). So let’s remove our filters and go back to the activity view

Click on filter “Action STAT”

Click on “Activity”

Page 16 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

C. Fix: Tune Regressed SQL

In previous section we identified three statements which are the main contributors for resource consumption.

Our goal is to improve the performance of these statements without causing any regressions. Tuning can be

done in several ways, one option is to send our statements back to our developers and they can run manual

tuning on them. But if it is complex statements or if it is an application delivered by an Application vendor then

we might get the answer that it will be implemented in the next release.

Instead we are going to use SQL Tuning Advisor to find out if it can help us with some advice.

3.1 As we can see there are a couple of SQL Statements that cause the major part of the activity. You should

see SQL id 99v2f8wz5am4x, 706vj3n6jvxzn and 807g9b91wb9s5. You can hover over the SQL id for each

SQL and see the initial part of the SQL text.

Select these Three SQL

Please Note: The number of SQLs to tune can be different, depending upon where your slider window in

ASH ANALYTICS is at. Also the SQL ID’s can be different. So the screen shots given below may differ.

Click Tune SQL

Page 17 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 3.2 Review job detail and for easier identification change the name to “SQL_TUNING_BB” where BB is your

initials.

Change “Time Limit per Statement”: 2 (minutes)

Click Submit

3.3 The tuning task will run for 5 to 10 minutes. You can see the progress and the cumulative benefits on the

screen

Page 18 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 3.4 As we can see Tuning advisor has given tuning recommendations for us in the form of SQL profiles,

indexes and new statistics that we should investigate further.

Click on Show all results.

This constitutes the “Fix” or the solution for the problem identified as part of the Find-Fix-Validate performance

methodology.

Page 19 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

D. Validate: Validate Tuning Using SPA Quick Check on Production

Estimated Time to Complete Use Case: 22 minutes

Business Case

SQL Tuning Advisor has now given us advice to create SQL Profiles, create indexes and gather new statistics. We

will now implement the advice that will give us most benefits. But since we are not allowed to jeopardize

current performance then we need to validate that our advices are risk free to implement. This will be done with

SPA Quick Check

4.1 The largest benefit comes by creating SQL Profiles. New indexes have almost as high a benefit. But indexes will

definitely cause performance penalty during insert and updates and can also cause other SQL to regress. We can

also see that statistics are stale so we should have a look at statistics gathering later on. This is because

refreshing Optimizer statistics during peak periods may impact the performance of the system.

Let’s start with SQL profiles. Will we be able to gain what SQL Tuning Advisor have predicted?

Click on “Validate All Profiles with SPA”

Page 20 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.2 Enterprise Manager has now created a SQL Performance Analyzer Task for validating the performance.

Click on the SPA Task

4.3 The SPA Task will run for about 1 minute when “Last Run Status” is “completed click on the Name.

Page 21 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.4 There are four trials executed, first and second are only comparing execution plans, third and forth are full

executions of regressed SQL statements (subset of the workload)

Let’s see the comparison result from the third and fourth trial.

Click on the glasses for the second comparison report.

4.5 As we can see the performance improvement is in line with what the SQL Tuning Advisor recommended.

The benefit may be slightly different from what you see on the screenshot below.

Per SQL Tuning Advisor's recommendations, let’s implement the SQL Profiles.

Click on the breadcrumb for Advisor Central.

Page 22 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.6 Identify your tuning task and click on the name

4.7 Click on SQL Profile

4.8 Click on Implement All SQL Profiles

Page 23 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.9 Check “Implement the new profile(s) with forced matching” and click Yes.

4.10 We have now implemented our new profiles. Let’s see if this had any impact on the workload.

Go to Performance -> ASH Analytics

4.11 Can we see any performance improvement?

Page 24 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.12 We had a tuning advice to refresh Optimizer statistics. Let’s see if this will improve the performance even

further.

4.13 Go to Enterprise -> Job -> Library

Select RESET_LAB_SPA_STATS_VALIDATE

4.14 Click Submit. (the second submit is a confirmation), you do not have to select the target

Page 25 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

The Job will take few seconds to finish.

4.15 Click on Targets -> Databases. Select pluggable database prod12.oracle.com_DW in prod12.oracle.com

4.16 Go to Performance -> SQL -> Optimizer Statistics

4.17 This database is configured to gather statistic in Pending mode. This is done to make sure that new

statistics doesn’t surprise us with changed plans that causing poor performance.

We are now going to gather statistics for the data warehouse schema STAT1

Click Gather

Page 26 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Page 27 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.18 Select Schema

Check “Validate impact ….” (Important and is at the bottom of the page)

Click Next

4.19 Click Add

4.20 Search for “STAT1”

Select schema STAT1

Click OK

Page 28 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.21 Click Next

4.22 Click Next

4.23 Click Submit

Page 29 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.24 A SPA task has just been created and will start as soon as the statistic gathering has finished.

Click on the job name

4.25 The SPA task will take about a minute.

When “Last Run Status” becomes completed click on the Name

Page 30 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.26 As in previous SPA task we have four trials. The first comparison will show changes in Optimizer cost which

indicates new execution plan but the second comparison will show the changes in buffer gets for

statements with new plans.

So let’s bring up the second SPA report by clicking on the glasses.

4.27 We can see that we have 2 improved statements and no regression. The improvement is 86 % on the

workload which is very good. We have now guaranteed that new statistic will improve performance. Lets

publish the new statistic

Click Publish Object Statistics.

Page 31 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 4.28 Let’s find schema STAT1 tables so we can publish the statistic

Change type to Table

Set Schema to STAT1

Change scope to Pending

Click GO

Change action to Publish.

Select one table, Click Go beside Action. Repeat this for each table. For purpose of the lab we’ll skip this

step of publishing statistics and move on to next section.

At this point we have succeeded with tuning of prod12 database. Our next mission is to consolidate our OLTP

database with the Data Warehouse. We can now stop database prod12 and start database prod.

This should be done from the started terminal window in VNC.

Page 32 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

E. Concurrent Replay [Optional]

Business Case

It is not uncommon that different applications have their peaks during different business hours. To have

individual servers for each application will cause many servers to be underutilized for the majority part of the

time. Consolidating applications that have peak workload during different hours/days/weeks/months will

therefore help us utilize database servers in a better way.

To simplify the exercise we have already performed a capture of the DW pluggable database and will perform a

capture of the OLTP database. We will then perform a concurrent replay of both workloads.

NOTE! - Best Practices for Database replay is to perform SQL Performance Analyzer to validate that there are no

regression due to the implemented change. Since we have been working with SQL Performance Analyzer in

previous exercise then we will skip this part. The DW pluggable database in test has all tuning recommendations

implemented from previous exercise and we have also made sure that there are no regressions in the OLTP

pluggable database either.

Estimated Time to Complete Use Case: 30 minutes, recommended workflow

5. Capture Workload from Prod Database

5.1. You should already be logged on to Enterprise Manager. If you are not, please follow the instructions

detailed in the “Configure SPA Quick Check” of this workbook. You should have shutdown prod12 db and

started prod database at the end of the previous lab.

5.2. Navigate to Database Replay : From the menu, Enterprise -> Quality Management -> Database replay

Page 33 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

5.3. We have already performed capture for both DW and OLTP that you can use if short on time. However,

we’ll walk you through capture workflow to get familiar with the steps.

Click on Captured Workload

Click on Create

5.4. Check both boxes in Capture Prerequisites

Click Next

5.5. Click “Add”.

Page 34 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

5.6. Enter:

Capture Name : OLTP_CAPTURE_10m_XX (where XX are your initials)

Database Target : prod.oracle.com (use Search icon )

Select prod.oracle.com and click Select

Select Named Database credential: DB-SYS-GLOBAL

Select Named Database Host Credential: HOST-ORACLE (ORACLE)

Enter “Database Capture Storage Root Location, type in: /home/oracle/scripts/dbpack/RAT_CAP

Click OK.

Note: Typing the directory name seemed to work, due to security privileges on folders.

Page 35 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

5.7. Click Next.

5.8. By default all user activity is captured but EM monitoring will be present during replay as well so this

activity should be excluded. This will be done by default when starting capture from Enterprise Manager.

Click Next.

5.9. Change “Duration” to 10 minutes.

Click Next.

Page 36 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

5.10. Review the information.

Click Submit

5.11. It’s time for us to start the OLTP production workload.

Navigate to Enterprise -> Job -> Library

Page 37 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 5.12. Select job “DB_REPLAY_RUN_WORKLOAD”

Click Submit

5.13. Click Submit

5.14. The capture job has now started and will be running for about 10 minutes. You can review the status from

Database replay Home page. The page does not have an auto refresh so please refresh it manually. Note it

may take a few minutes to refresh the page.

When Capture status is changed to “Completed” in about 10 minutes, continue to the next step.

Page 38 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 5.15. We are now ready with capture from prod.

5.16. Next step is to replay the workload against test. To do this we need to stop prod.oracle.com and start

test.oracle.com. Go to your terminal in VNC

Page 39 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6. Concurrent Replay

6.1 You should already be logged on to Enterprise Manager. If you are not, please follow the instructions

detailed in the “Configure SPA Quick Check” of this workbook.

6.2 First we need to reset the test database for the replay

6.3 Go to Enterprise -> Job -> Library

6.4 Select “DB_REPLAY_SETUP”

Click Submit.

Page 40 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6.5 Click Submit

6.6 Navigate to Database Replay

From the Menu, Enterprise Quality Management Database replay

Page 41 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6.7 Go to Tab replay Task.

Click on Replay Task

Click Create

Page 42 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.8 Enter:

Name: Replay_For_Consolidation_XX (where XX should be your initials)

Select your created OLTP capture and the pre-created DW_CAPTURE_10m

Select “Copy workload to a new replay directory”

For each source workload in drop down menu make sure to use named credential:

“HOST-ORACLE(ORACLE)”

For Source Workload Directories:

Capture Name: select 'OLTP_CAPTURE_10m_XX' (where XX should be your initials)

Select Named Credential “HOST-ORACLE”

For Replay Directory:

Host: em12.oracle.com

Select Named Credential “HOST-ORACLE”

Enter “Consolidated Replay Directory”: /home/oracle/scripts/dbpack/RAT_REPLAY

Click Submit.

Note: Typing the directory name seemed to work, due to security privileges on folders.

Page 43 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6.9 Click on the newly created replay task

Page 44 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.10 Click ‘Create’ to create a replay.

Enter a name for the Replay

Select target database

If “test .oracle.com” is not preselected use following procedure to select the database:

Use to find the test database

In Target Name enter “test”

Click “Search”

Select “test.oracle.com”

Click “Select”

Page 45 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.11 Each workload is stored in our capture repository and is not currently preprocessed.

Click the icon for “Preprocess Workload”.

6.12 Select named Credential “DB-SYS-GLOBAL”

6.13 Check “Use an existing directory with Multiple workload subdirectories on this host””

Click Next

Page 46 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.14 Select Directory Object “RAT_REPLAY”

Click Next

6.15 Select: Credential Name: HOST-ORACLE(ORACLE)

Click Next

6.16 Click Submit

Page 47 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.17 A preprocess job has now been created. You can monitor the job by Right click on the job and open in

another tab. The job will take 2 – 3 minutes to complete.

When pre-processing has completed

Click the icon for “Replay Workload”

6.18 For Database Credential use named credential “DB-SYS-GLOBAL”

For Database Host Credential use named credential “HOST-ORACLE(ORACLE)”

Click “Next”

Page 48 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.19 Select “Use an existing directory object”

Select Directory Object: RAT_REPLAY

Click “Next”

6.20 Our replay is against the container database but our different workloads shall be replayed against

different pluggable databases. Therefore we need to remap our connections against each pluggable

database

Select “Capture Name”: DW_CAPTURE_10m

Change “SID=test” to “service_name=dw_test.oracle.com”

Click “Test Connection” and make sure you get

Page 49 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab Select “Capture Name”: OLTP_CAPTURE_10m_XX (where XX are your initials)

Change “SID=test” to “service_name=oltp.oracle.com”

Click “Test Connection” and make sure you get .

Click Next

6.21 Click “Add”

Page 50 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.22 Select

Host: em12.oracle.com

Host Credential:

Use Named: “HOST-ORACLE(ORACLE)”

Enter the following information: Replay Client Host Configuration:

Client Oracle Home: /u01/home/oracle/db12c

Client Replay Directory: /home/oracle/scripts/dbpack/RAT_REPLAY

Client Work Directory: /tmp

Database Credential

Use Named: “DB-SYSTEM-NORMAL”

Click “OK”

Page 51 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.23 Click “Next”

6.24 Click “Start Clients”

6.25 When client have been started click “Next”

Page 52 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.26 Click “Submit”

6.27 The replay has now started. If you want to monitor the replay

Click on the “Replay Task:…” breadcrumb

Page 53 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.28 Click on the Replay Name

6.29 During the replay you will be able to see the progress and Replay statistics like number of user calls, and

DBtime.

We can follow and compare “User Calls”, “Average Active Sessions” and Database time between capture

and replay

Turn on Auto Refresh and set it to “1 Minute”. Replay statistics will only be updated once a minute. This

page takes a few minutes to refresh in the beginning.

Page 54 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.30 When the replay has finished then we have the ability to generate a various types of reports for

performance analysis. We have also the ability to generate reports from the replay.

There is a section for Replay divergence summary where we can see the overall quality of the replay. In

general terms, a replay with less than 5% divergence in total should be seen as a very successful replay.

It is possible to drill down on Replay divergence where we can group errors or drill down to individual

calls.

Click on “Reports” tab

If you have time please review the replay report.

You can even produce ASH reports for a deeper analysis.

Page 55 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.31 Let’s view the Database Replay Report. Click on “View”

Page 56 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.32 This report contains an overview information of individual captures and replays

Page 57 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.33 Let’s view the Replay Compare report. Click on “View”

6.34 This report includes a more detailed analysis of our replays. First section is an overview.

Page 58 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab 6.35 Further down in the report we can see a more detailed analysis with breakdown on individual SQL

statements.

Page 59 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6.36 Top SQL statements divided by DB Time, CPU Time and Wait Time

6.37 We can also use Ash Analytics to generate Active reports. Active reports are HTML reports where we can

perform drilldown.

So let’s choose the first report. Select “Total Workload Service and Module by workload” Click “View”

Page 60 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

6.38 This report shows when we had a peak workload and which application generated the workload.

We can see that The Data warehouse application is very well tuned but there is a spike in the OLTP

application. As we can see we are actually exceeding the number of CPU’s during this peak and to be able

to consolidate these applications we should perform a deeper investigation. But it is out of the scope for

this exercise.

You have now learned how to investigate and troubleshoot performance issues by replaying and using the

find fix and validate methodology. It should be possible to solve these issues as well.

This is an active report so we can change the display type to load map which will give us another view and a more detailed picture of where the DBtime is spent.

Page 61 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

You have now finished the consolidated replay lab.

We have now seen how you can use Real Application Testing Database Replay to validate a database

consolidation using Database 12c pluggable databases. We have also seen the extensive reporting that

will help us to find and analyze bottlenecks or peaks during certain workloads. We hope that this has given

you a good overview on how to use the new consolidated replay feature.

Page 62 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Appendix A. How to start EM 12c?

This lab requires targets to be started by running the script em_start.sh. If you haven’t already, please make sure to run the em_start.sh script now. Refer steps in the lab cheat sheet. ssh or VNC to your VM as OS user oracle. The password for this account is oracle12. Once logged in open a terminal window and change directory to scripts $cd scripts

$./em_start.sh

Appendix B. How to Reset SPA Lab in Section B, C and D?

Go to Enterprise -> Job -> Library

Select “SPA_STATS_SETUP”

Page 63 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab Click Submit.

Click Submit

Page 64 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

Appendix C. Real Application Testing overview and Business Cases

A. SQL Performance Analyzer Optimizer Statistics

Business Case

When gathering new statistics it is not uncommon that the new statistics cause the optimizer to choose a new query plan. In most cases the new plan will be more efficient but sometimes it causes query regression. As a DBA it is important to proactively predict how new statistics will change the overall performance in the database. With SQL Performance Analyzer (SPA) you have the ability to execute most of your SQL statements that occur in your database. We have the ability to gather production SQL statements into SQL Tuning Sets either incrementally from Cursor Cache or AWR history for the period of time that we want to test. For the purpose of the lab, we have one pre-created SQL Tuning Set which is representative of the peak workload of this application. We have also gathered statistics in pending mode for validation. So, let’s see if the new statistics will change the performance for this application.

Workflow overview

In this exercise we will learn to use the SPA guided workflow for Optimizer Statistics validation, how to tune regressed statements and how to implement pending statistics on tables. This exercise will be done against a pluggable database, DW, in a container database ‘test’. The condition is that we have a warehouse based on 2 schemas ‘STAT1’ and ‘STAT2’. The warehouse has been populated and used for a couple of month and we have currently not updated the statistics. We want to make sure that when we publish new statistics then we would not be surprised with a performance regression. Some of the activities that are associated with this exercise are time consuming so we have performed those tasks in advance for lab purposes to save some time. The following task has already been executed in the setup of the environment:

Gather Pending Optimizer Statistics on schema STAT1 and STAT2 Steps that we are going to perform in this exercise:

Run SPA Optimizer Statistics workflow

Compare performance of current statistics and pending statistics

Use SQL Plan base lines to fix plan regressions

Run SPA Optimizer Statistics workflow validate implemented SQL Plan Base lines

Compare performance of current statistics and pending statistics + SQL Plan Baselines implemented

Implement (Publish) new statistics

Page 65 of 65

ORACLE ENTERPRISE MANAGER Cloud Control 12c Database Performance Management Lab

Lab

B. Consolidated Database Replay

Business Case

During consolidation testing phase it is important to know that the server will handle all workloads that shall be consolidated. On this front we have the Consolidated Database Replay feature that will help us to predict the workload behavior. We can test different scenarios to make sure that we are as much prepared as possible. Examples of scenarios to test could be:

Worst case scenario with simultaneous peaks.

Growing user activity

Consolidating more applications

In this use case we are going to consolidate two workloads that have been pre-captured on different databases and we are going to concurrently replay them in a Container database. For the purpose of the lab, we’ve already setup the test system and the Container database appropriately to the point-in-time of captures so that replay can be started.

Lab Overview

Objective:

The objective of this document is to provide high-level guidelines on new features associated with Real Application Testing in Oracle Enterprise Manager Cloud Control 12c.

To perform real-world testing of Oracle databases, by capturing production workloads and replaying them on test systems enables you to perform real-world testing quickly and accurately. This allows enterprises to assess the impact of any planned system change before deploying it on production reliably. Additionally, with Oracle Database 12c we have the Multitenant capability that will make it possible to consolidate application into different pluggable databases within one container database. This allows us to reduce the footprint of each database and reduce resource usage on the server.

In this lab we will learn how to perform multiple replays against the same database to validate how the new database and the server will handle the workload.