Oracle Platinum Services – Remote Patch Deployment › us › support › library › platinum-remote-patch... · Platinum Services remote patch installation include, but are not

Oracle Platinum Services–Remote Patch Installation

What to Expect

Updated: July 29, 2019 Page 2 of 30

Copyright © 2019, Oracle and/or its affiliates. All rights reserved.

Document Objective ..................................................................................................................................... 3

Overview ....................................................................................................................................................... 3

Oracle Platinum Services Remote Patch Installation Activities ................................................................... 4

Oracle Platinum Services Remote Patch Installation Roles ........................................................................ 14

Appendix I – Access Requirements ............................................................................................................ 16

Appendix II – Communication Plan ........................................................................................................... 17

Appendix III – Rolling vs. Non-Rolling Patching ...................................................................................... 19

Rolling .................................................................................................................................................... 19

Non-rolling .............................................................................................................................................. 19

Appendix IV - Scheduling and Patch Duration .......................................................................................... 20

Patch Scheduling ..................................................................................................................................... 20

Patch Duration ........................................................................................................................................ 20

Split Window Patching ........................................................................................................................... 21

Cancellations ........................................................................................................................................... 22

Appendix V – Example: Oracle Platinum Services Remote Patch Installation Process Flow.................... 23

Appendix VI – Example: Patching Schedule .............................................................................................. 24

Appendix VII - Creating a Platinum Patch Event Service Request ............................................................ 25

Appendix VIII – Example: Exadata Patch Plan (Non-Rolling) .................................................................. 26

Appendix IX – Roles and Responsibilities for Oracle Databases for SAP Applications............................ 29



Document Objective The objective of this document is to detail a sample list of activities that may be performed to complete a

remote patch installation under Oracle Platinum Services. The information included in this document is

for informational purposes only and is subject to change. This document is not binding on either party,

will not be deemed an agreement between the parties and does not amend and/ or modify the terms of any

order or agreement.

Overview

Remote Patch Installation: To receive remote patch installation services you must be a current Oracle Platinum Services customer.

Remote patch installation services are subject to the Oracle Platinum Services Technical Support Policy.

Please review the Oracle Platinum Services Technical Support Policy at

http://www.oracle.com/us/support/library/platinum-services-policies-1652886.pdf.

A list of Certified Platinum Configurations is available at

http://www.oracle.com/us/support/library/certified-platinum-configs-1652888.pdf.

The programs patched and the scope of the remote patching installations are available at

http://www.oracle.com/us/support/library/remote-quarterly-patching-scope-1652890.pdf

You may purchase additional patching services for a fee. Examples of items not covered by Oracle

Platinum Services remote patch installation include, but are not limited to:

Database Upgrades (i.e. 11.2.x to 12.1.x or 12.x to 18.x)

Patch Set Release Upgrades (i.e. 11.2.0.3 to 11.2.0.4) – See the Oracle Platinum Services:

Remote Patch Installation section in the Oracle Platinum Services Technical Support Policy for

details

Critical Patch Updates outside of bundle patches

Platinum Readiness patching

Reactive patching

Standalone one-off patches

For additional assistance, please contact [email protected].

http://www.oracle.com/us/support/library/platinum-services-policies-1652886.pdf

http://www.oracle.com/us/support/library/certified-platinum-configs-1652888.pdf



mailto:[email protected]



Oracle Platinum Services Remote Patch Installation Activities The picture below provides an example of Oracle’s process for remote patch installation.

Customer responsibility Oracle responsibility with customer involvement

Initiate Patch Event

• If applicable, see Appendix XI for self-scheduling instructions

• For all other scheduling, contact Oracle Support Services

• Open Service Request (SR) in My Oracle Support

• Enter initial information

Patch Assessment

• Number of Databases (Exadata, SuperCluster)

• Context

• One Off / Merge patches requirement

• Exachk

• Discuss and document in SR

• Virtualization Yes/no

• zscheck

Make Patch Plan

• Details for execution during patch event

• Agree on Date

• Document in SR

Patch Download and Pre-Checks

• Download patches

• Pre-check on space, potential HW errors etc ...

• Document in SR

Patch System

• Go/No Go

• Execution of Patch Plan

• Increase SR to Severity 1 situation (time critical)

• Progress communication in SR

• After initial verification back to Severity 2

Verify and Close Patch Event

• Verification

• Closure of SR

< -12 to -8 Weeks -4 Weeks -2 weeks -2 days Point 0 +1 day



The table below provides an example of remote patch installation activities, owners, and estimated

timelines associated with an Oracle Platinum Services remote patch installation. If an activity is not

completed, it may result in delays or cancellation of the remote patch installation. The information listed

below is for informational purposes only, may vary for the purpose of your remote patch installation and

is subject to change.

Phone conversation Platinum patch event Service Request (SR) update



Activity Who When

1. Build Customer Patch Profile

After Oracle Platinum Services is implemented, the Oracle

Patch Coordinator* will work with you to build a Customer

Patch Profile. Following initial creation, the Customer Patch

Profile will be reviewed/updated prior to each patch

installation. The Customer Patch Profile normally contains the

following information:

Key customer contact information

Desired frequency of remote patch installation

Patch installation type (rolling / non-rolling) (see

Appendix III)

Configuration of Certified Platinum Configuration

o Rack Type (Exadata; Exalogic; SuperCluster;

Zero Data Loss Recovery Appliance; ZFS

Storage Appliance; Private Cloud Appliance)

o Rack Size (1/8; ¼; ½; Full)

o Rack Name

o Rack Status (Production; Test; QA)

o Exalogic is physical or virtual

o Oracle Database hosted for SAP Applications Time between non-production and production patch

installation – sometimes referred to as “burn-in time.”

Whether or not Oracle DataGuard is configured

Party responsible for patching the Certified Platinum

Configuration (Oracle or the customer)

* For more information regarding the role of the Oracle Patch

Coordinator, please see the Oracle Platinum Services Remote Patch

Installation Roles section below.

Customer

Oracle

Both

Prior to first

patch

installation

2. Initiate Remote Patch Installation

If the self-scheduling tool applies to your system, follow the

instructions in Appendix XI – Patch Scheduling Reference

Table, for the appropriate method for your Engineered System.

For all other cases, you initiate the Oracle Platinum Services

remote patch installation by contacting Oracle. Oracle also

requests that you log a platinum patch event SR (the “Service

Request”) in My Oracle Support

For more detailed instructions on opening a patching SR, see:

Appendix VIII. The Oracle Patch Coordinator will assist with a

review of your Certified Platinum Configuration to help verify

that it continues to be a Certified Platinum Configuration per

the Platinum Technical Support Policy and will discuss patch

scheduling, roles, and responsibilities with you as appropriate.

Customer

Oracle

Both

Minimum

eight (8) to

twelve (12)

weeks

before

customer

wants (or

needs) to be

patched.

Up to four

(4) times per

year

http://support.oracle.com/




3. Meet to Discuss Patch Process & Tentative Dates

When applicable, the Oracle Patch Coordinator will discuss the

patch process with you, such as:

Associated timelines

Passwords as per the defined access requirement for

remote patch installation (see Appendix I)

The first date for the patch assessment discussion (step

5)

Party responsible for running the Exachk (prerequisite

for next step)

For Oracle SuperCluster specifically, the Image

Packaging System (IPS) repository must be set to local

in accordance with Oracle SuperCluster best practices.

See Oracle SuperCluster - Patching Best Practices For

The Quarterly Full Stack Download Patch (Doc ID

1569461.1)

For Oracle Database hosted for SAP Applications,

suggest scheduling patch event > 2 weeks after SAP

bundle patch released. Also, see Appendix IX for

roles and responsibilities diagram.

For Oracle ZFSSA log the output from Platinum ZFS

Check (Doc ID 2068328.1) and Basic ZS Check

(2046539.1)

See Appendix XI – Patch Scheduling Reference Table

for specific scheduling instructions

Customer

Oracle

Both

Four (4)

weeks prior

to patch

installation

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1569461.1&displayIndex=2




4. Conduct Patch Assessment

The Oracle Patch Coordinator will initiate an assessment of the

Certified Platinum Configuration to be patched (“Patch

Assessment Report”). The Patch Assessment Report helps

identify pertinent details of Certified Platinum Configuration,

such as the number of databases and Oracle_Homes (database

versions), one-off patches needed, current health of the

Certified Platinum Configuration, number of virtual servers,

etc. The Oracle Patch Engineer will upload the Patch

Assessment Report to the Service Request and will review the

Patch Assessment Report with you during the Patch

Deployment Requirements Meeting described below.

Please review the Oracle Platinum Services Remote

Patch Deployment Scope document for details on the

number of databases and Oracle_Homes covered under

Oracle Platinum Services. Patching for databases /

Oracle_Homes over the Platinum limitations can be

done by Oracle for a fee.

During the assessment of your Certified Platinum

Configuration, Oracle will help review the patches

current installed. If patch conflicts between your

currently installed patches and the remote installation

patch bundle are identified, a merge patch will be

created by Oracle and installed as part of the remote

patch installation. This information will be included in

the Patch Assessment Report.

Oracle Databases for SAP Applications: Oracle

Patch Engineer will look for some known SAP

unsupported configurations that may prevent Platinum

Services from patching the environment. A few

examples are role separation (see SAP note 1590515),

no SAP bundles installed, and hostname not properly

formulated (see SAP note SAP notes 611361 and

1996481).

Customer

Oracle

Both

Four (4)

weeks prior

to patch

installation





5. Meet to Discuss Patch Installation Requirements

This meeting will be scheduled by the Oracle Patch

Coordinator. During this meeting, you and the Oracle Patch

Coordinator will discuss topics, such as the ones listed below,

in preparation for the remote patch installation. Please find

specific scheduling instructions in Appendix XI – Patch

Scheduling Reference Table.

Review the Patch Assessment

o Discuss the actions needed for a healthy

Certified Platinum Configuration before

patching

o Discuss the one-off patches detected on the

Certified Platinum Configuration that need to

be installed again with the new patch bundle

o Discuss one-off patches that may need to be

installed during this patching window

Discuss open patching dates, as applicable, for

Exalogic, SuperCluster, Zero Data Loss Recovery

Appliance, Private Cloud Appliance, and ZFS Racked

Systems. For Exadata, refer to the My Oracle Support

document “How to Schedule Exadata Patching Request

using Oracle Advanced Support Gateway (OASG)

Portal for Platinum Customers” (DOC ID 2190010.1)

for information on how to schedule patching via the

self- service scheduling tool.

Review and, if applicable, update the Customer Patch

Profile

Review the scope of the remote patch installation

Review the remote patch installation process

Review your obligations (see Step 11)

Discuss the remote patch installation Communication

Plan (see Step 8 for more details) and certain

information that may be incorporated into the

Communication Plan such as:

o How to communicate whether patch

installation will proceed (“Go/ No Go”)

o Customer contacts to receive remote patching

installation progress

o Frequency of communication

o End of patching alert

o Who and how should be alerted on issues

(See Appendix II - Communication Plan)

Customer

Oracle

Both

Three - Four

(3-4) weeks

prior to

patch

installation




6. Confirm Patch Installation Dates

For patch requests created with the online tool, see Appendix

XI – Patch Scheduling Reference Table.

For all other requests, you will discuss remote patching

installation dates with the Oracle patch scheduling team during

the Patch Installation Requirements Meeting. Following that

meeting, the Oracle Patch Coordinator will confirm the

availability of the remote patching installation dates discussed

with you. If they are available, the Oracle Patch Coordinator

will provide confirmation back to you and will schedule the

remote patch installation. If the dates discussed with you are

not available, the Oracle Patch Coordinator will contact you to

discuss alternative dates. For scheduling and cancellation

details, see Appendix IV Scheduling and Patch Duration.

Customer

Oracle

Both

Two (2)

weeks prior

to patch

installation

7. Change Management Ticket

For patch requests created via the self-scheduling online tool,

Change Management tickets will be created automatically.

For all other patch requests, the Oracle Patch Coordinator will

open a Change Management ticket to stop monitoring the

Certified Platinum Configuration during the remote patch

installation period. Communication regarding the change

management will occur via the Service Request.

Customer

Oracle

Both

Two (2)

weeks prior

to patch

installation

8. Create Patch Plan

The Oracle Patch Engineer* will create a patch plan (“Patch

Plan”) and the associated communication plan

(“Communication Plan”) for the remote patch installation to the

Certified Platinum Configuration.

The Plans will be uploaded to the Service Request.

(See Appendix VII for an example patch schedule).

NOTE 1: For more information regarding the role of the Oracle

Patch Engineer, please see the Oracle Platinum Services

Remote Patch Installation Roles section below.

NOTE 2: Oracle databases for SAP Applications: You may

need to revise your schedule to accommodate the sequencing of

the patch plan created. The Oracle patch engineer will discuss

with you and Advanced Customer Support (ACS)—where

engaged—if required.

Customer

Oracle

Both

Two (2)

weeks prior

to patch

installation

9. Meet to Review Patch & Communication Plan

The Oracle Patch Coordinator will review the Patch Plan and

the Communication Plan with you.

Customer

Oracle

Both

One (1)

week prior

to patch

installation



C

t

u

a

l

10. Perform Pre-Check & Download

The Oracle Patch Engineer will perform final pre-checks on the

Certified Platinum Configuration the week before the remote

patch installation (e.g. hardware error checks, space availability

checks, review of recently discovered issues, etc.).

The patch bundles to be installed will be downloaded to the

Certified Platinum Configuration and file comparison (such as

checksum) will verify if the download was successful.

For ZFS Storage Appliance Racked Systems the Appliance Kit

Software will be downloaded and a HealthCheck will be

performed on the system to ensure there are no outstanding

issues to prevent the update from successfully installing.

The Oracle Patch Engineer will update the Service Request

with certain information from the pre-check and download.

Customer

Oracle

Both

In the week

before the

patch

installation

11. Perform Customer Prerequisite Activities

You must perform the following activities prior to the start of

the remote patch installation:

a) Perform full backups of the Certified Platinum

Configuration and components.

b) Confirm the successful backup of the Certified

Platinum Configuration and components.

c) Provide information on the location of the backups to

Oracle and ensure such information is included in the

Service Request.

d) If using Oracle DataGuard, ensure the primary and

standby databases are in sync.

Customer

Oracle

Both

In the week

before the

patch

installation

12. Give Approval to Start

You must give Oracle approval to begin the remote patch

installation by updating the Service Request.

Customer

Oracle

Both

Thirty (30)

minutes

before Patch

Installation



13. Deploy Patches

The Oracle Patch Engineer deploys the patches remotely and

will communicate information, if applicable, regarding the

remote patch installation in the Service Request.

Upon commencement of the remote patch installation, Oracle

will

Take ownership of the Service Request

Increase the Service Requests severity level to severity

1, non 24x7 to indicate time sensitivity

Set the outage-status according to the status of the

patching. (e.g.: Awaiting Customer Patching Approval,

Patching Started, Patching In Progress – On Schedule,

Patching In Progress – Delayed, Patching Completed –

Pending Review, Patching Completed, Patching

Canceled by Customer, Patching Partially Being Rolled

Back, Patching Completed – Pending Review, Patching

Completed)

Execute the remote patch installation

Communicate information , if applicable, regarding the

remote patch installation via the Service Request

Customer

Oracle

Both

Patch

Installation

14. Address Issues During Remote Patch Installation

If issues are encountered during the remote patch installation,

the Oracle Patch Engineer will update the Service Request to

notify you of the issue(s) and, as applicable, associated

activities.

The Oracle Patch Engineer will assist with the collection of

diagnostics related to the issue(s). The global Oracle Support

team will assist with the review and resolution of issue(s) per

Oracle’s technical support policies.

Customer

Oracle

Both

Patch

Installation

15. Signal & Escalate Issues During Patching

Should the remote patch installation not go as expected, or if

something must be addressed urgently during remote patch

installation, you can:

Call the local Oracle Support Center. Enter the Service

Request number to be transferred to the Oracle Patch

Engineer working on your remote patch installation.

Phone numbers may be found on the Oracle Support

Contacts Global Directory page.

To escalate the situation, use the Service Request

escalation process as documented in How to Escalate a

Service Request (SR) with Oracle Support Services

(DOC ID 199389.1)

Customer

Oracle

Both

Patch

Installation

http://www.oracle.com/us/support/contact-068555.html


https://mosemp.us.oracle.com/epmos/faces/ui/km/SearchDocDisplay.jspx?_afrLoop=476939908071421&recommended=true&type=DOCUMENT&id=199389.1&_afrWindowMode=0&_adf.ctrl-state=526qzm2px_178



16. Perform Post-Installation Testing & Verification

You are responsible for performing testing and validation of the

patched Certified Platinum Configuration.

If issues are identified during the planned verification phase

immediately following the remote patch installation, you may

use the Service Request to document them.

This Service Request will remain in severity 1 status until

Oracle receives feedback on the post-installation testing in the

first one (1) to two (2) hours after remote patch installation is

complete.

Post patching, the Oracle Patch Engineer will run Exachk and

document the output in the Service Request. In the cases where

you need to verify the system before the Oracle Engineer can

perform this step within the post-patching window, the Oracle

Patch Engineer will ask you to upload the Exachk output to the

Service Request.

If you do not document any issues in the Service Request, it

will be set to severity 2. If you detect and document issues in

the Service Request, Oracle Support Services will assist with

their resolution per Oracle’s technical support policies.

Customer

Oracle

Both

Post-Patch

Installation

17: Raise Post Installation Production Phase Issues

The Service Request will remain open until the next business

day after completion of the remote patch installation. The

Oracle Patch Coordinator will contact you to verify that there

are no issue(s) to be documented in the Service Request and to

close the Service Request.

If you encounter issues after the Service Request has been

closed, the regular Oracle Platinum Service Request process

will need to be used. The Platinum Patch Event Service

Request should be referred to in the newly created Service

Request.

Customer

Oracle

Both

Post-Patch

Installation



Oracle Platinum Services Remote Patch Installation Roles

Some sample role definitions for the remote patch installation are listed below:

Oracle Patch

Coordinator

Coordinates service delivery with customer for remote patch installation.

Documents information regarding the remote patch installation event in

the Service Request.

Schedules and hosts remote patch installation coordination calls and tracks

remote patch installation schedule.

Oracle Patch

Engineer

Performs technical aspects of the remote patch installation, for example

performs the technical patch assessment, creates a patch plan, completes

the pre-checks, downloads the patch bundles and performs the actual patch

installation.

Assists obtaining information for Oracle Support Services to review and

resolve issues, per Oracle’s technical support policies, that arise during the

remote patch installation.

Updates the Service Request as applicable during the remote patch

installation.



RASCI-Table

Highlights of Patching Responsibilities Except for the patching services included in Oracle Platinum Services as outlined above, you are

responsible for the installation of all other upgrades and patches to the Certified Platinum Configuration,

including software updates to non-critical components (i.e. power distribution units, Cisco switches etc.).

Remote patch installations do not include Oracle Database version upgrades (e.g., 12.x to 18.x). Grid

Infrastructure upgrades are not normally included in the remote patch installations described in this

document; however, you may request an Oracle Grid Infrastructure upgrade and, if Oracle agrees to

install the upgrade, it will count as a separate remote patch installation. Your total remote patch

installations for the year will be reduced accordingly.

All upgrades and patches to non-eligible configurations (non-certified Platinum configurations) are your

responsibility. When conflicts resulting from customer-added software are detected, Oracle will remove

the conflicting software in order to complete the customer-approved Platinum Services patch update. Post

patching, you will reinstall any software that was causing a conflict.



Appendix I – Access Requirements Remote patch installation cannot be completed without proper access to the Certified Platinum

Configuration, such as:

Patch Assessments

Patch Planning Pre-Check & Download

Patching

Root/sudo Access

o Database nodes Root Root/Sudo Root/Sudo Root/Sudo

o Cell nodes Root Root/Sudo Root/Sudo Root/Sudo

o InfiniBand Switches Root Root Root Root

o ZFSSA Root

Root/Sudo Root/Sudo

ILOM Root Access

o Database nodes Sudo Sudo

o Cell nodes Sudo Sudo

o ZFSSA Root

Root Root

DBSNMP *

Oracle Root Sudo Sudo

Orarom Sudo

* Your DBSNMP password is needed by Oracle to monitor your Certified Platinum Configuration per the Oracle Platinum Services policy. Note: Root access is only needed if the Patching team needs to run Exachk in the environment (including post Exachk), but is not needed if Exachk report(s) are uploaded to create patch assessment(s) Patch Planning and Pre-Check: orarom with sudo access is enough (ssh equivalence for root user should be configured across the nodes) Patching: orarom with sudo access is sufficient to perform patching and also for investigating issues with

the Engineered Systems Support team, although it is advisable to have root access because it is

unknown whether a customer has limited access for a sudo user.



Appendix II – Space Requirements Remote patch installation cannot be completed without meeting adequate space requirements for each

product/component of the Certified Platinum Configuration. Recommended space requirements are:

Component Exadata Exalogic Supercluster ZDLRA

YUM (bare metal configuration) /file system : 3.5 GB /u01 file system : 20 GB NA NA

/file system : 3.5 GB /u01 file system : 20 GB

YUM (Oracle VM X5/X6/X7/X8 conf.)

/file system : 3.5 GB /u01 file system : 12 GB Grid / Oracle home partitions being patched: 10 GB NA

Grid / Oracle home partitions being patched: 15 GB NA

CELL /file system : 3.5 GB NA /file system : 3.5 GB / file system : 3.5 GB

IB /file system : 80 MB /tmp file system : 120 MB

/file system : 80 MB /tmp file system : 120 MB



Solaris Domains & Zones NA NA /file system : 12GB NA

Physical - Compute nodes NA /file system : 4.2GB NA NA

Virtual - Compute nodes NA /file system : 1.1GB NA NA



Appendix III – Communication Plan During the remote patch installation process, from initiation until closure, communications will be

documented in the Service Request. When significant updates happen in the Service Request, an email

will be sent to the contact persons listed in the Service Request. Some customers have created a separate

mailing list to be used in the Platinum Patch Event Service Request (in the contact email id field) which

contains all the customer contacts who need to be informed when changes are made to the Service

Request. This helps makes the target group maintenance easy and reusable.

Telephone contact can be established at any time during the remote patch installation. Call your local

Oracle Support Center and enter your Service Request number. You will be transferred to the Oracle

Patch Engineer assisting with the remote patch installation. Phone numbers may be found on the Oracle

Support Contacts Global Directory page.

If you need management attention to your situation, you can escalate the Platinum Patch Event Service

Request. This is per the standard escalation process as documented in How to Escalate a Service Request

(SR) with Oracle Support Services (DOC ID 199389.1).

For quality control purposes, Oracle does not permit the use of bridge calls, web conferences or other

tools to follow engineers during the remote patch installation. Remote patch installation progress can be

monitored by customers through the Service Request.



https://mosemp.us.oracle.com/epmos/faces/ui/km/SearchDocDisplay.jspx?_afrLoop=476939908071421&recommended=true&type=DOCUMENT&id=199389.1&_afrWindowMode=0&_adf.ctrl-state=526qzm2px_178



Appendix IV – Rolling vs. Non-Rolling Patching The quarterly Full Stack Download Patch contains patches for each layer of Engineered System that

require patching, each of which can be applied in a rolling or non-rolling fashion*. The decision of

whether to patch in a rolling or non-rolling fashion is determined by you.

* For Zero Data Loss Recovery Appliance, patching is applied in a non-rolling fashion ONLY.

* For ZFS Storage Appliance, patching is applied in a rolling fashion ONLY. Resources transition

between the two controller nodes and are available to clients throughout the patching cycle.

Rolling Rolling patch application refers to patching where Oracle will remotely perform the patch installation

while your Certified Platinum Configuration is online. Patches will be installed on each application,

server, storage server, or storage array covered under Platinum Services. To minimize downtime, patches

will be installed one server at a time, in a rolling fashion.

For example, with Exadata Storage Cell patching, patches can be applied one cell node at a time. Each

cell node is taken offline individually and patched while the other cell nodes remain available.

Storage Cell

Advantages:

o Does not require database downtime

Disadvantages:

o Requires Automatic Storage Management (ASM) high redundancy to reduce the risk of

disk failure.

o Length of time to patch.

Database Node

Advantages:

o Does not require database-wide downtime. Depending on configuration, there will be

minimal downtime as instances are shut down on the node being patched

Disadvantages:

o Must be on Oracle Database 11.2.0.2 Bundle Patch 2 or higher

Non-rolling

Non-rolling patch application—also known as all node patch application—refers to patching where

Oracle will remotely perform the patch installation while your Certified Platinum Configuration is offline.

Patches and updates will be performed in parallel, when technically feasible, to complete the update as

quickly as possible.

In some cases, the Certified Platinum Configuration may need to be shutdown, the hardware system may

have to be rebooted, and the Certified Platinum Configuration may have to be brought back up in order to

complete the patch installation successfully. Note that some components cannot be patched in parallel.

Applying the Exadata Storage Cell example to a non-rolling patch scenario:



Storage Cell

Advantages:

o Requires least amount of downtime to patch.

o Eliminates the risk of a single disk failure.

Disadvantage:

o Does require database downtime.

Database Node

Advantages:

o Requires least amount of downtime to patch.

Disadvantage:

o Does require database downtime.

Appendix V - Scheduling and Patch Duration

Patch Scheduling

Oracle suggests that a remote patch installation be scheduled eight (8) to twelve (12) weeks before the

date the patch is to be installed.

Many customers know their maintenance windows well in advance. Some even plan them more than a

year ahead. It is important to share those dates with the Oracle Patch Coordinator.

Patch Duration

When planning for a remote patch installation, you need to take into account the time a remote patch

installation will take. This is dependent on size, type, method, patching needs, etc.

The table below shows example timing for remote patch installations in rolling and non-rolling fashion.

These timings represent estimates only and not the actual time a customer will experience. Note for

rolling remote patch installations, the times may be split over multiple shifts.

Exadata

Rack Size Non-Rolling Rolling

Quarter 8 – 12 Hours 24 Hours

Half 12 – 16 Hours 3 Days

Full 16 – 24 Hours 6 Days

Exalogic

Includes physical and virtual Patch Set Update (PSU) upgrades as well as upgrading the components of

Exalogic: switch, storage and compute nodes.


Eighth 6-8 hours 14 hours

Quarter 8-10 hours 16 hours

Half 10-12 hours 24 hours

Full 16-20 hours 38 hours



SuperCluster


Half 10-14 hours 2 days

Full 12-16 hours 5 days

Split Window Patching Oracle understands the customer business requirements may dictate limited patching windows such that

system patching must be split into multiple windows over time.

A Platinum patch event will allow for this split window scheduling for consecutive days between Sunday

9PM EST and Friday 9AM EST and is subject to the following guidelines independent of rolling or non-

rolling:

- Cell storage patching does not impact the databases, therefore patching can be continuous. But, if

customer requires a split window for cell storage patching, a minimum of 9 hours is required per

window.

- All other components require a minimum of 6 hour windows per node

Example of Split Window Scheduling

Consider example optimal timing for each component below. Note: all components may not be part of

one patch event:

Component Hours

Yum upgrade 2.5 (per compute node)

Bundle Patch 2 (per compute node)

Cell Patching 3.5 (per storage node)

OJVM + cat bundles 2

Switches .75 per switch

Grid Infrastructure 2.5

New home installation 1

Database maintenance upgrade 2.5 hours (per database)

*Split window patching is not available for ZFS Storage Appliance.



Rolling Back Patches In general, each of the components may be patched independently; therefore, one component can be

patched without the requirement to patch any of the other components. This means, unless otherwise

directed by the product engineering team, once a component is patched successfully across all nodes, that

component will not be rolled back.

Cancellations A scheduled remote patch installation may need to be cancelled. If canceled by either party, Oracle will

work with you to find a new date that fits both parties’ schedules.

For patching requests created via the online self-scheduling tool, customers can reschedule new dates

using available time slots. For more information, see Appendix XI – Patch Scheduling Reference Table.



Appendix VI – Example: Oracle Platinum Services Remote Patch

Installation Process Flow



Appendix VII – Example: Patching Schedule The below example lays out the annual schedule for patching four (4) racks over a two (2) week interval

every six (6) months.

Month 1 Month 2 Month 3 Month 4 wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12 wk13 wk14 wk15 wk16 R1 R2

CPC1 R1

CPC2 R1

CPC3 R1

CPC4 R1

Month 5 Month 6 Month 7 Month 8

wk17 wk18 wk19 wk20 wk21 wk22 wk23 wk24 wk25 wk26 wk27 wk28 wk29 wk30 wk31 wk32 R1 R3

CPC1 R3

CPC2 R3

CPC3 R3

Month 9 Month 10 Month 11

wk33 wk34 wk35 wk36 wk37 wk38 wk39 wk40 wk41 wk42 wk43 wk44 R2 CPC4 R3

Release of QFSDB (R1 = Release 1 of QFSDB)

Patching of Certified Platinum Configuration (R1=Release 1 of QFSDB) (CPC1=Certified Platinum Configuration 1)

Last Date on Release T-6 months



Appendix VIII - Creating a Platinum Patch Event Service Request

1. Go to My Oracle Support and login with your Username and Password.

2. From the Systems tab, in the Systems region, find the Platinum-certified system you are looking

for and click on the System Name to drill down into the System.

NOTE: Platinum-certified systems will be indicated as such in the Level field.

3. Click Create SR.

This will take you to the Configuration tab of the Create Service Request page. The System

Name will be populated in the System field.

4. In the Host field, select or search/select the appropriate Host.

Once you select a Host, the Host details (Serial Number, Asset Name, and Organization) will be

returned and the Product and Operating System/Version fields presented.

5. For Product, select “Oracle Platinum Services” and select appropriate Operating System/Version.

6. For Problem Type, select “Patching Request”.

7. Continue with the remaining steps in the Service Request creation process ensuring all mandatory

fields are complete.

8. Click the Submit button. A Service Request will be created with System and Host information

attached.




Appendix IX – Example: Exadata Patch Plan (Non-Rolling)

1. Overview and Notes:

1.1. Patches to be applied for the Exadata Rack:

14522699: 11.2.3.2.1 for storage servers

16432033: 11.2.3.2.1 base repository iso for database nodes (via YUM)

17395890: 11.2.0.3.21 Bundle Patch for Oracle databases

1.2. Environment Details:

1.2.1. Customer Name : XYZ

1.2.2. Rack Type : Half-rack

1.2.3. Database Servers Names : db01 – db04

1.2.4. Exadata cell node Names : cel01 – cel07

1.2.5. Oracle_Home : /u01/app/oracle/product/11.2.0.3/dbhome_1

1.2.6. Grid_Home : /u01/app/11.2.0.3/grid

1.2.7. Databases : dbm1, prod1, dev1

1.2.8. OPatch Version : 11.2.0.3.4

1.3. Software Locations:

/u01/patches/cell

/u01/patches/YUM

/u01/patches/BP21

2. Pre-checks Storage Software on Cell nodes:

2.1.1. Check ILOM access to cell nodes

2.1.2. Check ILOM access to database nodes

2.1.3. Check for clean file systems on cells and the database hosts

2.1.4. Ensure the various cell network configurations are consistent

2.1.5. Validate cell disks for valid physicalInsertTime

2.1.6. Check disk repair time set to default 3.6 Hrs

2.1.7. Check if all disk groups mounted by all ASM instances

2.1.8. Address if any potential problems found with Infiniband switch

2.1.9. Prepare cell_group, dbs_group files

2.1.10. Check for existing root SSH equivalence

2.1.11. Check the output of /usr/local/bin/imageinfo on all cell nodes

2.1.12. Copy the patch software from CTA to database node 1

2.1.13. Ensure customer has a file system and database backups

3. Applying Storage Software to Exadata Cells:

3.1. General notes on patching and monitoring

3.2. Stopping Services:

3.2.1.1. Stopping all services and agents in db nodes 3.2.1.2. Stopping services in cell nodes

3.3. Patchmgr Utility

3.4. Start a screen session from the CTA and ssh to database node 1

3.4.1.1. Run cleanup command

3.4.1.2. Start the prerequisites check

3.4.1.3. Start the patch in all cell nodes



3.4.1.4. Monitor the patch progress

3.5. Checking patch status after a successful operation (to be performed on all cells)

3.6. Clean up the cells using the -cleanup

3.7. Check the InfiniBand switch software and opensm configuration

4. Post-checks Storage Software on Cell nodes:

4.1. Restore NFS file systems

4.2. Bring up CRS, ASM, DB Instances

5. Pre-checks Yum update on Database nodes:

5.1. Check image version

5.2. Stop services

6. Applying Yum update to Database nodes:

6.1. Upgrade automatically using dbnodeupdate.sh*

*manual steps, if needed, are included in Appendix A

6.1.1.1. Patch with dbnodeupdate

6.1.1.2. Restart services

6.2. Repeat step 7 for all other database nodes. <<<This can be done in parallel for non-

rolling

7. Post-checks Yum update on Database nodes:

7.1. Check image version

8. Pre-checks Bundle Patch on database nodes:

8.1. Unzipping the Quarterly Database Patch for Exadata (11.2.0.3.21)

8.2. OCM Configuration:

8.3. Validation of Oracle Inventory:

8.4. Run OPatch Conflict Check:

8.5. Run OPatch System Space Check:

8.6. Stop all agents running on database nodes

8.7. Check the running resources

9. Applying Bundle Patch to database nodes:

9.1. Upgrade automatically using opatch auto*

*manual steps, if needed, are included in Appendix B

9.2. Repeat step 10 for all other database nodes. This can be done in parallel for non-rolling

10. Post-checks Bundle Patch on database nodes:

10.1. Check for streams

10.2. Run catbundle, utlrp

10.3. Review catbundle logs

10.4. Check for invalid objects

10.5. Patch Apply Crosscheck:

10.5.1.1. opatch lsinventory

10.5.1.2. Verify the catbundle apply

10.5.1.3. Check the running resources



A. Appendix A: Applying Yum update to Database nodes (Manual Steps, if needed)

A.1. Download required patches

A.2. Review the patch README

A.3. Performing One-Time Setup on Database Servers

A.4. Mount ISO on compute node

A.5. Execute bootstrap.sh to perform the initial phase of One-Time Setup on Database Servers

A.6. After the system reboot completes, continue on with the final steps of One-Time Setup

A.7. Relink the Oracle software components

A.8. Check the agents and mounts

B. Appendix B: Applying Bundle Patch to Database nodes (Manual Steps, if needed)

B.1. export TMOUT=18000

B.2. Stop the CRS

B.3. Run the pre root script:

B.4. Apply the Quarterly Database Patch (11.2.0.3.21) patch to GI Home

B.5. Run the pre script for DB component of the patch

B.6. Apply the Quarterly Database Patch (11.2.0.3.21) patch to DB Home

B.7. Run the post script for DB component of the patch

B.8. Run the post root script

C. Appendix C: Rollback

C.1. Rollback procedure for storage software

C.2. Rollback procedure for YUM update

C.3. Rollback procedure for bundle patch



Appendix X – Roles and Responsibilities for Oracle Databases for

SAP Applications

In accordance with the Oracle Platinum Technical Support Policy, Oracle Platinum Services includes

limited patching for Oracle Database hosted environments.

Out of scope components may be patched by the customer or the customer may purchase the branded

service “Oracle Engineered Systems Incremental Patch Deployment for Oracle Platinum Services”

directly from ACS. If contracted, ACS will coordinate directly with Platinum Services to assist with

patching of the customer’s engineered systems.

Below is a diagram of example responsibilities for a patch event:




Appendix XI – Patch Scheduling Reference Table

System Patching Request Scheduling Method Cancellation Method

Exadata Use the self-scheduling tool to

create a new request

The tool automatically creates a

Change Management ticket

Use the self-scheduling tool to

remove the existing patching

request

Create a new request based on

the available slots

Exadata Split event Contact Oracle Support Services

Discuss tentative dates with your Patch

Coordinator

Open Service a Request (SR) in My

Oracle Support

Oracle will work with customers to

find a new mutually convenient date, if

patching is canceled by either party Exadata IPSEC

Exalogic

SuperCluster

ZDLRA

ZFSSA




