Oracle Platinum Services–Remote Patch Installation
What to Expect
Updated: July 29, 2019 Page 2 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Document Objective ..................................................................................................................................... 3
Overview ....................................................................................................................................................... 3
Oracle Platinum Services Remote Patch Installation Activities ................................................................... 4
Oracle Platinum Services Remote Patch Installation Roles ........................................................................ 14
Appendix I – Access Requirements ............................................................................................................ 16
Appendix II – Communication Plan ........................................................................................................... 17
Appendix III – Rolling vs. Non-Rolling Patching ...................................................................................... 19
Rolling .................................................................................................................................................... 19
Non-rolling .............................................................................................................................................. 19
Appendix IV - Scheduling and Patch Duration .......................................................................................... 20
Patch Scheduling ..................................................................................................................................... 20
Patch Duration ........................................................................................................................................ 20
Split Window Patching ........................................................................................................................... 21
Cancellations ........................................................................................................................................... 22
Appendix V – Example: Oracle Platinum Services Remote Patch Installation Process Flow.................... 23
Appendix VI – Example: Patching Schedule .............................................................................................. 24
Appendix VII - Creating a Platinum Patch Event Service Request ............................................................ 25
Appendix VIII – Example: Exadata Patch Plan (Non-Rolling) .................................................................. 26
Appendix IX – Roles and Responsibilities for Oracle Databases for SAP Applications............................ 29
Updated: July 29, 2019 Page 3 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Document Objective The objective of this document is to detail a sample list of activities that may be performed to complete a
remote patch installation under Oracle Platinum Services. The information included in this document is
for informational purposes only and is subject to change. This document is not binding on either party,
will not be deemed an agreement between the parties and does not amend and/ or modify the terms of any
order or agreement.
Overview
Remote Patch Installation: To receive remote patch installation services you must be a current Oracle Platinum Services customer.
Remote patch installation services are subject to the Oracle Platinum Services Technical Support Policy.
Please review the Oracle Platinum Services Technical Support Policy at
http://www.oracle.com/us/support/library/platinum-services-policies-1652886.pdf.
A list of Certified Platinum Configurations is available at
http://www.oracle.com/us/support/library/certified-platinum-configs-1652888.pdf.
The programs patched and the scope of the remote patching installations are available at
http://www.oracle.com/us/support/library/remote-quarterly-patching-scope-1652890.pdf
You may purchase additional patching services for a fee. Examples of items not covered by Oracle
Platinum Services remote patch installation include, but are not limited to:
Database Upgrades (i.e. 11.2.x to 12.1.x or 12.x to 18.x)
Patch Set Release Upgrades (i.e. 11.2.0.3 to 11.2.0.4) – See the Oracle Platinum Services:
Remote Patch Installation section in the Oracle Platinum Services Technical Support Policy for
details
Critical Patch Updates outside of bundle patches
Platinum Readiness patching
Reactive patching
Standalone one-off patches
For additional assistance, please contact [email protected].
Updated: July 29, 2019 Page 4 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Oracle Platinum Services Remote Patch Installation Activities The picture below provides an example of Oracle’s process for remote patch installation.
Customer responsibility Oracle responsibility with customer involvement
Initiate Patch Event
• If applicable, see Appendix XI for self-scheduling instructions
• For all other scheduling, contact Oracle Support Services
• Open Service Request (SR) in My Oracle Support
• Enter initial information
Patch Assessment
• Number of Databases (Exadata, SuperCluster)
• Context
• One Off / Merge patches requirement
• Exachk
• Discuss and document in SR
• Virtualization Yes/no
• zscheck
Make Patch Plan
• Details for execution during patch event
• Agree on Date
• Document in SR
Patch Download and Pre-Checks
• Download patches
• Pre-check on space, potential HW errors etc ...
• Document in SR
Patch System
• Go/No Go
• Execution of Patch Plan
• Increase SR to Severity 1 situation (time critical)
• Progress communication in SR
• After initial verification back to Severity 2
Verify and Close Patch Event
• Verification
• Closure of SR
< -12 to -8 Weeks -4 Weeks -2 weeks -2 days Point 0 +1 day
Updated: July 29, 2019 Page 5 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
The table below provides an example of remote patch installation activities, owners, and estimated
timelines associated with an Oracle Platinum Services remote patch installation. If an activity is not
completed, it may result in delays or cancellation of the remote patch installation. The information listed
below is for informational purposes only, may vary for the purpose of your remote patch installation and
is subject to change.
Phone conversation Platinum patch event Service Request (SR) update
Updated: July 29, 2019 Page 6 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Activity Who When
1. Build Customer Patch Profile
After Oracle Platinum Services is implemented, the Oracle
Patch Coordinator* will work with you to build a Customer
Patch Profile. Following initial creation, the Customer Patch
Profile will be reviewed/updated prior to each patch
installation. The Customer Patch Profile normally contains the
following information:
Key customer contact information
Desired frequency of remote patch installation
Patch installation type (rolling / non-rolling) (see
Appendix III)
Configuration of Certified Platinum Configuration
o Rack Type (Exadata; Exalogic; SuperCluster;
Zero Data Loss Recovery Appliance; ZFS
Storage Appliance; Private Cloud Appliance)
o Rack Size (1/8; ¼; ½; Full)
o Rack Name
o Rack Status (Production; Test; QA)
o Exalogic is physical or virtual
o Oracle Database hosted for SAP Applications Time between non-production and production patch
installation – sometimes referred to as “burn-in time.”
Whether or not Oracle DataGuard is configured
Party responsible for patching the Certified Platinum
Configuration (Oracle or the customer)
* For more information regarding the role of the Oracle Patch
Coordinator, please see the Oracle Platinum Services Remote Patch
Installation Roles section below.
Customer
Oracle
Both
Prior to first
patch
installation
2. Initiate Remote Patch Installation
If the self-scheduling tool applies to your system, follow the
instructions in Appendix XI – Patch Scheduling Reference
Table, for the appropriate method for your Engineered System.
For all other cases, you initiate the Oracle Platinum Services
remote patch installation by contacting Oracle. Oracle also
requests that you log a platinum patch event SR (the “Service
Request”) in My Oracle Support
For more detailed instructions on opening a patching SR, see:
Appendix VIII. The Oracle Patch Coordinator will assist with a
review of your Certified Platinum Configuration to help verify
that it continues to be a Certified Platinum Configuration per
the Platinum Technical Support Policy and will discuss patch
scheduling, roles, and responsibilities with you as appropriate.
Customer
Oracle
Both
Minimum
eight (8) to
twelve (12)
weeks
before
customer
wants (or
needs) to be
patched.
Up to four
(4) times per
year
Updated: July 29, 2019 Page 7 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
3. Meet to Discuss Patch Process & Tentative Dates
When applicable, the Oracle Patch Coordinator will discuss the
patch process with you, such as:
Associated timelines
Passwords as per the defined access requirement for
remote patch installation (see Appendix I)
The first date for the patch assessment discussion (step
5)
Party responsible for running the Exachk (prerequisite
for next step)
For Oracle SuperCluster specifically, the Image
Packaging System (IPS) repository must be set to local
in accordance with Oracle SuperCluster best practices.
See Oracle SuperCluster - Patching Best Practices For
The Quarterly Full Stack Download Patch (Doc ID
1569461.1)
For Oracle Database hosted for SAP Applications,
suggest scheduling patch event > 2 weeks after SAP
bundle patch released. Also, see Appendix IX for
roles and responsibilities diagram.
For Oracle ZFSSA log the output from Platinum ZFS
Check (Doc ID 2068328.1) and Basic ZS Check
(2046539.1)
See Appendix XI – Patch Scheduling Reference Table
for specific scheduling instructions
Customer
Oracle
Both
Four (4)
weeks prior
to patch
installation
Updated: July 29, 2019 Page 8 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
4. Conduct Patch Assessment
The Oracle Patch Coordinator will initiate an assessment of the
Certified Platinum Configuration to be patched (“Patch
Assessment Report”). The Patch Assessment Report helps
identify pertinent details of Certified Platinum Configuration,
such as the number of databases and Oracle_Homes (database
versions), one-off patches needed, current health of the
Certified Platinum Configuration, number of virtual servers,
etc. The Oracle Patch Engineer will upload the Patch
Assessment Report to the Service Request and will review the
Patch Assessment Report with you during the Patch
Deployment Requirements Meeting described below.
Please review the Oracle Platinum Services Remote
Patch Deployment Scope document for details on the
number of databases and Oracle_Homes covered under
Oracle Platinum Services. Patching for databases /
Oracle_Homes over the Platinum limitations can be
done by Oracle for a fee.
During the assessment of your Certified Platinum
Configuration, Oracle will help review the patches
current installed. If patch conflicts between your
currently installed patches and the remote installation
patch bundle are identified, a merge patch will be
created by Oracle and installed as part of the remote
patch installation. This information will be included in
the Patch Assessment Report.
Oracle Databases for SAP Applications: Oracle
Patch Engineer will look for some known SAP
unsupported configurations that may prevent Platinum
Services from patching the environment. A few
examples are role separation (see SAP note 1590515),
no SAP bundles installed, and hostname not properly
formulated (see SAP note SAP notes 611361 and
1996481).
Customer
Oracle
Both
Four (4)
weeks prior
to patch
installation
Updated: July 29, 2019 Page 9 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
5. Meet to Discuss Patch Installation Requirements
This meeting will be scheduled by the Oracle Patch
Coordinator. During this meeting, you and the Oracle Patch
Coordinator will discuss topics, such as the ones listed below,
in preparation for the remote patch installation. Please find
specific scheduling instructions in Appendix XI – Patch
Scheduling Reference Table.
Review the Patch Assessment
o Discuss the actions needed for a healthy
Certified Platinum Configuration before
patching
o Discuss the one-off patches detected on the
Certified Platinum Configuration that need to
be installed again with the new patch bundle
o Discuss one-off patches that may need to be
installed during this patching window
Discuss open patching dates, as applicable, for
Exalogic, SuperCluster, Zero Data Loss Recovery
Appliance, Private Cloud Appliance, and ZFS Racked
Systems. For Exadata, refer to the My Oracle Support
document “How to Schedule Exadata Patching Request
using Oracle Advanced Support Gateway (OASG)
Portal for Platinum Customers” (DOC ID 2190010.1)
for information on how to schedule patching via the
self- service scheduling tool.
Review and, if applicable, update the Customer Patch
Profile
Review the scope of the remote patch installation
Review the remote patch installation process
Review your obligations (see Step 11)
Discuss the remote patch installation Communication
Plan (see Step 8 for more details) and certain
information that may be incorporated into the
Communication Plan such as:
o How to communicate whether patch
installation will proceed (“Go/ No Go”)
o Customer contacts to receive remote patching
installation progress
o Frequency of communication
o End of patching alert
o Who and how should be alerted on issues
(See Appendix II - Communication Plan)
Customer
Oracle
Both
Three - Four
(3-4) weeks
prior to
patch
installation
Updated: July 29, 2019 Page 10 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
6. Confirm Patch Installation Dates
For patch requests created with the online tool, see Appendix
XI – Patch Scheduling Reference Table.
For all other requests, you will discuss remote patching
installation dates with the Oracle patch scheduling team during
the Patch Installation Requirements Meeting. Following that
meeting, the Oracle Patch Coordinator will confirm the
availability of the remote patching installation dates discussed
with you. If they are available, the Oracle Patch Coordinator
will provide confirmation back to you and will schedule the
remote patch installation. If the dates discussed with you are
not available, the Oracle Patch Coordinator will contact you to
discuss alternative dates. For scheduling and cancellation
details, see Appendix IV Scheduling and Patch Duration.
Customer
Oracle
Both
Two (2)
weeks prior
to patch
installation
7. Change Management Ticket
For patch requests created via the self-scheduling online tool,
Change Management tickets will be created automatically.
For all other patch requests, the Oracle Patch Coordinator will
open a Change Management ticket to stop monitoring the
Certified Platinum Configuration during the remote patch
installation period. Communication regarding the change
management will occur via the Service Request.
Customer
Oracle
Both
Two (2)
weeks prior
to patch
installation
8. Create Patch Plan
The Oracle Patch Engineer* will create a patch plan (“Patch
Plan”) and the associated communication plan
(“Communication Plan”) for the remote patch installation to the
Certified Platinum Configuration.
The Plans will be uploaded to the Service Request.
(See Appendix VII for an example patch schedule).
NOTE 1: For more information regarding the role of the Oracle
Patch Engineer, please see the Oracle Platinum Services
Remote Patch Installation Roles section below.
NOTE 2: Oracle databases for SAP Applications: You may
need to revise your schedule to accommodate the sequencing of
the patch plan created. The Oracle patch engineer will discuss
with you and Advanced Customer Support (ACS)—where
engaged—if required.
Customer
Oracle
Both
Two (2)
weeks prior
to patch
installation
9. Meet to Review Patch & Communication Plan
The Oracle Patch Coordinator will review the Patch Plan and
the Communication Plan with you.
Customer
Oracle
Both
One (1)
week prior
to patch
installation
Updated: July 29, 2019 Page 11 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
C
t
u
a
l
10. Perform Pre-Check & Download
The Oracle Patch Engineer will perform final pre-checks on the
Certified Platinum Configuration the week before the remote
patch installation (e.g. hardware error checks, space availability
checks, review of recently discovered issues, etc.).
The patch bundles to be installed will be downloaded to the
Certified Platinum Configuration and file comparison (such as
checksum) will verify if the download was successful.
For ZFS Storage Appliance Racked Systems the Appliance Kit
Software will be downloaded and a HealthCheck will be
performed on the system to ensure there are no outstanding
issues to prevent the update from successfully installing.
The Oracle Patch Engineer will update the Service Request
with certain information from the pre-check and download.
Customer
Oracle
Both
In the week
before the
patch
installation
11. Perform Customer Prerequisite Activities
You must perform the following activities prior to the start of
the remote patch installation:
a) Perform full backups of the Certified Platinum
Configuration and components.
b) Confirm the successful backup of the Certified
Platinum Configuration and components.
c) Provide information on the location of the backups to
Oracle and ensure such information is included in the
Service Request.
d) If using Oracle DataGuard, ensure the primary and
standby databases are in sync.
Customer
Oracle
Both
In the week
before the
patch
installation
12. Give Approval to Start
You must give Oracle approval to begin the remote patch
installation by updating the Service Request.
Customer
Oracle
Both
Thirty (30)
minutes
before Patch
Installation
Updated: July 29, 2019 Page 12 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
13. Deploy Patches
The Oracle Patch Engineer deploys the patches remotely and
will communicate information, if applicable, regarding the
remote patch installation in the Service Request.
Upon commencement of the remote patch installation, Oracle
will
Take ownership of the Service Request
Increase the Service Requests severity level to severity
1, non 24x7 to indicate time sensitivity
Set the outage-status according to the status of the
patching. (e.g.: Awaiting Customer Patching Approval,
Patching Started, Patching In Progress – On Schedule,
Patching In Progress – Delayed, Patching Completed –
Pending Review, Patching Completed, Patching
Canceled by Customer, Patching Partially Being Rolled
Back, Patching Completed – Pending Review, Patching
Completed)
Execute the remote patch installation
Communicate information , if applicable, regarding the
remote patch installation via the Service Request
Customer
Oracle
Both
Patch
Installation
14. Address Issues During Remote Patch Installation
If issues are encountered during the remote patch installation,
the Oracle Patch Engineer will update the Service Request to
notify you of the issue(s) and, as applicable, associated
activities.
The Oracle Patch Engineer will assist with the collection of
diagnostics related to the issue(s). The global Oracle Support
team will assist with the review and resolution of issue(s) per
Oracle’s technical support policies.
Customer
Oracle
Both
Patch
Installation
15. Signal & Escalate Issues During Patching
Should the remote patch installation not go as expected, or if
something must be addressed urgently during remote patch
installation, you can:
Call the local Oracle Support Center. Enter the Service
Request number to be transferred to the Oracle Patch
Engineer working on your remote patch installation.
Phone numbers may be found on the Oracle Support
Contacts Global Directory page.
To escalate the situation, use the Service Request
escalation process as documented in How to Escalate a
Service Request (SR) with Oracle Support Services
(DOC ID 199389.1)
Customer
Oracle
Both
Patch
Installation
Updated: July 29, 2019 Page 13 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
16. Perform Post-Installation Testing & Verification
You are responsible for performing testing and validation of the
patched Certified Platinum Configuration.
If issues are identified during the planned verification phase
immediately following the remote patch installation, you may
use the Service Request to document them.
This Service Request will remain in severity 1 status until
Oracle receives feedback on the post-installation testing in the
first one (1) to two (2) hours after remote patch installation is
complete.
Post patching, the Oracle Patch Engineer will run Exachk and
document the output in the Service Request. In the cases where
you need to verify the system before the Oracle Engineer can
perform this step within the post-patching window, the Oracle
Patch Engineer will ask you to upload the Exachk output to the
Service Request.
If you do not document any issues in the Service Request, it
will be set to severity 2. If you detect and document issues in
the Service Request, Oracle Support Services will assist with
their resolution per Oracle’s technical support policies.
Customer
Oracle
Both
Post-Patch
Installation
17: Raise Post Installation Production Phase Issues
The Service Request will remain open until the next business
day after completion of the remote patch installation. The
Oracle Patch Coordinator will contact you to verify that there
are no issue(s) to be documented in the Service Request and to
close the Service Request.
If you encounter issues after the Service Request has been
closed, the regular Oracle Platinum Service Request process
will need to be used. The Platinum Patch Event Service
Request should be referred to in the newly created Service
Request.
Customer
Oracle
Both
Post-Patch
Installation
Updated: July 29, 2019 Page 14 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Oracle Platinum Services Remote Patch Installation Roles
Some sample role definitions for the remote patch installation are listed below:
Oracle Patch
Coordinator
Coordinates service delivery with customer for remote patch installation.
Documents information regarding the remote patch installation event in
the Service Request.
Schedules and hosts remote patch installation coordination calls and tracks
remote patch installation schedule.
Oracle Patch
Engineer
Performs technical aspects of the remote patch installation, for example
performs the technical patch assessment, creates a patch plan, completes
the pre-checks, downloads the patch bundles and performs the actual patch
installation.
Assists obtaining information for Oracle Support Services to review and
resolve issues, per Oracle’s technical support policies, that arise during the
remote patch installation.
Updates the Service Request as applicable during the remote patch
installation.
Updated: July 29, 2019 Page 15 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
RASCI-Table
Highlights of Patching Responsibilities Except for the patching services included in Oracle Platinum Services as outlined above, you are
responsible for the installation of all other upgrades and patches to the Certified Platinum Configuration,
including software updates to non-critical components (i.e. power distribution units, Cisco switches etc.).
Remote patch installations do not include Oracle Database version upgrades (e.g., 12.x to 18.x). Grid
Infrastructure upgrades are not normally included in the remote patch installations described in this
document; however, you may request an Oracle Grid Infrastructure upgrade and, if Oracle agrees to
install the upgrade, it will count as a separate remote patch installation. Your total remote patch
installations for the year will be reduced accordingly.
All upgrades and patches to non-eligible configurations (non-certified Platinum configurations) are your
responsibility. When conflicts resulting from customer-added software are detected, Oracle will remove
the conflicting software in order to complete the customer-approved Platinum Services patch update. Post
patching, you will reinstall any software that was causing a conflict.
Updated: July 29, 2019 Page 16 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix I – Access Requirements Remote patch installation cannot be completed without proper access to the Certified Platinum
Configuration, such as:
Patch Assessments
Patch Planning Pre-Check & Download
Patching
Root/sudo Access
o Database nodes Root Root/Sudo Root/Sudo Root/Sudo
o Cell nodes Root Root/Sudo Root/Sudo Root/Sudo
o InfiniBand Switches Root Root Root Root
o ZFSSA Root
Root/Sudo Root/Sudo
ILOM Root Access
o Database nodes Sudo Sudo
o Cell nodes Sudo Sudo
o ZFSSA Root
Root Root
DBSNMP *
Oracle Root Sudo Sudo
Orarom Sudo
* Your DBSNMP password is needed by Oracle to monitor your Certified Platinum Configuration per the Oracle Platinum Services policy. Note: Root access is only needed if the Patching team needs to run Exachk in the environment (including post Exachk), but is not needed if Exachk report(s) are uploaded to create patch assessment(s) Patch Planning and Pre-Check: orarom with sudo access is enough (ssh equivalence for root user should be configured across the nodes) Patching: orarom with sudo access is sufficient to perform patching and also for investigating issues with
the Engineered Systems Support team, although it is advisable to have root access because it is
unknown whether a customer has limited access for a sudo user.
Updated: July 29, 2019 Page 17 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix II – Space Requirements Remote patch installation cannot be completed without meeting adequate space requirements for each
product/component of the Certified Platinum Configuration. Recommended space requirements are:
Component Exadata Exalogic Supercluster ZDLRA
YUM (bare metal configuration) /file system : 3.5 GB /u01 file system : 20 GB NA NA
/file system : 3.5 GB /u01 file system : 20 GB
YUM (Oracle VM X5/X6/X7/X8 conf.)
/file system : 3.5 GB /u01 file system : 12 GB Grid / Oracle home partitions being patched: 10 GB NA
Grid / Oracle home partitions being patched: 15 GB NA
CELL /file system : 3.5 GB NA /file system : 3.5 GB / file system : 3.5 GB
IB /file system : 80 MB /tmp file system : 120 MB
/file system : 80 MB /tmp file system : 120 MB
/file system : 80 MB /tmp file system : 120 MB
/file system : 80 MB /tmp file system : 120 MB
Solaris Domains & Zones NA NA /file system : 12GB NA
Physical - Compute nodes NA /file system : 4.2GB NA NA
Virtual - Compute nodes NA /file system : 1.1GB NA NA
Updated: July 29, 2019 Page 18 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix III – Communication Plan During the remote patch installation process, from initiation until closure, communications will be
documented in the Service Request. When significant updates happen in the Service Request, an email
will be sent to the contact persons listed in the Service Request. Some customers have created a separate
mailing list to be used in the Platinum Patch Event Service Request (in the contact email id field) which
contains all the customer contacts who need to be informed when changes are made to the Service
Request. This helps makes the target group maintenance easy and reusable.
Telephone contact can be established at any time during the remote patch installation. Call your local
Oracle Support Center and enter your Service Request number. You will be transferred to the Oracle
Patch Engineer assisting with the remote patch installation. Phone numbers may be found on the Oracle
Support Contacts Global Directory page.
If you need management attention to your situation, you can escalate the Platinum Patch Event Service
Request. This is per the standard escalation process as documented in How to Escalate a Service Request
(SR) with Oracle Support Services (DOC ID 199389.1).
For quality control purposes, Oracle does not permit the use of bridge calls, web conferences or other
tools to follow engineers during the remote patch installation. Remote patch installation progress can be
monitored by customers through the Service Request.
Updated: July 29, 2019 Page 19 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix IV – Rolling vs. Non-Rolling Patching The quarterly Full Stack Download Patch contains patches for each layer of Engineered System that
require patching, each of which can be applied in a rolling or non-rolling fashion*. The decision of
whether to patch in a rolling or non-rolling fashion is determined by you.
* For Zero Data Loss Recovery Appliance, patching is applied in a non-rolling fashion ONLY.
* For ZFS Storage Appliance, patching is applied in a rolling fashion ONLY. Resources transition
between the two controller nodes and are available to clients throughout the patching cycle.
Rolling Rolling patch application refers to patching where Oracle will remotely perform the patch installation
while your Certified Platinum Configuration is online. Patches will be installed on each application,
server, storage server, or storage array covered under Platinum Services. To minimize downtime, patches
will be installed one server at a time, in a rolling fashion.
For example, with Exadata Storage Cell patching, patches can be applied one cell node at a time. Each
cell node is taken offline individually and patched while the other cell nodes remain available.
Storage Cell
Advantages:
o Does not require database downtime
Disadvantages:
o Requires Automatic Storage Management (ASM) high redundancy to reduce the risk of
disk failure.
o Length of time to patch.
Database Node
Advantages:
o Does not require database-wide downtime. Depending on configuration, there will be
minimal downtime as instances are shut down on the node being patched
Disadvantages:
o Must be on Oracle Database 11.2.0.2 Bundle Patch 2 or higher
Non-rolling
Non-rolling patch application—also known as all node patch application—refers to patching where
Oracle will remotely perform the patch installation while your Certified Platinum Configuration is offline.
Patches and updates will be performed in parallel, when technically feasible, to complete the update as
quickly as possible.
In some cases, the Certified Platinum Configuration may need to be shutdown, the hardware system may
have to be rebooted, and the Certified Platinum Configuration may have to be brought back up in order to
complete the patch installation successfully. Note that some components cannot be patched in parallel.
Applying the Exadata Storage Cell example to a non-rolling patch scenario:
Updated: July 29, 2019 Page 20 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Storage Cell
Advantages:
o Requires least amount of downtime to patch.
o Eliminates the risk of a single disk failure.
Disadvantage:
o Does require database downtime.
Database Node
Advantages:
o Requires least amount of downtime to patch.
Disadvantage:
o Does require database downtime.
Appendix V - Scheduling and Patch Duration
Patch Scheduling
Oracle suggests that a remote patch installation be scheduled eight (8) to twelve (12) weeks before the
date the patch is to be installed.
Many customers know their maintenance windows well in advance. Some even plan them more than a
year ahead. It is important to share those dates with the Oracle Patch Coordinator.
Patch Duration
When planning for a remote patch installation, you need to take into account the time a remote patch
installation will take. This is dependent on size, type, method, patching needs, etc.
The table below shows example timing for remote patch installations in rolling and non-rolling fashion.
These timings represent estimates only and not the actual time a customer will experience. Note for
rolling remote patch installations, the times may be split over multiple shifts.
Exadata
Rack Size Non-Rolling Rolling
Quarter 8 – 12 Hours 24 Hours
Half 12 – 16 Hours 3 Days
Full 16 – 24 Hours 6 Days
Exalogic
Includes physical and virtual Patch Set Update (PSU) upgrades as well as upgrading the components of
Exalogic: switch, storage and compute nodes.
Rack Size Non-Rolling Rolling
Eighth 6-8 hours 14 hours
Quarter 8-10 hours 16 hours
Half 10-12 hours 24 hours
Full 16-20 hours 38 hours
Updated: July 29, 2019 Page 21 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
SuperCluster
Rack Size Non-Rolling Rolling
Half 10-14 hours 2 days
Full 12-16 hours 5 days
Split Window Patching Oracle understands the customer business requirements may dictate limited patching windows such that
system patching must be split into multiple windows over time.
A Platinum patch event will allow for this split window scheduling for consecutive days between Sunday
9PM EST and Friday 9AM EST and is subject to the following guidelines independent of rolling or non-
rolling:
- Cell storage patching does not impact the databases, therefore patching can be continuous. But, if
customer requires a split window for cell storage patching, a minimum of 9 hours is required per
window.
- All other components require a minimum of 6 hour windows per node
Example of Split Window Scheduling
Consider example optimal timing for each component below. Note: all components may not be part of
one patch event:
Component Hours
Yum upgrade 2.5 (per compute node)
Bundle Patch 2 (per compute node)
Cell Patching 3.5 (per storage node)
OJVM + cat bundles 2
Switches .75 per switch
Grid Infrastructure 2.5
New home installation 1
Database maintenance upgrade 2.5 hours (per database)
*Split window patching is not available for ZFS Storage Appliance.
Updated: July 29, 2019 Page 22 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Rolling Back Patches In general, each of the components may be patched independently; therefore, one component can be
patched without the requirement to patch any of the other components. This means, unless otherwise
directed by the product engineering team, once a component is patched successfully across all nodes, that
component will not be rolled back.
Cancellations A scheduled remote patch installation may need to be cancelled. If canceled by either party, Oracle will
work with you to find a new date that fits both parties’ schedules.
For patching requests created via the online self-scheduling tool, customers can reschedule new dates
using available time slots. For more information, see Appendix XI – Patch Scheduling Reference Table.
Updated: July 29, 2019 Page 23 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix VI – Example: Oracle Platinum Services Remote Patch
Installation Process Flow
Updated: July 29, 2019 Page 24 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix VII – Example: Patching Schedule The below example lays out the annual schedule for patching four (4) racks over a two (2) week interval
every six (6) months.
Month 1 Month 2 Month 3 Month 4 wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12 wk13 wk14 wk15 wk16 R1 R2
CPC1 R1
CPC2 R1
CPC3 R1
CPC4 R1
Month 5 Month 6 Month 7 Month 8
wk17 wk18 wk19 wk20 wk21 wk22 wk23 wk24 wk25 wk26 wk27 wk28 wk29 wk30 wk31 wk32 R1 R3
CPC1 R3
CPC2 R3
CPC3 R3
Month 9 Month 10 Month 11
wk33 wk34 wk35 wk36 wk37 wk38 wk39 wk40 wk41 wk42 wk43 wk44 R2 CPC4 R3
Release of QFSDB (R1 = Release 1 of QFSDB)
Patching of Certified Platinum Configuration (R1=Release 1 of QFSDB) (CPC1=Certified Platinum Configuration 1)
Last Date on Release T-6 months
Updated: July 29, 2019 Page 25 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix VIII - Creating a Platinum Patch Event Service Request
1. Go to My Oracle Support and login with your Username and Password.
2. From the Systems tab, in the Systems region, find the Platinum-certified system you are looking
for and click on the System Name to drill down into the System.
NOTE: Platinum-certified systems will be indicated as such in the Level field.
3. Click Create SR.
This will take you to the Configuration tab of the Create Service Request page. The System
Name will be populated in the System field.
4. In the Host field, select or search/select the appropriate Host.
Once you select a Host, the Host details (Serial Number, Asset Name, and Organization) will be
returned and the Product and Operating System/Version fields presented.
5. For Product, select “Oracle Platinum Services” and select appropriate Operating System/Version.
6. For Problem Type, select “Patching Request”.
7. Continue with the remaining steps in the Service Request creation process ensuring all mandatory
fields are complete.
8. Click the Submit button. A Service Request will be created with System and Host information
attached.
Updated: July 29, 2019 Page 26 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix IX – Example: Exadata Patch Plan (Non-Rolling)
1. Overview and Notes:
1.1. Patches to be applied for the Exadata Rack:
14522699: 11.2.3.2.1 for storage servers
16432033: 11.2.3.2.1 base repository iso for database nodes (via YUM)
17395890: 11.2.0.3.21 Bundle Patch for Oracle databases
1.2. Environment Details:
1.2.1. Customer Name : XYZ
1.2.2. Rack Type : Half-rack
1.2.3. Database Servers Names : db01 – db04
1.2.4. Exadata cell node Names : cel01 – cel07
1.2.5. Oracle_Home : /u01/app/oracle/product/11.2.0.3/dbhome_1
1.2.6. Grid_Home : /u01/app/11.2.0.3/grid
1.2.7. Databases : dbm1, prod1, dev1
1.2.8. OPatch Version : 11.2.0.3.4
1.3. Software Locations:
/u01/patches/cell
/u01/patches/YUM
/u01/patches/BP21
2. Pre-checks Storage Software on Cell nodes:
2.1.1. Check ILOM access to cell nodes
2.1.2. Check ILOM access to database nodes
2.1.3. Check for clean file systems on cells and the database hosts
2.1.4. Ensure the various cell network configurations are consistent
2.1.5. Validate cell disks for valid physicalInsertTime
2.1.6. Check disk repair time set to default 3.6 Hrs
2.1.7. Check if all disk groups mounted by all ASM instances
2.1.8. Address if any potential problems found with Infiniband switch
2.1.9. Prepare cell_group, dbs_group files
2.1.10. Check for existing root SSH equivalence
2.1.11. Check the output of /usr/local/bin/imageinfo on all cell nodes
2.1.12. Copy the patch software from CTA to database node 1
2.1.13. Ensure customer has a file system and database backups
3. Applying Storage Software to Exadata Cells:
3.1. General notes on patching and monitoring
3.2. Stopping Services:
3.2.1.1. Stopping all services and agents in db nodes 3.2.1.2. Stopping services in cell nodes
3.3. Patchmgr Utility
3.4. Start a screen session from the CTA and ssh to database node 1
3.4.1.1. Run cleanup command
3.4.1.2. Start the prerequisites check
3.4.1.3. Start the patch in all cell nodes
Updated: July 29, 2019 Page 27 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
3.4.1.4. Monitor the patch progress
3.5. Checking patch status after a successful operation (to be performed on all cells)
3.6. Clean up the cells using the -cleanup
3.7. Check the InfiniBand switch software and opensm configuration
4. Post-checks Storage Software on Cell nodes:
4.1. Restore NFS file systems
4.2. Bring up CRS, ASM, DB Instances
5. Pre-checks Yum update on Database nodes:
5.1. Check image version
5.2. Stop services
6. Applying Yum update to Database nodes:
6.1. Upgrade automatically using dbnodeupdate.sh*
*manual steps, if needed, are included in Appendix A
6.1.1.1. Patch with dbnodeupdate
6.1.1.2. Restart services
6.2. Repeat step 7 for all other database nodes. <<<This can be done in parallel for non-
rolling
7. Post-checks Yum update on Database nodes:
7.1. Check image version
8. Pre-checks Bundle Patch on database nodes:
8.1. Unzipping the Quarterly Database Patch for Exadata (11.2.0.3.21)
8.2. OCM Configuration:
8.3. Validation of Oracle Inventory:
8.4. Run OPatch Conflict Check:
8.5. Run OPatch System Space Check:
8.6. Stop all agents running on database nodes
8.7. Check the running resources
9. Applying Bundle Patch to database nodes:
9.1. Upgrade automatically using opatch auto*
*manual steps, if needed, are included in Appendix B
9.2. Repeat step 10 for all other database nodes. This can be done in parallel for non-rolling
10. Post-checks Bundle Patch on database nodes:
10.1. Check for streams
10.2. Run catbundle, utlrp
10.3. Review catbundle logs
10.4. Check for invalid objects
10.5. Patch Apply Crosscheck:
10.5.1.1. opatch lsinventory
10.5.1.2. Verify the catbundle apply
10.5.1.3. Check the running resources
Updated: July 29, 2019 Page 28 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
A. Appendix A: Applying Yum update to Database nodes (Manual Steps, if needed)
A.1. Download required patches
A.2. Review the patch README
A.3. Performing One-Time Setup on Database Servers
A.4. Mount ISO on compute node
A.5. Execute bootstrap.sh to perform the initial phase of One-Time Setup on Database Servers
A.6. After the system reboot completes, continue on with the final steps of One-Time Setup
A.7. Relink the Oracle software components
A.8. Check the agents and mounts
B. Appendix B: Applying Bundle Patch to Database nodes (Manual Steps, if needed)
B.1. export TMOUT=18000
B.2. Stop the CRS
B.3. Run the pre root script:
B.4. Apply the Quarterly Database Patch (11.2.0.3.21) patch to GI Home
B.5. Run the pre script for DB component of the patch
B.6. Apply the Quarterly Database Patch (11.2.0.3.21) patch to DB Home
B.7. Run the post script for DB component of the patch
B.8. Run the post root script
C. Appendix C: Rollback
C.1. Rollback procedure for storage software
C.2. Rollback procedure for YUM update
C.3. Rollback procedure for bundle patch
Updated: July 29, 2019 Page 29 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix X – Roles and Responsibilities for Oracle Databases for
SAP Applications
In accordance with the Oracle Platinum Technical Support Policy, Oracle Platinum Services includes
limited patching for Oracle Database hosted environments.
Out of scope components may be patched by the customer or the customer may purchase the branded
service “Oracle Engineered Systems Incremental Patch Deployment for Oracle Platinum Services”
directly from ACS. If contracted, ACS will coordinate directly with Platinum Services to assist with
patching of the customer’s engineered systems.
Below is a diagram of example responsibilities for a patch event:
Updated: July 29, 2019 Page 30 of 30
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Appendix XI – Patch Scheduling Reference Table
System Patching Request Scheduling Method Cancellation Method
Exadata Use the self-scheduling tool to
create a new request
The tool automatically creates a
Change Management ticket
Use the self-scheduling tool to
remove the existing patching
request
Create a new request based on
the available slots
Exadata Split event Contact Oracle Support Services
Discuss tentative dates with your Patch
Coordinator
Open Service a Request (SR) in My
Oracle Support
Oracle will work with customers to
find a new mutually convenient date, if
patching is canceled by either party Exadata IPSEC
Exalogic
SuperCluster
ZDLRA
ZFSSA