Upload
ibm-ims
View
1.009
Download
2
Tags:
Embed Size (px)
Citation preview
1 © 2013 IBM Corporation
IMS Online Reorganization Facility
Dennis EichelbergerIT Specialist IMS Advanced Technical [email protected]
2013 IBM Corporation2
IMS Online Reorganization FacilityAgenda
• IMS ORF Overview
• Database Types supported
• Other required IMS Tools
• Phases
2013 IBM Corporation3
IMS Online Reorganization FacilityOverview
• A one-step reorganization of IMS databases with minimal impact to its availability
• Reduces database downtime from hours to seconds• Eliminates the need for manual intervention before,
during and after a reorganization • Supports:
• Internal logical relationships• Secondary indexes• Full Function Databases• BMP Pause and Resume
2013 IBM Corporation4
IMS Online Reorganization Facility Why Reorganize a Database?
■ Performance– Fragmentation– Overflow– CI/CA Splits
■ Space– Nearing Limitations– Current Definitions– Reclaim– IMS Limitations
■ DBD Changes
2013 IBM Corporation5
IMS Online Reorganization Facility All Full Function Databases
■ HDAM■ HIDAM■ SHISAM■ HALDB - PHDAM/PHIDAM
– Entire database – all partitions– Single partition
■ Partitioned secondary indexes– Entire index– Single partition
■ Secondary indexes– When primary DB reorganized– Index only
IBM IMS Online Reorganization
Facility
2013 IBM Corporation6
IMS Online Reorganization Facility Other Required IMS Tools
• Unload– IMS Basic Unload– IMS High Speed Sequential Retrieval (HSSR/HPUNLOAD)
• Load– IMS Basic Reload– IMS High Performance Load (HPLOAD)
• Prefix Resolution/Update– IMS Basic Prefix Resolution & IMS Basic Prefix Update– IMS High Performance Prefix Resolution/Update
• Image Copy– IMS Basic Image Copy– IMS High Performance Image Copy
Note: Secondary Index tools are not used; indexes are built by ORF
Page 7
The Offline Reorg Process
Unload
Load
Build Sec Indexes
Image Copy (w/Ptr Check)
Prefix Res/Upd
/DBR DB /STA DB
Data Unavailable
Page 8
The Online Reorg Process – with shadow data sets
Allocate Shadow data sets
HP Unload
HP Load
Start Change Capture
Build Sec Indexes
Image Copy (w/Ptr Check)
Prefix Res/Upd
/DBR DB /STA DB
Update DBRC &
Rename Data SetsCopy Primary
DB
Finish Change Capture
Apply Changes
2013 IBM Corporation9
IMS Online Reorganization Facility Phases
• ORF phases occur in the following sequence:–Verification –Copy –Reorganization –Apply –Takeover The only phase allowing restart processing
–Completion
2013 IBM Corporation10
IMS Online Reorganization FacilityData Flow
2013 IBM Corporation11
IMS Online Reorganization Facility Reorganization Process
• ORF can be run while:– IMS system(s) are up– IMS system(s) are down– IMS databases are online– IMS databases are offline
• No JCL changes or manual intervention needed for any of the processes above
2013 IBM Corporation12
IMS Online Reorganization Facility VERIFICATION Phase
If Normal Execution (not Restart): • IMS Release and component requirements are verified• Operating system requirements are verified• DBRC registration/database state requirements are verified• Database DBD requirements are verified• Database data set requirements are met (cannot exceed 42
characters in length and are cataloged)• ORF level for STEPLIB in batch job/IMS control regions is verified• JCL requirements are met• Control statement syntax is verified• Shadow data sets (.S data sets) are constructed/verified/allocated• Temporary (.T data sets) data sets deleted and deletions verified• Other miscellaneous checking is performed
2013 IBM Corporation13
IMS Online Reorganization Facility COPY Phase
• BMPs are paused (if usermod is installed) • Databases are closed/reopened using TOSI to flush buffers
across all IMS subsystems and to update the catalog/VTOC• BMPs are resumed (if usermod is installed)• DBRC status remains unchanged - allocated and authorized• Waits for database syncpoint, then online component of ORF
is notified to: Begins capturing all changes made to the primary database data sets – type 50
records
• Copies the original primary database only to shadow copy• Near the end of the Copy phase, another database
close/reopen is done and: Captured changes are sent to ORF batch utility & applied to shadow primary
database datasets. The shadow primary index (for HIDAM or PHIDAM) is created from the shadow
primary database The ILDS (for HALDB) is created for the shadow ILDS
2013 IBM Corporation14
IMS Online Reorganization Facility REORGANIZATION Phase
• Shadow database is Unloaded• Shadow database is Reloaded• Shadow Secondary indexes are rebuilt except for HALDB
PSINDEX databases• Shadow internal logical relationships are rebuilt by Prefix
Resolution/Update (if any) • Shadow databases are Image Copied (and optionally
Pointer Checked if rquested)• Online component of ORF continues to capture update calls
to the original databases and are written to a temporary data set for use in the subsequent Apply phase.
2013 IBM Corporation15
IMS Online Reorganization Facility APPLY Phase
– Captured changes are applied to the shadows– ORF pauses any BMPs and issues a /DBR FEOV
command to stop original databases– DBRC flag is set to PROHIBIT AUTH– Apply phase ends after all remaining captured-updates
have been applied to shadows If heavy activity, the apply Is done in iterations
(continues to capture while applying)– Original database and its indexes are unavailable until
after the Takeover phase (swapping the shadow database to the original database)
2013 IBM Corporation16
IMS Online Reorganization Facility TAKEOVER Phase
• TAKEOVER(Y)• Verifies the database and its indexes are unavailable
• Notifies DBRC of the REORG with a timestamp after the /DBR command
• Notifies DBRC of the image copy with a timestamp that is greater than the REORG timestamp
• Notifies DBRC of a DB ALLOC that is needed for log data sets • Notifies DBRC of logs that were created during the Apply phase• Swaps the reorganized shadow data sets with the original
database data sets• Performs the ACBGEN and ACBLIB or DMB replacements (if
DBD changes have occurred) and copies the NEWDBD to the current DBDLIB if DBDCOPY(Y) is specified
• Authorizes the databases in DBRC (turns off REORG INTENT and PROHIBIT AUTH flags)
• Starts the databases by using a /START command• Continues to recover the log marked in error• Notifies DBRC of the log with the new timestamp
2013 IBM Corporation17
IMS Online Reorganization Facility TAKEOVER (with DBD changes)
– NEWDBD and ONLINECHANGE keywords are ignored for HALDB
– If NEWDBD(ddname) was specified with ONLINECHANGE(Y) and DBDCOPY(Y), ORF automates the replacement of DBDs and ACBs/DMBs to eliminate manual intervention and increase database availability
– Automatically generates an ACB– Copies changed DMB to ACBLIBA and ACBLIBB– Activates new DMB– Performs a DMB replacement in the staging and active ACBLIBs– Starts database(s) and resets DBRC flags – If NEWDBD(Y) was specified with ONLINECHANGE(N), ORF does
not automate the replacement of DBDs and ACBs/DMBs; manual intervention is required
– ORF does not automatically generate an ACB – Database(s) remain offline and DBRC flags are not reset– Manual intervention is required
2013 IBM Corporation18
IMS Online Reorganization Facility TAKEOVER Restart
If ORF detects that there is a restart pending by reading the data that was saved in the restart data set:
– Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords
– Databases remain in PROHIBIT AUTH and REORG INTENT status until restart is complete
– Databases remain stopped until restart is complete– ORF Takeover Restart utility is invoked– Automatically restarts from where it stopped in previous
Takeover phase− DBRC flags are reset when TAKEOVER restart has
successfully completed− Databases are started
2013 IBM Corporation19
IMS Online Reorganization Facility TAKEOVER (scheduled)
Scheduled Takeover phase Allows the reorganization Takeover process to be scheduled
within a window that you specify Allows you to manually perform steps after the reorganization
process and before the databases are brought online, if needed. Use the TAKEOVER.WINDOW(time) keyword Must specify TAKEOVER(Y) on REORG command The database is left in DB recovery needed state with PROHIBIT
AUTH and REORG status set in DBRC. Restart information is saved in a restart data set. Takeover can be completed by resubmitting your job with
TAKEOVER(YES) and RESTART(AUTO) keywords specified
Format of Scheduled Time: (begHH:MM[,endHH:MM[,endaction]]) Format '00:00' to '23:59'. Times are in 24–hour time
2013 IBM Corporation20
IMS Online Reorganization Facility COMPLETION Phase
Final reports and messages written
Shadow data sets deleted, if DELETE(Y)
Data sets are deallocated
Miscellaneous cleanup tasks performed
ORF job completes
2013 IBM Corporation21
IMS Online Reorganization Facility TAKEOVER (delayed)
Allows you to delay the Takeover phase before the original database datasets are affected
Allows you to manually perform steps after the reorganization process and before the databases are brought online, if needed
Original job must have TAKEOVER(DELAY) Databases are left in DB recovery needed state with PROHIBIT
AUTH and REORGI status set in DBRC Restart information is saved in a restart data set Takeover can be completed by resubmitting your job with
TAKEOVER(YES) and RESTART(AUTO) keywords
2013 IBM Corporation22
IMS Online Reorganization Facility BMP Management
• The Pause/Resume: – ORF sets a flag in a control block allocated in the CTL address
space– ORF verify on a CHKP call if the flag has been set– The BMP checks during the suspend state if the flag has cleared– ORF uses logger exit for communication (DFSFLGX0)– It cleans up if a failure occurs (the XCF connection fails)– Failure is posted from XCF that the ORF job went away– ORF then cleans itself up– Once the BMP determine the suspend flag has been cleared, the
BMP will be resumed from the last checkpoint Note: DLIBATCH is not Supported by ORF:
• If there is a DLI Batch running, IMS ORF will fail – Unable to obtain authorization from DBRC.
• Vice Versa – If ORF is active, DLI batch will fail to initiate as the database is marked REORGI is active
2013 IBM Corporation23
IMS Online Reorganization Facility Online Processing
IMS • New transaction arrives when the database is DBR’d
– Transaction placed on suspend queue• Exit/Automation to process suspend queue and reissue
transaction• /STA DB will requeue message
CICS• SCHEDULE PSB request when the database is DBR’d
– ORF detects that it has DBR’d the database• The Thread is put into temporary wait
– Typically short lived
2013 IBM Corporation24
IMS Online Reorganization Facility Online Processing
ODBA• APSB request when DB is DBR’d
– ORF detects that it has DBR’d the DB• Application TCB is put into temporary wait
– Typically short lived
2013 IBM Corporation25
IMS Online Reorganization Facility Installation Considerations
• IBM IMS Generic Exits is required and part of the IMS Tools Base
– Partner Exit (DFSPPUE0)– Logger Exit (DFSFLGX0)
• IBM IMS Tools Online System Interface (TOSI) is required
• IBM IMS BMP Pauser Interface – DFSRRC99
– No JCL Changes• By aliasing DFSRRC99 for DFSRRC00 in the running reslib
– With JCL Changes
• By Steplib with DFSRRC99 aliased as DFSRRC00 and invoking HRFRRC00 as the region controller in the BMP
• IMS Control Regions (STEPLIB) modification and recycle needed
2013 IBM Corporation26
IMS Online Reorganization Facility Restrictions
• IMS Online Reorganization Facility does not support:– Batch jobs running as DLI, DBB, or ULU types during reorganization– HSAM databases – HIDAM databases with compressed root keys – IMS Utility Control Facility (UCF) – Databases and data sets that are not registered to DBRC – External logical relationships – Fast Path databases – Reorganization of multiple HALDB partitions in one step (you can do
this type of reorganization in parallel with separate IMS Online Reorganization jobs)
– HALDB DBD changes – Reorganization of HALDB Multi-Volume data sets – Uncataloged or corrupted databases
2013 IBM Corporation27
IMS Online Reorganization Facility DBD Changes not allowed
• IMS Online Reorganization Facility does not allow the following DBD changes: – DBD name or type– Access method – Randomizer name– Segment changes
• type• variable to fixed or fixed to variable lengths• length decrease
– Conversion from Full Function to HALDB– Heirarchical structure change– Logical relationships added, modified or deleted– VSAM blocksize– Secondary index
• delete• conversion from or to shared secondary index
2013 IBM Corporation28
IMS Online Reorganization Facility Common Questions/Answers (continued)
1) How many times do the databases get stopped and a FEOV is done?
They are temporarily paused twice (close and reopen the database (to get the control blocks updated in the catalog/VTOC) and stopped (/DBR’d) once with one force-end-of-volume unless FEOV(N) is specified.
2) What does ORF use to communicate between the ORF batch job and the IMS online control region(s)?
ORF uses XCF to communicate between shared IMS systems. Two XCFGROUP names need to be setup during ORF installation (one for TOSI (if none exists) and one for ORF online capturing).
2013 IBM Corporation29
IMS Online Reorganization Facility Common Questions/Answers (continued)
3) How does ORF handle CICS/ODBA applications when ORF needs to pause/stop the databases?
When a CICS or ODBA APSB request detects that ORF needs to momentarily stop or issue a /DBR on a database, the thread in which the APSB request is made is put into a temporary wait state until ORF restarts the database.
No special implementation is needed. This feature can be disabled, if desired.
4) Do I need to recycle IMS to apply ORF maintenance?
Not always, depends on APAR. specified
2013 IBM Corporation30
IMS Online Reorganization Facility Common Questions/Answers (continued)
5) How does the BMP Pauser Interface work during an ORF job with concurrent BMP(s)?
If an ORF job needs to do a /DBR for a HALDB database, the PSB is automatically paused (no abend) after the next checkpoint and waits for ORF to restart the database and the BMP wait is released.
If an ORF job needs to do a /DBR for a non-HALDB database, the PSB is automatically stopped (U3303 abend) after the next checkpoint. After the database(s) are restarted, ORF restarts the BMP at the last checkpoint.
If a BMP attempts to start in the middle of an ORF job, a check is performed to see if a /DBR needs to be done or has not yet completed. If so, the job step TCB waits until ORF restarts the database.
6) Do I need to preallocate the shadow datasets? You must preallocate the shadows if SHADOWS(E) for Existing was
specified You do not need to preallocate the shadows if SHADOW(A) was
specified. ORF will use the same attributes of the production database datasets.
2013 IBM Corporation31
IMS Online Reorganization Facility Common Questions/Answers (continued)
7. Can I run an ORF job even if no online systems are up? Yes, ORF relies on DBRC subsystem records and if none exist, it can
run without active IMS system(s).
8. Can I restart a failed ORF job? Yes, if the original job which failed had RESTART(Y) and it failed in the
TAKEOVER Phase. Failures in any other phases are not restartable; however, those phases do not affect the production database so no special action is needed.
9. How does ORF prevent any other utilities (reorg/recovery, etc.) from running during an ORF reorganization? ORF sets the REORG INTENT flag in DBRC which prevents any other
utilities from running against the databases being reorganized by ORF.
2013 IBM Corporation32
IMS Online Reorganization Facility Benefits
Reduced database downtime
One step database reorganization
Eliminates manual intervention during reorganization
Automated replacement of DBDs and ACBs
Improved DBA productivity
Flexible reorganization schedule
Integration with IBM IMS High Performance Tools
2013 IBM Corporation33
IMS Online Reorganization Facility
QUESTIONS AND COMMENTS