CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
David Webster
HOT OR NOT?HOW TO RELEASE CHANGES WITHOUT IMPACTING CUSTOMERS
April 2014
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHAT IS THIS SESSION NOT ABOUT?
2
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHAT THIS SESSION IS ABOUT?
3
HOW DO WE MAKE CODE AND TABLE CHANGES TO AN ORACLE DATABASE WITHOUT IMPACTING ANY OF THE USERS?
HOW DO WE DO THAT EVEN ON A REALLY COMPLEX DATABASE WITH 1000S OF USERS/SESSIONS?
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHAT IS THIS SESSION ABOUT, SPECIFICALLY?
4
• Why put code in the DB?• How to change that code while the DB
is active.• Under what circumstances is that
challenging? (24x7, high executions, daemons, DB Links, dependencies)
• What can we do about it?
• ELEPHANT IN THE ROOM : EDITIONS
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
JUST BEFORE I START….
5
ALL TESTS ARE ON 11.2.0.3
HOT OR NOT….
HOT DOESN’T MEAN HOT, COLD DOESN’T MEAN COLD
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
ABOUT BETFAIR
6
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
ABOUT BETFAIR
7
PIONEERED ONLINE PERSON-TO-PERSON BETTING. BETFAIR IS THE BROKER, THE EBAY OF BETTING, NOT THE “HOUSE”
COMPANY STARTED IN 1999 WITH 2 FOUNDERS
OVER 1500 EMPLOYEES ACROSS THE WORLD
DATA CENTRES ACROSS EUROPE, IN AUSTRALIA AND US
FULL OFFERING OF BETTING PRODUCTS, INCLUDING SPORTSBOOK, POKER, CASINO AND ARCADE.
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
ABOUT DAVID WEBSTER
8
15 YEARS WORKING WITH ORACLE
SITE RELIABILITY ENGINEER – ORACLE PERFORMANCE
BLOG : HTTP://ANOTHERDAVEWEBSTER.WORDPRESS.COM/
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHY PUT CODE IN THE DATABASE?
9
• CENTRALISED LOGIC AND VERSIONING (I KNOW…)
• PL/SQL APIS. ABSTRACTION AND ENCAPSULATION
• PERFORMANCE
• DEPENDENCY TRACKING
• OTHERS……
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHY SHOULD I NOT PUT CODE IN THE DATABASE?
10
• PERFORMANCE…..DIGRESSION ALERT!!!PREFETCH & READ CONSISTENCY
• MAKES RELEASING CODE MORE DIFFICULT!
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
WHAT MAKES OUR EXCHANGE A COMPLEX DB?
11
1. IT’S 4 DATABASES
2. OVER 100K EXECUTIONS A SECOND
3. 1000S OF SESSIONS, 100S OF CLIENTS
4. LOTS AND LOTS OF CODE
5. BACKGROUND JOBS
6. 24X7 OPERATION
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
(RE)COMPILING PACKAGES
12
Q : WHAT DOES ORACLE NEED TO COMPILE A PACKAGE BODY?A : ORACLE REQUIRES AN EXCLUSIVE LOCK ON THE OBJECT
Q : DOES ANYTHING GET INVALIDATED WHEN I COMPILE A BODY?A : NO
Q : WHAT ABOUT A PACKAGE SPEC?A : COMPILING A PACKAGE SPEC INVALIDATES CALLING BODIES*
Q : IS MY (RE)COMPILATION LIMITED TO THE CODE THAT I CHANGE?A : NO…..
COMPILE ONLY WHAT YOU NEED
AVOID INVALIDATING PACKAGES
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
GETTING THE LOCK
13
• STOP BACKGROUND JOBS
• NEED A WINDOW
10 200 30 5040 60
Minute
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
PACKAGE STATE
14
WHAT IS PACKAGE STATE?
ORA-04068: existing state of
packages has been discarded
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
PACKAGE STATE
15
CONSTANTS
CONSTANT WORKAROUND
ORA-04068: existing state of
packages has been discarded
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
UNNECESSARY INVALIDATIONS : FGDT
16
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
UNNECESSARY INVALIDATIONS : FGDT
17
FINE GRAINED DEPENDENCY TRACKING
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
UNNECESSARY INVALIDATIONS : FGDT
18
NEEDED NOT NEEDED
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
CHANGING PACKAGE SPECIFICATIONS
19
CHANGING THE PACKAGE SPECIFICATION INVALIDATES ALL CALLING PACKAGES
EXCEPT THAT’S NOT TRUE
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
CHANGING PACKAGE SPECIFICATIONS
20
END MEANS AFTER THE LAST REFERENCED PROCEDURE!
TO BE SAFE, ADD TO THE VERY END
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
CHANGING PACKAGE SPECIFICATIONS
21
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
CHANGING PACKAGE SPECIFICATIONS
22
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
LEFT WITH A MESS?
23
1. WHEN OVERLOADING, REFACTOR INTERNALLY
2. IF YOU CAN’T DELETE, THROW AN ERROR
3. DO A COLD RELEASE
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
DATABASE LINKS
24
1. DON’T USE THEM
2. SESSION-TASTIC
3. CONFUSE THE POOR CBO
4. DON’T USE THEM
5. EDGE CASES AND COMPLEXITY
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
DATABASE LINKS
25
FIXED USER DATABASE LINK
CONNECTED USER DATABASE LINK
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
DATABASE LINKS
26
RED_TIGER
RED_SCOTT BLUE_SCOTT
RED_SCOTT
TABLE
RED_TIGER
Package PACKAGE reads BLUE_SCOTT.TABLE@BLUE
GRANT EXECUTE ON PACKAGE TO RED_TIGER
GRANT SELECT ON TABLE TO RED_SCOTT
GRANT SELECT ON TABLE TO RED_TIGER
Bug 13415892 - UNNECESSARY INVALIDATION OF REMOTE STUBS
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
SUMMARY
27
1. TO COMPILE (OR RECOMPILE) A PACKAGE, YOU NEED AN EXCLUSIVE LOCK
2. PACKAGE STATE LIMITS OPTIONS
3. MINIMISE UNECESSARY COMPILATIONa. SCOPE PL/SQL VARIABLESb. MODIFY SPECIFICATION CAREFULLY
4. BEWARE DATABASE LINKS
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
THANK YOUQ&A
28
CONFIDENTIAL and not for reproduction without prior written consent. © of The Sporting Exchange Limited.
29
Recommended
