Chs. 15, 16, 17: Transactions, Conc. Control and Recovery Transaction Concept Transaction State ...
113
Chs. 15, 16, 17: Chs. 15, 16, 17: Transactions, Conc. Control Transactions, Conc. Control and Recovery and Recovery Transaction Concept Transaction Concept Transaction State Transaction State Implementation of Atomicity and Implementation of Atomicity and Durability Durability Concurrent Executions Concurrent Executions Serializability Serializability Recoverability Recoverability Implementation of Isolation Implementation of Isolation Transaction Definition in SQL Transaction Definition in SQL Testing for Serializability. Testing for Serializability. Lock-Based and Timestamp-based Protocols Lock-Based and Timestamp-based Protocols Deadlock Handling Deadlock Handling Failure Classification Failure Classification Storage Structure Storage Structure Recovery and Atomicity Recovery and Atomicity Log-Based and Shadow Paging Recovery Log-Based and Shadow Paging Recovery Remote Backup Systems Remote Backup Systems
Chs. 15, 16, 17: Transactions, Conc. Control and Recovery Transaction Concept Transaction State Implementation of Atomicity and Durability Concurrent
Chs. 15, 16, 17: Transactions, Conc. Control and Recovery
Transaction Concept Transaction State Implementation of Atomicity
and Durability Concurrent Executions Serializability Recoverability
Implementation of Isolation Transaction Definition in SQL Testing
for Serializability. Lock-Based and Timestamp-based Protocols
Deadlock Handling Failure Classification Storage Structure Recovery
and Atomicity Log-Based and Shadow Paging Recovery Remote Backup
Systems
Slide 2
Transaction Concept A transaction is a unit of program
execution that accesses and possibly updates various data items.
Usually delimited by statements begin transaction/end transaction
Usually delimited by statements begin transaction/end transaction A
transaction must see a consistent database. A transaction must see
a consistent database. During transaction execution the database
may be inconsistent. During transaction execution the database may
be inconsistent. When the transaction is committed, the database
must be consistent. When the transaction is committed, the database
must be consistent. Two main issues to deal with: Failures of
various kinds, such as hardware failures and system crashes
Failures of various kinds, such as hardware failures and system
crashes Concurrent execution of multiple transactions Concurrent
execution of multiple transactions
Slide 3
ACID Properties Atomicity. Either all operations of the
transaction are properly reflected in the database or none are.
Consistency. Execution of a transaction in isolation preserves the
consistency of the database. Isolation. Although multiple
transactions may execute concurrently, each transaction must be
unaware of other concurrently executing transactions. Intermediate
transaction results must be hidden from other concurrently executed
transactions. That is, for every pair of transactions T i and T j,
it appears to T i that either T j, finished execution before T i
started, or T j started execution after T i finished. That is, for
every pair of transactions T i and T j, it appears to T i that
either T j, finished execution before T i started, or T j started
execution after T i finished. Durability. After a transaction
completes successfully, the changes it has made to the database
persist, even if there are system failures. To preserve integrity
of data, the database system must ensure:
Slide 4
Example of Fund Transfer Transaction to transfer $50 from
account A to account B: 1. read(A) 2. A := A 50 3. write(A) 4.
read(B) 5. B := B + 50 6. write(B) Consistency requirement: the sum
of A and B is unchanged by the execution of the transaction.
Atomicity requirement: if the transaction fails after step 3 and
before step 6, the system should ensure that its updates are not
reflected in the database, else an inconsistency will result.
Durability requirement: once the user has been notified that the
transaction has completed (i.e., the transfer of the $50 has taken
place), the updates to the database by the transaction must persist
despite failures. Isolation requirement: if between steps 3 and 6,
another transaction is allowed to access the partially updated
database, it will see an inconsistent database (the sum A + B will
be less than it should be). Can be ensured trivially by running
transactions serially, that is one after the other. However,
executing multiple transactions concurrently has significant
benefits, as we will see.
Slide 5
Transaction State Active: the initial state; the transaction
stays in this state while it is executing Partially committed:
after the final statement has been executed. Failed: after the
discovery that normal execution can no longer proceed. Aborted:
after the transaction has been rolled back and the database
restored to its state prior to the start of the transaction. Two
options after it has been aborted: restart the transaction only if
no internal logical error restart the transaction only if no
internal logical error kill the transaction kill the transaction
Committed: after successful completion.
Slide 6
Transaction State (Cont.)
Slide 7
Implementation of Atomicity and Durability The
recovery-management component of a database system implements the
support for atomicity and durability by different methods log-based
recovery (later) log-based recovery (later) shadow-paging
shadow-paging The shadow-database scheme: assume that only one
transaction is active at a time. assume that only one transaction
is active at a time. a pointer called db_pointer always points to
the current consistent copy of the database. a pointer called
db_pointer always points to the current consistent copy of the
database. after the transaction reaches partial commit and all
updated pages have been flushed to disk, db_pointer is made to
point to the current copy. after the transaction reaches partial
commit and all updated pages have been flushed to disk, db_pointer
is made to point to the current copy. If the transaction fails, old
consistent (shadow) copy pointed to by db_pointer can be used, and
the current copy can be deleted. If the transaction fails, old
consistent (shadow) copy pointed to by db_pointer can be used, and
the current copy can be deleted.
Slide 8
Shadow-database scheme Assumes disks to not fail Useful for
text editors, but extremely inefficient for large databases:
executing a single transaction requires copying the entire
database.
Slide 9
Concurrent Executions Multiple transactions are allowed to run
concurrently in the system. Advantages are: increased processor and
disk utilization, leading to better transaction throughput: one
transaction can be using the CPU while another is reading from or
writing to the disk increased processor and disk utilization,
leading to better transaction throughput: one transaction can be
using the CPU while another is reading from or writing to the disk
reduced average response time for transactions: short transactions
need not wait behind long ones. reduced average response time for
transactions: short transactions need not wait behind long ones.
Concurrency control schemes: mechanisms to achieve isolation, i.e.,
to control the interaction among the concurrent transactions in
order to prevent them from destroying the consistency of the
database First, study the notion of correctness of concurrent
executions. First, study the notion of correctness of concurrent
executions.
Slide 10
Schedules Sequences that indicate the chronological order in
which instructions of concurrent transactions are executed a
schedule for a set of transactions must consist of all instructions
of those transactions a schedule for a set of transactions must
consist of all instructions of those transactions must preserve the
order in which the instructions appear in each individual
transaction. must preserve the order in which the instructions
appear in each individual transaction.
Slide 11
Example: Schedule 1 Let T 1 transfer $50 from A to B, and T 2
transfer 10% of the balance from A to B. The following is a serial
schedule in which T 1 is followed by T 2. Serial schedule: sequence
of instructions from various transactions where the instructions
belonging to a single transaction appear together in that
schedule
Slide 12
Example: Schedule 3 Let T 1 and T 2 be the transactions defined
previously. The following schedule is not a serial schedule, but it
is equivalent to Schedule 1. In both Schedule 1 and 3, the sum A +
B is preserved.
Slide 13
Example: Schedule 4 The following concurrent schedule does not
preserve the value of the sum A + B.
Slide 14
Another example (Raghu Ch.16, 17) Consider two transactions
(Xacts): T1:BEGIN A=A+100, B=B-100 END T2:BEGIN A=1.06*A, B=1.06*B
END Intuitively, the first transaction is transferring $100 from Bs
account to As account. The second is crediting both accounts with a
6% interest payment. There is no guarantee that T1 will execute
before T2 or vice- versa, if both are submitted together. However,
the net effect must be equivalent to these two transactions running
serially in some order.
Slide 15
Example (Contd.) Consider a possible interleaving (schedule):
T1: A=A+100, B=B-100 T2: A=1.06*A, B=1.06*B This is OK. But what
about: T1: A=A+100, B=B-100 T2: A=1.06*A, B=1.06*B The DBMSs view
of the second schedule: T1: R(A), W(A), R(B), W(B) T2: R(A), W(A),
R(B), W(B)
Serializability Basic Assumption: Each transaction preserves
database consistency. Serial execution of a set of transactions
preserves database consistency. Serial execution of a set of
transactions preserves database consistency. A (possibly
concurrent) schedule is serializable if it is equivalent to a
serial schedule. Different forms of schedule equivalence give rise
to the notions of: 1. conflict serializability 2. view
serializability Note: We ignore operations other than read and
write instructions, and we assume that transactions may perform
arbitrary computations on data in local buffers in between reads
and writes. Our simplified schedules consist of only read and write
instructions.
Slide 19
Conflict Serializability Instructions l i and l j of
transactions T i and T j respectively, conflict if and only if
there exists some item Q accessed by both l i and l j, and at least
one of these instructions wrote Q. l i = read(Q), l j = read(Q). l
i and l j dont conflict. l i = read(Q), l j = read(Q). l i and l j
dont conflict. l i = read(Q), l j = write(Q). They conflict l i =
read(Q), l j = write(Q). They conflict l i = write(Q), l j =
read(Q). They conflict l i = write(Q), l j = write(Q). They
conflict Intuitively, a conflict between l i and l j forces a
(logical) temporal order between them. If l i and l j are
consecutive in a schedule and they do not conflict, their results
would remain the same even if they had been interchanged in the
schedule.
Slide 20
Conflict Serializability (Cont.) If a schedule S can be
transformed into a schedule S by a series of swaps of
non-conflicting instructions, we say that S and S are conflict
equivalent. We say that a schedule S is conflict serializable if it
is conflict equivalent to a serial schedule Example of a schedule
that is not conflict serializable: T 3 T 4 read(Q) write(Q)
write(Q) We are unable to swap instructions in the above schedule
to obtain either the serial schedule, or the serial schedule.
Slide 21
Conflict Serializability (Cont.) Schedule 3 below can be
transformed into Schedule 1, a serial schedule where T 2 follows T
1, by series of swaps of non-conflicting instructions. Therefore
Schedule 3 is conflict serializable.
Slide 22
Testing for Serializability Consider some schedule of a set of
transactions T 1, T 2,..., T n Precedence graph: a direct graph
where the vertices are the transactions (names). We draw an arc
from T i to T j if the two transaction conflict, and T i accessed
the data item on which the conflict arose earlier. We may label the
arc by the item that was accessed. Example: A B
Slide 23
Example: Schedule A T 1 T 2 T 3 T 4 T 5 read(X) read(Y) read(Z)
read(V) read(W) read(W) read(Y) write(Y) write(Z) read(U) read(Y)
write(Y) read(Z) write(Z) read(U) write(U)
Slide 24
Example: Schedule A T 1 T 2 T 3 T 4 T 5 read(X) read(Y) read(Z)
read(V) read(W) read(W) read(Y) write(Y) write(Z) read(U) read(Y)
write(Y) read(Z) write(Z) read(U) write(U)
Slide 25
Precedence Graph for Schedule A T3T3 T4T4 T1T1 T2T2
Slide 26
Test for Conflict Serializability A schedule is conflict
serializable if and only if its precedence graph is acyclic.
Cycle-detection algorithms exist which take order n 2 time, where n
is the number of vertices in the graph. (Better algorithms take
order n + e where e is the number of edges.) If precedence graph is
acyclic, the serializability order can be obtained by a topological
sorting of the graph. This is a linear order consistent with the
partial order of the graph. For example, a serializability order
for Schedule A would be T 5 T 1 T 3 T 2 T 4.
Slide 27
Concurrency Control vs. Serializability Tests Tests for
serializability help understand why a concurrency control protocol
is correct. Testing a schedule for serializability after it has
executed is a little too late! Goal: to develop concurrency control
protocols that will assure serializability. They will generally not
examine the precedence graph as it is being created They will
generally not examine the precedence graph as it is being created
Instead, a protocol will impose a discipline that avoids
nonseralizable schedules. Instead, a protocol will impose a
discipline that avoids nonseralizable schedules.
Slide 28
View Serializability Let S and S be two schedules with the same
set of transactions. S and S are view equivalent if the following
three conditions are met: 1.For each data item Q, if transaction T
i reads the initial value of Q in schedule S, then transaction T i
must, in schedule S, also read the initial value of Q. 2.For each
data item Q if transaction T i executes read(Q) in schedule S, and
that value was produced by transaction T j (if any), then
transaction T i must in schedule S also read the value of Q that
was produced by transaction T j. 3.For each data item Q, the
transaction (if any) that performs the final write(Q) operation in
schedule S must perform the final write(Q) operation in schedule S.
As can be seen, view equivalence is also based purely on reads and
writes alone.
Slide 29
View Serializability (Cont.) A schedule S is view serializable
if it is view equivalent to a serial schedule. Every conflict
serializable schedule is also view serializable. Schedule 9 (from
text) a schedule which is view-serializable but not conflict
serializable. Every view serializable schedule that is not conflict
serializable has blind writes.
Slide 30
Test for View Serializability The precedence graph test for
conflict serializability must be modified to apply to a test for
view serializability. The problem of checking if a schedule is view
serializable falls in the class of NP-complete problems. Thus
existence of an efficient algorithm is unlikely. However practical
algorithms that just check some sufficient conditions for view
serializability can still be used.
Slide 31
Other Notions of Serializability Schedule 8 (from text) given
below produces same outcome as the serial schedule, yet is not
conflict equivalent or view equivalent to it. Determining such
equivalence requires analysis of operations other than read and
write.
Slide 32
Recoverability Recoverable schedule: if a transaction T j reads
a data item previously written by a transaction T i, the commit
operation of T i appears before the commit operation of T j. The
following schedule (Schedule 11) is not recoverable if T 9 commits
immediately after the read If T 8 should abort, T 9 would have read
(and possibly shown to the user) an inconsistent database state.
Hence database must ensure that schedules are recoverable. Need to
address the effect of transaction failures on concurrently running
transactions.
Slide 33
Cascading rollback A single transaction failure leads to a
series of transaction rollbacks. Consider the following schedule
where none of the transactions has yet committed (so the schedule
is recoverable) If T 10 fails, T 11 and T 12 must also be rolled
back. Can lead to the undoing of a significant amount of work
Slide 34
Cascadeless schedules Cascading rollbacks cannot occur; for
each pair of transactions T i and T j such that T j reads a data
item previously written by T i, the commit operation of T i appears
before the read operation of T j. Every cascadeless schedule is
also recoverable It is desirable to restrict the schedules to those
that are cascadeless
Slide 35
Implementation of Isolation Schedules must be conflict or view
serializable, and recoverable, for the sake of database
consistency, and preferably cascadeless. A policy in which only one
transaction can execute at a time generates serial schedules, but
provides a poor degree of concurrency. A policy in which only one
transaction can execute at a time generates serial schedules, but
provides a poor degree of concurrency. Concurrency-control schemes
tradeoff between the amount of concurrency they allow and the
amount of overhead that they incur. Concurrency-control schemes
tradeoff between the amount of concurrency they allow and the
amount of overhead that they incur. Some schemes allow only
conflict-serializable schedules to be generated, while others allow
view-serializable schedules that are not conflict-
serializable.
Slide 36
Transaction Definition in SQL Data manipulation language must
include a construct for specifying the set of actions that comprise
a transaction. In SQL, a transaction begins implicitly. A
transaction in SQL ends by: Commit work commits current transaction
and begins a new one. Commit work commits current transaction and
begins a new one. Rollback work causes current transaction to
abort. Rollback work causes current transaction to abort. Levels of
consistency specified by SQL-92: Serializable default Serializable
default Repeatable read Repeatable read Read committed Read
committed Read uncommitted Read uncommitted
Slide 37
Levels of Consistency in SQL-92 Serializable (default)
Repeatable read: only committed records to be read, repeated reads
of same record must return same value. However, a transaction may
not be serializable it may find some records inserted by a
transaction but not find others (Phantom problem allowed) Read
committed: only committed records can be read, but successive reads
of record may return different (but committed) values.
(unrepeatable read allowed) Read uncommitted: even uncommitted
records may be read. (dirty read allowed)
Slide 38
Chapter 16 (part): Concurrency Control Lock-based protocols
Timestamp-based protocols Deadlock handling Insert and delete
operations Index locking protocols
Slide 39
Lock-Based Protocols Lock is a mechanism to control concurrent
access to a data item Data items can be locked in two modes :
exclusive (X) mode. Data item can be both read as well as written.
X-lock is requested using lock-X instruction. exclusive (X) mode.
Data item can be both read as well as written. X-lock is requested
using lock-X instruction. shared (S) mode. Data item can only be
read. S-lock is requested using lock-S instruction. shared (S)
mode. Data item can only be read. S-lock is requested using lock-S
instruction. Lock requests are made to concurrency-control manager.
Transaction can proceed only after request is granted.
Slide 40
Lock-compatibility matrix A transaction may be granted a lock
on an item if the requested lock is compatible with locks already
held on the item by other transactions Any number of transactions
can hold shared locks on an item, but if any transaction holds an
exclusive on the item no other transaction may hold any lock on the
item. If a lock cannot be granted, the requesting transaction is
made to wait till all incompatible locks held by other transactions
have been released. The lock is then granted.
Slide 41
Example T 1 : lock-X(B); read (B); read (B); B:=B-50; B:=B-50;
write(B); write(B); unlock(B); unlock(B); lock-X(A); lock-X(A);
read (A); read (A); A:=A+50; A:=A+50; write(A); write(A);
unlock(A); unlock(A); T 2 : lock-S(A); read (A); read (A);
unlock(A); unlock(A); lock-S(B); lock-S(B); read (B); read (B);
unlock(B); unlock(B); display(A+B) display(A+B) Locking as above is
not sufficient to guarantee serializability if A and B get updated
in-between the read of A and B, the displayed sum would be wrong. A
locking protocol is a set of rules followed by all transactions
while requesting and releasing locks. Locking protocols restrict
the set of possible schedules.
Slide 42
Pitfalls of Lock-Based Protocols Consider the partial schedule
Neither T 3 nor T 4 can make progress: executing lock-S(B) causes T
4 to wait for T 3 to release its lock on B, while executing
lock-X(A) causes T 3 to wait for T 4 to release its lock on A. Such
a situation is called a deadlock. To handle a deadlock one of T 3
or T 4 must be rolled back and its locks released. To handle a
deadlock one of T 3 or T 4 must be rolled back and its locks
released. The potential for deadlock exists in most locking
protocols. Deadlocks are a necessary evil.
Slide 43
Pitfalls of Lock-Based Protocols (Cont.) Starvation is also
possible if concurrency control manager is badly designed. For
example: A transaction may be waiting for an X-lock on an item,
while a sequence of other transactions request and are granted an
S-lock on the same item. A transaction may be waiting for an X-lock
on an item, while a sequence of other transactions request and are
granted an S-lock on the same item. The same transaction is
repeatedly rolled back due to deadlocks. The same transaction is
repeatedly rolled back due to deadlocks. Concurrency control
manager can be designed to prevent starvation. Check if there is no
other transaction holding a lock on the data item nor a transaction
that is waiting for a lock on the data item and that made its
request before. Check if there is no other transaction holding a
lock on the data item nor a transaction that is waiting for a lock
on the data item and that made its request before.
Slide 44
The Two-Phase Locking Protocol Phase 1: Growing Phase
transaction may obtain locks transaction may obtain locks
transaction cannot release locks transaction cannot release locks
Phase 2: Shrinking Phase transaction may release locks transaction
may release locks transaction cannot obtain locks transaction
cannot obtain locks Ensures conflict-serializable schedules. The
protocol assures serializability. It can be proved that the
transactions can be serialized in the order of their lock points
(i.e. the point where a transaction acquired its final lock).
Slide 45
The Two-Phase Locking Protocol (Cont.) Two-phase locking does
not avoid deadlocks Cascading roll-back is possible under two-phase
locking. To avoid this, follow a modified protocol called strict
two-phase locking. Here, a transaction must hold all its exclusive
locks till it commits/aborts. Rigorous two-phase locking is even
stricter: here all locks are held till commit/abort. transactions
can be serialized in the order in which they commit. transactions
can be serialized in the order in which they commit.
Slide 46
Lock Conversions Two-phase locking with lock conversions: First
Phase: First Phase: can acquire a lock-S on item can acquire a
lock-S on item can acquire a lock-X on item can acquire a lock-X on
item can convert a lock-S to a lock-X (upgrade) can convert a
lock-S to a lock-X (upgrade) Second Phase: Second Phase: can
release a lock-S can release a lock-S can release a lock-X can
release a lock-X can convert a lock-X to a lock-S (downgrade) can
convert a lock-X to a lock-S (downgrade) This protocol assures
serializability. But still relies on the programmer to insert the
various locking instructions.
Slide 47
Automatic Acquisition of Locks A transaction T i issues the
standard read/write instruction, without explicit locking calls.
The operation read(D) is processed as: if T i has a lock on D if T
i has a lock on D then then read(D) read(D) else else begin begin
if necessary wait until no other if necessary wait until no other
transaction has a lock-X on D transaction has a lock-X on D grant T
i a lock-S on D; grant T i a lock-S on D; read(D) read(D) end
end
Slide 48
Automatic Acquisition of Locks (Cont.) write(D) is processed
as: if T i has a lock-X on D if T i has a lock-X on D then then
write(D) write(D) else else begin begin if necessary wait until no
other transaction has any lock on D, if necessary wait until no
other transaction has any lock on D, if T i has a lock-S on D if T
i has a lock-S on D then then upgrade lock on D to lock-X upgrade
lock on D to lock-X else else grant T i a lock-X on D grant T i a
lock-X on D write(D) write(D) end; end; All locks are released
after commit or abort
Slide 49
Implementation of Locking A Lock manager can be implemented as
a separate process to which transactions send lock and unlock
requests The lock manager replies to a lock request by sending a
lock grant messages (or a message asking the transaction to roll
back, in case of a deadlock) The requesting transaction waits until
its request is answered The lock manager maintains a data structure
called a lock table to record granted locks and pending requests
usually implemented as an in-memory hash table indexed on the name
of the data item being locked usually implemented as an in-memory
hash table indexed on the name of the data item being locked
Slide 50
Lock Table Black rectangles indicate granted locks, white ones
indicate waiting requests Lock table also records the type of lock
granted or requested New request is added to the end of the queue
of requests for the data item, and granted if it is compatible with
all earlier locks Unlock requests result in the request being
deleted, and later requests are checked to see if they can now be
granted If transaction aborts, all waiting or granted requests of
the transaction are deleted lock manager may keep a list of locks
held by each transaction, to implement this efficiently lock
manager may keep a list of locks held by each transaction, to
implement this efficiently
Slide 51
Deadlock Handling Consider the following two transactions: T 1
: write (X) T 2 : write(Y) T 1 : write (X) T 2 : write(Y) write(Y)
write(X) write(Y) write(X) Schedule with deadlock T1T1 T2T2 lock-X
on X write (X) lock-X on Y write (Y) wait for lock-X on X wait for
lock-X on Y
Slide 52
Deadlock Prevention Ensure that the system will never enter
into a deadlock state. Some prevention strategies : Require that
each transaction locks all its data items before it begins
execution (predeclaration). Require that each transaction locks all
its data items before it begins execution (predeclaration). Impose
partial ordering of all data items and require that a transaction
can lock data items only in the order specified by the partial
order (graph-based protocol). Impose partial ordering of all data
items and require that a transaction can lock data items only in
the order specified by the partial order (graph-based
protocol).
Slide 53
More Deadlock Prevention Strategies Based on preemption and
transaction rollbacks Use transaction timestamps: wait-die scheme
non-preemptive wait-die scheme non-preemptive Older transaction may
wait for younger one to release data item. Younger transactions
never wait for older ones; they are rolled back instead.Older
transaction may wait for younger one to release data item. Younger
transactions never wait for older ones; they are rolled back
instead. A transaction may die several times before acquiring
needed data itemA transaction may die several times before
acquiring needed data item wound-wait scheme preemptive wound-wait
scheme preemptive Older transaction wounds (forces rollback) of
younger transaction instead of waiting for it. Younger transactions
may wait for older ones.Older transaction wounds (forces rollback)
of younger transaction instead of waiting for it. Younger
transactions may wait for older ones. May be fewer rollbacks than
wait-die scheme.May be fewer rollbacks than wait-die scheme.
Slide 54
Timeout-based schemes Both in wait-die and in wound-wait
schemes, a rolled back transaction is restarted with its original
timestamp. Older transactions thus have precedence over newer ones
starvation is hence avoided. starvation is hence avoided. but
unnnecessary rollbacks may occur but unnnecessary rollbacks may
occur Timeout-Based Schemes : a transaction waits for a lock only
for a specified amount of time. After that, the wait times out and
the transaction is rolled back. a transaction waits for a lock only
for a specified amount of time. After that, the wait times out and
the transaction is rolled back. deadlocks are not possible
deadlocks are not possible simple to implement; but starvation is
possible. Also difficult to determine good value of the timeout
interval. simple to implement; but starvation is possible. Also
difficult to determine good value of the timeout interval.
Slide 55
Deadlock Detection Deadlocks can be described as a wait-for
graph, which consists of a pair G = (V,E), V is a set of vertices
(all the transactions in the system) V is a set of vertices (all
the transactions in the system) E is a set of edges; each element
is an ordered pair T i T j. E is a set of edges; each element is an
ordered pair T i T j. If T i T j is in E, then there is a directed
edge from T i to T j, implying that T i is waiting for T j to
release a data item. When T i requests a data item currently being
held by T j, then the edge T i T j is inserted in the wait-for
graph. When T i requests a data item currently being held by T j,
then the edge T i T j is inserted in the wait-for graph. This edge
is removed only when T j is no longer holding a data item needed by
T i. This edge is removed only when T j is no longer holding a data
item needed by T i. The system is in a deadlock state if and only
if the wait-for graph has a cycle. Must invoke a deadlock-detection
algorithm periodically to look for cycles. Must invoke a
deadlock-detection algorithm periodically to look for cycles.
Slide 56
Deadlock Detection (Cont.) Wait-for graph without a cycle
Wait-for graph with a cycle
Slide 57
Deadlock Recovery When deadlock is detected : Some transaction
will have to rolled back (made a victim) to break deadlock. Select
that transaction as victim that will incur minimum cost. Some
transaction will have to rolled back (made a victim) to break
deadlock. Select that transaction as victim that will incur minimum
cost. Rollback: determine how far to roll back transaction
Rollback: determine how far to roll back transaction Total
rollback: Abort the transaction and then restart it.Total rollback:
Abort the transaction and then restart it. More effective to roll
back transaction only as far as necessary to break deadlock.More
effective to roll back transaction only as far as necessary to
break deadlock. Starvation happens if same transaction is always
chosen as victim. Include the number of rollbacks in the cost
factor to avoid starvation Starvation happens if same transaction
is always chosen as victim. Include the number of rollbacks in the
cost factor to avoid starvation
Slide 58
Chapter 17: Recovery System Failure Classification Storage
Structure Recovery and Atomicity Log-Based Recovery
Slide 59
Failure Classification Transaction failure : Logical errors:
transaction cannot complete due to some internal error condition
Logical errors: transaction cannot complete due to some internal
error condition System errors: the database system must terminate
an active transaction due to an error condition (e.g., deadlock)
System errors: the database system must terminate an active
transaction due to an error condition (e.g., deadlock) System
crash: a power failure or other hardware or software failure causes
the system to crash. Fail-stop assumption: non-volatile storage
contents are assumed not be corrupted by system crash Fail-stop
assumption: non-volatile storage contents are assumed not be
corrupted by system crash Database systems have numerous integrity
checks to prevent corruption of disk dataDatabase systems have
numerous integrity checks to prevent corruption of disk data Disk
failure: a head crash or similar disk failure destroys all or part
of disk storage Destruction is assumed to be detectable: disk
drives use checksums to detect failures Destruction is assumed to
be detectable: disk drives use checksums to detect failures
Slide 60
Recovery Algorithms Techniques to ensure database consistency
and transaction atomicity and durability despite failures Recovery
algorithms have two parts: 1. Actions taken during normal
transaction processing to ensure enough information exists to
recover from failures 2. Actions taken after a failure to recover
the database contents to a state that ensures atomicity,
consistency and durability
Slide 61
Storage Structure Volatile storage: does not survive system
crashes does not survive system crashes examples: main memory,
cache memory examples: main memory, cache memory Nonvolatile
storage: survives system crashes survives system crashes examples:
disk, tape, flash memory, non-volatile (battery backed up) RAM
examples: disk, tape, flash memory, non-volatile (battery backed
up) RAM Stable storage: a mythical form of storage that survives
all failures a mythical form of storage that survives all failures
approximated by maintaining multiple copies on distinct nonvolatile
media approximated by maintaining multiple copies on distinct
nonvolatile media
Slide 62
Stable-Storage Implementation Maintain multiple copies of each
block on separate disks that can be at remote sites to protect
against disasters such as fire or flooding Failure during data
transfer can still result in inconsistent copies, because block
transfer can result in: Successful completion Successful completion
Partial failure: destination block has incorrect information
Partial failure: destination block has incorrect information Total
failure: destination block was never updated Total failure:
destination block was never updated
Slide 63
Protecting storage media from failure during data transfer
Execute output operation as follows (assuming two copies of each
block): 1.Write the information onto the first physical block.
2.When the first write successfully completes, write the same
information onto the second physical block. 3.The output is
completed only after the second write successfully completes.
Slide 64
Recovery from failures Copies of a block may differ due to
failure during output operation. To recover from failure: 1. First
find inconsistent blocks: Expensive solution: Compare the two
copies of every disk block.Expensive solution: Compare the two
copies of every disk block. Better solution:Better solution: n
Record in-progress disk writes on non-volatile storage (Non-
volatile RAM or special area of disk). n Use this information
during recovery to find blocks that may be inconsistent, and only
compare copies of these. n Used in hardware RAID systems 2. If
either copy of an inconsistent block is detected to have an error
(bad checksum), overwrite it by the other copy. If both have no
error, but are different, overwrite the second block by the first
block.
Slide 65
Data Access The database resides permanently on nonvolatile
storage (disks) and is partitioned into fixed-length storage units
called blocks Blocks are the units of transfer Blocks are the units
of transfer Physical blocks are those blocks residing on the disk.
Physical blocks are those blocks residing on the disk. Buffer
blocks are the blocks residing temporarily in main memory. Buffer
blocks are the blocks residing temporarily in main memory. Block
movements between disk and main memory are initiated through the
following two operations: input(B) transfers the physical block B
to main memory. input(B) transfers the physical block B to main
memory. output(B) transfers the buffer block B to the disk, and
replaces the appropriate physical block there. output(B) transfers
the buffer block B to the disk, and replaces the appropriate
physical block there. We assume, for simplicity, that each data
item fits in, and is stored inside, a single block.
Slide 66
Transactions data access (1) Each transaction T i has its
private work-area in which local copies of all data items accessed
and updated by it are kept. T i 's local copy of a data item X is
called x i. T i 's local copy of a data item X is called x i.
Transaction transfers data items between system buffer blocks and
its private work-area using the following operations : read(X)
assigns the value of data item X to the local variable x i. read(X)
assigns the value of data item X to the local variable x i.
write(X) assigns the value of local variable x i to data item {X}
in the buffer block. write(X) assigns the value of local variable x
i to data item {X} in the buffer block. both these commands may
necessitate the issue of an input(B X ) instruction before the
assignment, if the block B X in which X resides is not already in
memory.both these commands may necessitate the issue of an input(B
X ) instruction before the assignment, if the block B X in which X
resides is not already in memory.
Slide 67
Transactions data access (2) Transactions Perform read(X) while
accessing X for the first time; Perform read(X) while accessing X
for the first time; All subsequent accesses are to the local copy.
All subsequent accesses are to the local copy. After last access,
transaction executes write(X). After last access, transaction
executes write(X). output(B X ) need not immediately follow
write(X). System can perform the output operation when it deems
fit.
Slide 68
Example of Data Access x Y A B x1x1 y1y1 buffer Buffer Block A
Buffer Block B input(A) output(B) read(X) write(Y) disk work area
of T 1 work area of T 2 memory x2x2
Slide 69
Recovery and Atomicity (1) Modifying the database without
ensuring that the transaction will commit may leave the database in
an inconsistent state. Consider transaction T i that transfers $50
from account A to account B; goal is either to perform all database
modifications made by T i or none at all. Several output operations
may be required for T i (to output A and B). A failure may occur
after one of these modifications have been made but before all of
them are made.
Slide 70
Recovery and Atomicity (2) To ensure atomicity despite
failures, we first output information describing the modifications
to stable storage without modifying the database itself. Two
approaches: log-based recovery log-based recovery shadow-paging
shadow-paging We assume (initially) that transactions run serially,
that is, one after the other.
Slide 71
Log-Based Recovery A log is kept on stable storage. The log is
a sequence of log records, and maintains a record of update
activities on the database. The log is a sequence of log records,
and maintains a record of update activities on the database. When
transaction T i starts, it registers itself by writing a log record
Before T i executes write(X), a log record is written, where V 1 is
the value of X before the write, and V 2 is the value to be written
to X. Log record notes that T i has performed a write on data item
X j X j had value V 1 before the write, and will have value V 2
after the write. Log record notes that T i has performed a write on
data item X j X j had value V 1 before the write, and will have
value V 2 after the write. When T i finishes its last statement,
the log record is written; in case It fails, writes When T i
finishes its last statement, the log record is written; in case It
fails, writes We assume for now that log records are written
directly to stable storage (that is, they are not buffered) Two
approaches using logs: Deferred database modification Deferred
database modification Immediate database modification Immediate
database modification
Slide 72
Deferred Database Modification Records all modifications to the
log, but defers all the writes to after partial commit. Assume that
transactions execute serially Transaction starts by writing record
to log. A write(X) operation results in a log record being written,
where V is the new value for X Note: old value is not needed for
this scheme Note: old value is not needed for this scheme The write
is not performed on X at this time, but is deferred. When T i
partially commits, is written to the log Finally, the log records
are read and used to actually execute the previously deferred
writes.
Slide 73
Recovery w/ Deferred Database Modification During recovery
after a crash, a transaction needs to be redone if and only if both
and are there in the log. Redoing a transaction T i ( redoT i )
sets the value of all data items updated by the transaction to the
new values. Crashes can occur while : the transaction is executing
the original updates, or the transaction is executing the original
updates, or while recovery action is being taken while recovery
action is being taken Example: transactions T 0 and T 1 (T 0
executes before T 1 ): T 0 : read (A) T 1 : read (C) T 0 : read (A)
T 1 : read (C) A: - A - 50 C:-C- 100 Write (A) write (C) read (B)
B:- B + 50 write (B)
Slide 74
Deferred Database Modification (Cont.) Below we show the log as
it appears at three instances of time. If log on stable storage at
time of crash is as in case: (a) No redo actions need to be taken
(b) redo(T 0 ) must be performed since is present (c) redo(T 0 )
must be performed followed by redo(T 1 ) since and are present and
are present
Slide 75
Immediate Database Modification Allows database updates of an
uncommitted transaction to be made as the writes are issued since
undoing may be needed, update logs must have both old value and new
value since undoing may be needed, update logs must have both old
value and new value Update log record must be written before
database item is written We assume that the log record is output
directly to stable storage We assume that the log record is output
directly to stable storage Can be extended to postpone log record
output, so long as prior to execution of an output(B) operation for
a data block B, all log records corresponding to items B must be
flushed to stable storage Can be extended to postpone log record
output, so long as prior to execution of an output(B) operation for
a data block B, all log records corresponding to items B must be
flushed to stable storage Output of updated blocks can take place
at any time before or after transaction commit Order in which
blocks are output can be different from the order in which they are
written.
Slide 76
Example Log Write Output T o, B, 2000, 2050 A = 950 A = 950 B =
2050 B = 2050 C = 600 C = 600 B B, B C B B, B C B A B A Note: B X
denotes block containing X.
Slide 77
Recovery w/ Immediate Database Modification Recovery procedure
has two operations instead of one: undo(T i ) restores the value of
all data items updated by T i to their old values, going backwards
from the last log record for T i undo(T i ) restores the value of
all data items updated by T i to their old values, going backwards
from the last log record for T i redo(T i ) sets the value of all
data items updated by T i to the new values, going forward from the
first log record for T i Both operations must be idempotent That
is, even if the operation is executed multiple times the effect is
the same as if it is executed once That is, even if the operation
is executed multiple times the effect is the same as if it is
executed once Needed since operations may get re-executed during
recoveryNeeded since operations may get re-executed during recovery
When recovering after failure: Transaction T i needs to be undone
if the log contains the record, but does not contain the record.
Transaction T i needs to be undone if the log contains the record,
but does not contain the record. Transaction T i needs to be redone
if the log contains both the record and the record. Transaction T i
needs to be redone if the log contains both the record and the
record. Undo operations are performed first, then redo
operations.
Slide 78
Immediate DB Modification Recovery Example Below we show the
log as it appears at three instances of time. Below we show the log
as it appears at three instances of time. Recovery actions in each
case above are: (a) undo (T 0 ): B is restored to 2000 and A to
1000. (b) undo (T 1 ) and redo (T 0 ): C is restored to 700, and
then A and B are set to 950 and 2050 respectively. set to 950 and
2050 respectively. (c) redo (T 0 ) and redo (T 1 ): A and B are set
to 950 and 2050 respectively. Then C is set to 600 respectively.
Then C is set to 600
Slide 79
Checkpoints Problems in recovery procedure as discussed earlier
1. searching the entire log is time-consuming 2. we might
unnecessarily redo transactions which have already output their
updates to the database. Streamline recovery procedure by
periodically performing checkpointing 1. Output all log records
currently residing in main memory onto stable storage. 2. Output
all modified buffer blocks to the disk. 3. Write a log record onto
stable storage.
Slide 80
Recovery w/ Checkpoints During recovery we need to consider
only the most recent transaction T i that started before the
checkpoint, and transactions that started after T i. 1. Scan
backwards from end of log to find the most recent record 2.
Continue scanning backwards till a record is found. 3. Need only
consider the part of log following above start record. Earlier part
of log can be ignored during recovery, and can be erased whenever
desired. 4. For all transactions (starting from T i or later) with
no, execute undo(T i ). (Done only in case of immediate
modification.) 5. Scanning forward in the log, for all transactions
starting from T i or later with a, execute redo(T i ).
Slide 81
Example of Checkpoints T 1 can be ignored (updates already
output to disk due to checkpoint) T 2 and T 3 redone. T 4 undone
TcTc TfTf T1T1 T2T2 T3T3 T4T4 checkpoint system failure
Slide 82
Recovery w/ Conc.Transactions (1) We modify the log-based
recovery schemes to allow multiple transactions to execute
concurrently. All transactions share a single disk buffer and a
single log All transactions share a single disk buffer and a single
log A buffer block can have data items updated by one or more
transactions A buffer block can have data items updated by one or
more transactions We assume concurrency control using strict 2PL
i.e. the updates of uncommitted transactions should not be visible
to other transactions i.e. the updates of uncommitted transactions
should not be visible to other transactions Otherwise how to
perform undo if T1 updates A, then T2 updates A and commits, and
finally T1 has to abort?Otherwise how to perform undo if T1 updates
A, then T2 updates A and commits, and finally T1 has to abort?
Logging is done as described earlier. Log records of different
transactions may be interspersed in the log. Log records of
different transactions may be interspersed in the log. The
checkpointing technique and actions taken on recovery have to be
changed since several transactions may be active when a checkpoint
is performed. since several transactions may be active when a
checkpoint is performed.
Slide 83
Example of Recovery Go over the steps of the recovery algorithm
on the following log:
Slide 84
Recovery w/ Conc.Transactions (2) Checkpoints are performed as
before, except that the checkpoint log record is now of the form
where L is the list of transactions active at the time of the
checkpoint We assume no updates are in progress while the
checkpoint is carried out (will relax this later) We assume no
updates are in progress while the checkpoint is carried out (will
relax this later) When the system recovers from a crash, it first
does the following: 1.Initialize undo-list and redo-list to empty
2.Scan the log backwards from the end, stopping when the first
record is found. For each record found during the backward scan: if
the record is, add T i to redo-listif the record is, add T i to
redo-list if the record is, then if T i is not in redo-list, add T
i to undo-listif the record is, then if T i is not in redo-list,
add T i to undo-list 3.For every T i in L, if T i is not in
redo-list, add T i to undo-list
Slide 85
Recovery w/ Conc.Transactions (3) At this point undo-list
consists of incomplete transactions which must be undone, and
redo-list consists of finished transactions that must be redone.
Recovery now continues as follows: 4.Scan log backwards from most
recent record, stopping when records have been encountered for
every T i in undo-list. During the scan, perform undo for each log
record that belongs to a transaction in undo-list.During the scan,
perform undo for each log record that belongs to a transaction in
undo-list. 5.Locate the most recent record. 6.Scan log forwards
from the record till the end of the log. During the scan, perform
redo for each log record that belongs to a transaction on
redo-listDuring the scan, perform redo for each log record that
belongs to a transaction on redo-list
Slide 86
Log Record Buffering Log record buffering: log records are
buffered in main memory, instead of being output directly to stable
storage. Log records are output to stable storage when a block of
log records in the buffer is full, or a log force operation is
executed. Log records are output to stable storage when a block of
log records in the buffer is full, or a log force operation is
executed. Log force is performed to commit a transaction by forcing
all its log records (including the commit record) to stable
storage. Several log records can thus be output using a single
output operation, reducing the I/O cost. Several log records can
thus be output using a single output operation, reducing the I/O
cost.
Slide 87
Write-ahead logging The rules below must be followed if log
records are buffered: Log records are output to stable storage in
the order in which they are created. Log records are output to
stable storage in the order in which they are created. Transaction
T i enters the commit state only when the log record has been
output to stable storage. Transaction T i enters the commit state
only when the log record has been output to stable storage. Before
a block of data in main memory is output to the database, all log
records pertaining to data in that block must have been output to
stable storage. Before a block of data in main memory is output to
the database, all log records pertaining to data in that block must
have been output to stable storage. This rule is called the
write-ahead logging or WAL ruleThis rule is called the write-ahead
logging or WAL rule Strictly speaking WAL only requires undo
information to be output Strictly speaking WAL only requires undo
information to be output
Slide 88
Failure with Loss of Nonvolatile Storage So far we assumed no
loss of non-volatile storage Technique similar to checkpointing
used to deal with loss of non-volatile storage Periodically dump
the entire content of the database to stable storage Periodically
dump the entire content of the database to stable storage No
transaction may be active during the dump procedure; a procedure
similar to checkpointing must take place No transaction may be
active during the dump procedure; a procedure similar to
checkpointing must take place Output all log records currently
residing in main memory onto stable storage.Output all log records
currently residing in main memory onto stable storage. Output all
buffer blocks onto the disk.Output all buffer blocks onto the disk.
Copy the contents of the database to stable storage.Copy the
contents of the database to stable storage. Output a record to log
on stable storage.Output a record to log on stable storage. To
recover from disk failure To recover from disk failure restore
database from most recent dump.restore database from most recent
dump. Consult the log and redo all transactions that committed
after the dumpConsult the log and redo all transactions that
committed after the dump Can be extended to allow transactions to
be active during dump; known as fuzzy dump or online dump
Slide 89
Remote Backup Systems Remote backup systems provide high
availability by allowing transaction processing to continue even if
the primary site is destroyed.
Slide 90
Remote Backup Systems (Cont.) Detection of failure: Backup site
must detect when primary site has failed to distinguish primary
site failure from link failure maintain several communication links
between the primary and the remote backup. to distinguish primary
site failure from link failure maintain several communication links
between the primary and the remote backup. Transfer of control: To
take over control backup site first perform recovery using its copy
of the database and all the log records it has received from the
primary. To take over control backup site first perform recovery
using its copy of the database and all the log records it has
received from the primary. Thus, completed transactions are redone
and incomplete transactions are rolled back. Thus, completed
transactions are redone and incomplete transactions are rolled
back. When the backup site takes over processing it becomes the new
primary When the backup site takes over processing it becomes the
new primary To transfer control back to old primary when it
recovers, old primary must receive redo logs from the old backup
and apply all updates locally. To transfer control back to old
primary when it recovers, old primary must receive redo logs from
the old backup and apply all updates locally.
Slide 91
Remote Backup Systems (Cont.) Time to recover: To reduce delay
in takeover, backup site periodically proceses the redo log records
(in effect, performing recovery from previous database state),
performs a checkpoint, and can then delete earlier parts of the
log. Hot-Spare configuration permits very fast takeover: Backup
continually processes redo log record as they arrive, applying the
updates locally. Backup continually processes redo log record as
they arrive, applying the updates locally. When failure of the
primary is detected the backup rolls back incomplete transactions,
and is ready to process new transactions. When failure of the
primary is detected the backup rolls back incomplete transactions,
and is ready to process new transactions. Alternative to remote
backup: distributed database with replicated data Remote backup is
faster and cheaper, but less tolerant to failure Remote backup is
faster and cheaper, but less tolerant to failure more on this in
Chapter 19more on this in Chapter 19
Slide 92
Remote Backup Systems (Cont.) Ensure durability of updates by
delaying transaction commit until update is logged at backup; avoid
this delay by permitting lower degrees of durability. One-safe:
commit as soon as transactions commit log record is written at
primary One-safe: commit as soon as transactions commit log record
is written at primary Problem: updates may not arrive at backup
before it takes over.Problem: updates may not arrive at backup
before it takes over. Two-very-safe: commit when transactions
commit log record is written at primary and backup Two-very-safe:
commit when transactions commit log record is written at primary
and backup Reduces availability since transactions cannot commit if
either site fails.Reduces availability since transactions cannot
commit if either site fails. Two-safe: proceed as in two-very-safe
if both primary and backup are active. If only the primary is
active, the transaction commits as soon as is commit log record is
written at the primary. Two-safe: proceed as in two-very-safe if
both primary and backup are active. If only the primary is active,
the transaction commits as soon as is commit log record is written
at the primary. Better availability than two-very-safe; avoids
problem of lost transactions in one-safe.Better availability than
two-very-safe; avoids problem of lost transactions in
one-safe.
Slide 93
Shadow Paging
Slide 94
Shadow paging is an alternative to log-based recovery; this
scheme is useful if transactions execute serially Idea: maintain
two page tables during the lifetime of a transaction the current
page table, and the shadow page table Store the shadow page table
in nonvolatile storage, such that state of the database prior to
transaction execution may be recovered. Shadow page table is never
modified during execution Shadow page table is never modified
during execution To start with, both the page tables are identical.
Only current page table is used for data item accesses during
execution of the transaction. Whenever any page is about to be
written for the first time A copy of this page is made onto an
unused page. A copy of this page is made onto an unused page. The
current page table is then made to point to the copy The current
page table is then made to point to the copy The update is
performed on the copy The update is performed on the copy
Slide 95
Sample Page Table
Slide 96
Example of Shadow Paging Shadow and current page tables after
write to page 4
Slide 97
Shadow Paging (Cont.) To commit a transaction : 1. Flush all
modified pages in main memory to disk 1. Flush all modified pages
in main memory to disk 2. Output current page table to disk 2.
Output current page table to disk 3. Make the current page table
the new shadow page table, as follows: 3. Make the current page
table the new shadow page table, as follows: keep a pointer to the
shadow page table at a fixed (known) location on disk. keep a
pointer to the shadow page table at a fixed (known) location on
disk. to make the current page table the new shadow page table,
simply update the pointer to point to current page table on disk to
make the current page table the new shadow page table, simply
update the pointer to point to current page table on disk Once
pointer to shadow page table has been written, transaction is
committed. No recovery is needed after a crash new transactions can
start right away, using the shadow page table. Pages not pointed to
from current/shadow page table should be freed (garbage
collected).
Slide 98
Advantages/Inconvenients Advantages of shadow-paging over
log-based schemes no overhead of writing log records no overhead of
writing log records recovery is trivial recovery is trivial
Disadvantages : Copying the entire page table is very expensive
Copying the entire page table is very expensive Can be reduced by
using a page table structured like a B + -treeCan be reduced by
using a page table structured like a B + -tree No need to copy
entire tree, only need to copy paths in the tree that lead to
updated leaf nodes No need to copy entire tree, only need to copy
paths in the tree that lead to updated leaf nodes Commit overhead
is high even with above extension Commit overhead is high even with
above extension Need to flush every updated page, and page
tableNeed to flush every updated page, and page table Data gets
fragmented (related pages get separated on disk) Data gets
fragmented (related pages get separated on disk) After every
transaction completion, the database pages containing old versions
of modified data need to be garbage collected After every
transaction completion, the database pages containing old versions
of modified data need to be garbage collected Hard to extend
algorithm to allow transactions to run concurrently Hard to extend
algorithm to allow transactions to run concurrently Easier to
extend log based schemesEasier to extend log based schemes
Slide 99
Aries Algorithm
Slide 100
Write-Ahead Logging (WAL) The Write-Ahead Logging Protocol:
Must force the log record for an update before the corresponding
data page gets to disk. Must write all log records for a Xact
before commit. #1 guarantees Atomicity. #2 guarantees Durability.
Exactly how is logging (and recovery!) done? Well study the ARIES
algorithms. Well study the ARIES algorithms.
Slide 101
WAL & the Log Each log record has a unique Log Sequence
Number (LSN). LSNs always increasing. LSNs always increasing. Each
data page contains a pageLSN. The LSN of the most recent log record
for an update to that page. The LSN of the most recent log record
for an update to that page. System keeps track of flushedLSN. The
max LSN flushed so far. The max LSN flushed so far. WAL: Before a
page is written, pageLSN flushedLSN pageLSN flushedLSN LSNs DB
pageLSNs RAM flushedLSN pageLSN Log records flushed to disk Log
tail in RAMs
Slide 102
Log Records Possible log record types: Update Commit Abort End
(signifies end of commit or abort) Compensation Log Records (CLRs)
for UNDO actions for UNDO actions prevLSN XID type length pageID
offset before-image after-image LogRecord fields: update records
only
Slide 103
Data structures Transaction Table: One entry per active Xact.
One entry per active Xact. Contains XID, status
(running/commited/aborted), and lastLSN. Contains XID, status
(running/commited/aborted), and lastLSN. Dirty Page Table: One
entry per dirty page in buffer pool. One entry per dirty page in
buffer pool. Contains recLSN -- the LSN of the log record which
first caused the page to be dirty. Contains recLSN -- the LSN of
the log record which first caused the page to be dirty.
Slide 104
Checkpointing Periodically, the DBMS creates a checkpoint, in
order to minimize the time taken to recover in the event of a
system crash. Write to log: begin_checkpoint record: Indicates when
chkpt began. begin_checkpoint record: Indicates when chkpt began.
end_checkpoint record: Contains current Xact table and dirty page
table. This is a fuzzy checkpoint: end_checkpoint record: Contains
current Xact table and dirty page table. This is a fuzzy
checkpoint: Other Xacts continue to run; so these tables accurate
only as of the time of the begin_checkpoint record.Other Xacts
continue to run; so these tables accurate only as of the time of
the begin_checkpoint record. No attempt to force dirty pages to
disk; effectiveness of checkpoint limited by oldest unwritten
change to a dirty page. (So its a good idea to periodically flush
dirty pages to disk!)No attempt to force dirty pages to disk;
effectiveness of checkpoint limited by oldest unwritten change to a
dirty page. (So its a good idea to periodically flush dirty pages
to disk!) Store LSN of chkpt record in a safe place (master
record). Store LSN of chkpt record in a safe place (master
record).
Slide 105
The Big Picture: Whats Stored Where DB Data pages each with a
pageLSN Xact Table lastLSN status Dirty Page Table recLSN
flushedLSN RAM prevLSN XID type length pageID offset before-image
after-image LogRecords LOG master record
Slide 106
Crash Recovery: Big Picture v Start from a checkpoint (found
via master record). v Three phases. Need to: Figure out which Xacts
committed since checkpoint, and which failed (ANALYSIS). REDO all
actions. u (repeat history) UNDO effects of failed Xacts. Oldest
log rec. of Xact active at crash Smallest recLSN in dirty page
table after Analysis Last chkpt CRASH A RU
Slide 107
Recovery: The Analysis Phase Reconstruct state at checkpoint.
via end_checkpoint record. via end_checkpoint record. Scan log
forward from checkpoint. End record: Remove Xact from Xact table.
End record: Remove Xact from Xact table. Other records: Add Xact to
Xact table, set lastLSN=LSN, change Xact status on commit. Other
records: Add Xact to Xact table, set lastLSN=LSN, change Xact
status on commit. Update record: If P not in Dirty Page Table,
Update record: If P not in Dirty Page Table, Add P to D.P.T., set
its recLSN=LSN.Add P to D.P.T., set its recLSN=LSN.
Slide 108
Recovery: The REDO Phase We repeat History to reconstruct state
at crash: Reapply all updates (even of aborted Xacts!), redo CLRs.
Reapply all updates (even of aborted Xacts!), redo CLRs. Scan
forward from log rec containing smallest recLSN in D.P.T. For each
CLR or update log rec LSN, REDO the action unless: Affected page is
not in the Dirty Page Table, or Affected page is not in the Dirty
Page Table, or Affected page is in D.P.T., but has recLSN > LSN,
or Affected page is in D.P.T., but has recLSN > LSN, or pageLSN
(in DB) LSN. pageLSN (in DB) LSN. To REDO an action: Reapply logged
action. Reapply logged action. Set pageLSN to LSN. No additional
logging! Set pageLSN to LSN. No additional logging!
Slide 109
Recovery: The UNDO Phase ToUndo={ l | l a lastLSN of a loser
Xact} Repeat Choose largest LSN among ToUndo. Choose largest LSN
among ToUndo. If this LSN is a CLR and undonextLSN==NULL If this
LSN is a CLR and undonextLSN==NULL Write an End record for this
Xact.Write an End record for this Xact. If this LSN is a CLR, and
undonextLSN != NULL If this LSN is a CLR, and undonextLSN != NULL
Add undonextLSN to ToUndoAdd undonextLSN to ToUndo Else this LSN is
an update. Undo the update, write a CLR, add prevLSN to ToUndo.
Else this LSN is an update. Undo the update, write a CLR, add
prevLSN to ToUndo. Until ToUndo is empty
Example: Crash During Restart! begin_checkpoint, end_checkpoint
update: T1 writes P5 update T2 writes P3 T1 abort CLR: Undo T1 LSN
10, T1 End update: T3 writes P1 update: T2 writes P5 CRASH, RESTART
CLR: Undo T2 LSN 60 CLR: Undo T3 LSN 50, T3 end CRASH, RESTART CLR:
Undo T2 LSN 20, T2 end LSN LOG 00,05 10 20 30 40,45 50 60 70 80,85
90 Xact Table lastLSN status Dirty Page Table recLSN flushedLSN
ToUndo undonextLSN RAM
Slide 112
Recovering From a Crash There are 3 phases in the Aries
recovery algorithm: Analysis: Scan the log forward (from the most
recent checkpoint) to identify all Xacts that were active, and all
dirty pages in the buffer pool at the time of the crash. Analysis:
Scan the log forward (from the most recent checkpoint) to identify
all Xacts that were active, and all dirty pages in the buffer pool
at the time of the crash. Redo: Redoes all updates to dirty pages
in the buffer pool, as needed, to ensure that all logged updates
are in fact carried out and written to disk. Redo: Redoes all
updates to dirty pages in the buffer pool, as needed, to ensure
that all logged updates are in fact carried out and written to
disk. Undo: The writes of all Xacts that were active at the crash
are undone (by restoring the before value of the update, which is
in the log record for the update), working backwards in the log.
(Some care must be taken to handle the case of a crash occurring
during the recovery process!) Undo: The writes of all Xacts that
were active at the crash are undone (by restoring the before value
of the update, which is in the log record for the update), working
backwards in the log. (Some care must be taken to handle the case
of a crash occurring during the recovery process!)
Slide 113
Database Buffering Database maintains an in-memory buffer of
data blocks When a new block is needed, if buffer is full an
existing block needs to be removed from buffer When a new block is
needed, if buffer is full an existing block needs to be removed
from buffer If the block chosen for removal has been updated, it
must be output to disk If the block chosen for removal has been
updated, it must be output to disk As a result of the write-ahead
logging rule, if a block with uncommitted updates is output to
disk, log records with undo information for the updates are output
to the log on stable storage first. No updates should be in
progress on a block when it is output to disk. Can be ensured as
follows. Before writing a data item, transaction acquires exclusive
lock on block containing the data item Before writing a data item,
transaction acquires exclusive lock on block containing the data
item Lock can be released once the write is completed.Such locks
held for short duration are called latches. Lock can be released
once the write is completed.Such locks held for short duration are
called latches. Before a block is output to disk, the system
acquires an exclusive latch on the block thus ensuring no update
can be in progress on the block Before a block is output to disk,
the system acquires an exclusive latch on the block thus ensuring
no update can be in progress on the block