mirror of
https://github.com/janishutz/eth-summaries.git
synced 2026-07-27 21:29:09 +02:00
[DMDB] Add concurrency control and recovery
Oops, luckily still figured out that that was missing...
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
\subsection{Transactions}
|
||||
Transactions are accesses to the database with the purpose of \textit{modifying the data}.
|
||||
The focus is of course on correctness, recovery and state management.
|
||||
|
||||
Transactions typically are expensive compared to queries due to the focus on correctness.
|
||||
|
||||
This means we need to employ concurrency control (ensuring data remains consistent with transactions and queries running at the same time,
|
||||
definition of consistency, correctness, conflicts and problematic situations, as well as locking).
|
||||
We furthermore want to implement recovery, to ensure that data remains consistent even when unexpected failures occur.
|
||||
@@ -0,0 +1,44 @@
|
||||
\subsection{ACID}
|
||||
\label{sec:acid}
|
||||
\acrshort{acid} is the conventional notion of correctness (informal because it's an acronym):
|
||||
\begin{itemize}
|
||||
\item \bi{Atomicity}: A group of operations must take place in their entirety or not at all
|
||||
\item \bi{Consistency}: Operations should take the database from a correct state to another correct state
|
||||
\item \bi{Isolation}: Concurrent execution of operations should yield predictable and correct results
|
||||
\item \bi{Durability}: The DB needs to remember the state it is in at all moments, even when failures occur
|
||||
\end{itemize}
|
||||
|
||||
\inlinedefinition[Database state] is the actual values stored in a database at a given point in time (in memory and storage).
|
||||
|
||||
\inlinedefinition[Consistent state] is a database state that is the result of \textit{correctly} applying \textit{operations} to the DB.
|
||||
|
||||
\inlinedefinition[Correct application] The model of correct executions of operations is sequential executions.
|
||||
|
||||
\inlinedefinition[Operations] The operations for transactions are inserts, updates and deletion of tuples.
|
||||
|
||||
\inlinedefinition[Consistency] The assumption of correctness of transactions is based on two things:
|
||||
\bi{(1)} the user doesn't apply wrong transactions and \bi{(2)} the DBMS has mechanisms to prevent incorrect modifications, such as violations of constraints, etc.
|
||||
|
||||
\inlinedefinition[Durability] ensures that the changes of a completed transaction are remembered by the system and can be recovered.
|
||||
This is often more strict and DBMS will keep the DB st ate and a history of all transactions.
|
||||
|
||||
The transaction history is captured in the DB log. The state at a given moment is captured in snapshots.
|
||||
The log and snapshots allow to go back and forward in the history of the database by undoing or redoing transactions from a given snapshot.
|
||||
Thus, very similar in concept to what \texttt{git} uses.
|
||||
|
||||
|
||||
\subsubsection{Atomicity}
|
||||
\inlinedefinition[Atomicity] dictates that a transaction only takes effect if it can be executed in its entirety, or in other words,
|
||||
a transaction has to be executed in its entirety or not at all.
|
||||
The issue with this is that violations are costly to correct.
|
||||
|
||||
Thus, atomicity requires isolation. If a transaction can see the intermediate state created by another transaction, then its input is not guaranteed to be consistent,
|
||||
implying that its output is not guaranteed to be consistent, either.
|
||||
|
||||
The DBMS get around this issue by isolating the transactions, such that they behave as if they were alone in the database.
|
||||
This of course means that we need to know when a transaction is completed so we can \textit{commit} it, if there are no conflicts.
|
||||
If there are conflicts, we have to resolve them, rerun the transaction (note that this can be very expensive) or to prevent them altogether.
|
||||
|
||||
The isolation is enforced through concurrency control mechanisms to prevent conflicts.
|
||||
The canonical concurrency control mechanism in DBs is locking.
|
||||
We use locking policies to ensure that the transactions are isolated.
|
||||
@@ -0,0 +1,35 @@
|
||||
\subsection{Transaction Model}
|
||||
\inlinedefinition Some terms for Transaction model:
|
||||
\begin{itemize}
|
||||
\item \bi{Begin of transaction (BOT)}: Often implicit
|
||||
\item \bi{Commit}: Transaction has finished, database confirms to client when all changes of the transaction have been made persistent
|
||||
\item \bi{Abort/rollback}: transaction is cancelled, all changes made already are rolled back
|
||||
\item $a <_T b$: $a$ happens before $b$, is a \textit{partial order}, which implies that $a$ will be done before $b$.
|
||||
\end{itemize}
|
||||
|
||||
In real systems, transactions are often associated with sessions and all SQL statements after an update statement are considered to be part of the transaction.
|
||||
This means they are explicitly managed, using a \texttt{BEGIN TRANSACTION} or \texttt{BEGIN} statement and ended with either a \texttt{COMMIT} or \texttt{ROLLBACK} statement.
|
||||
|
||||
So, we have the following statements:
|
||||
\begin{itemize}
|
||||
\item \texttt{COMMIT}: Transaction will not make any order
|
||||
\item \texttt{ROLLBACK}: Roll back all changes
|
||||
\item \texttt{SAVEPOINT $<$name$>$}: Create a save point / history point, used to roll back some changes and try some others in case of failure.
|
||||
\item \texttt{ROLLBACK TO SAVEPOINT $<$name$>$}: Roll back to a specific save point.
|
||||
\item \texttt{RELEASE SAVEPOINT $<$name$>$}: Delete a save point (doesn't delete the changes made before, just the log entry)
|
||||
\end{itemize}
|
||||
|
||||
\subsubsection{Operations}
|
||||
Below a list of very important syntax for the exam (where the subscript $i$ denotes that this operation is part of transaction $T_i$):
|
||||
\begin{itemize}
|
||||
\item Read operations: $r_i[x]$ or $r_i(x)$ is an access to tuple $x$ without modifying it.
|
||||
\item Write operations: $w_i[x]$ or $r_i(x)$ represents a write operation on the tuple $x$.
|
||||
\item Abort: $a_j$ is an abort of transaction $T_j$
|
||||
\item Commit $c_j$ is the commit of transaction $T_j$
|
||||
\end{itemize}
|
||||
Conflicting operations are two operations on the same item with at least one of them being a write operation, such as $r_1[x] w_2[x]$ or $w_1[y] w_2[y]$.
|
||||
|
||||
\subsubsection{History}
|
||||
A history $H$ is a partially ordered $<_H$ sequence of operations from a set of transactions.
|
||||
If two operations are ordered within a transaction, they are equally ordered in the history.
|
||||
If two operations $p$ and $q$ conflict, then they are ordered with respect to each other ($p <_H q$ or $q <_H p$)
|
||||
@@ -0,0 +1,39 @@
|
||||
\newpage
|
||||
\subsection{Concurrency Control}
|
||||
The goal of concurrency control is to ensure the correct executions of concurrent transactions.
|
||||
The baseline for correctness is, as previously mentioned, serial execution.
|
||||
|
||||
\inlinedefinition[Serial History] A history $H$ is serial if, for every pair of two transactions $T_i$ and $T_j$ that appear in $H$,
|
||||
either all operations from $T_i$ appear before all all operations of $T_j$, or vice versa.
|
||||
A serial history with only committed transactions is correct by definition, because transactions are isolated and each transaction starts in a consistent state
|
||||
and leaves the DB in a consistent state.
|
||||
|
||||
\inlinedefinition[Equivalent History] Two histories are equivalent if and only if
|
||||
\begin{enumerate}
|
||||
\item They are over the same transactions and contain the same operations
|
||||
\item Conflicting operations of non-aborted transactions are ordered in the same way in both histories
|
||||
\end{enumerate}
|
||||
|
||||
\inlinedefinition[Serializable History] A history is serializable if and only it is equivalent to a serial history.
|
||||
|
||||
\inlinetheorem[Serializability] A history is serializable if and only if its serializability graph is acyclic.
|
||||
The seerializabililty graph is constructed from the history graph as follows:
|
||||
\begin{enumerate}
|
||||
\item The start node is the transaction the first operation.
|
||||
\item Then, follow the operations, adding any new transaction to the tree and connect any further dependencies.
|
||||
\end{enumerate}
|
||||
More visually, see the below graphics
|
||||
|
||||
\begin{figure}[h!]
|
||||
\begin{center}
|
||||
\begin{subfigure}{0.49\textwidth}
|
||||
\includegraphics[width=0.9\textwidth]{assets/serializability-graph-history.png}
|
||||
\caption{The history}
|
||||
\end{subfigure}
|
||||
\begin{subfigure}{0.49\textwidth}
|
||||
\includegraphics[width=0.7\textwidth]{assets/serializability-graph-built.png}
|
||||
\caption{The resulting serializability graph}
|
||||
\end{subfigure}
|
||||
\end{center}
|
||||
\caption{Building a serializability graph (Figure from lecture slides 19, slide 37 and 38)}
|
||||
\end{figure}
|
||||
@@ -0,0 +1,54 @@
|
||||
\subsection{2 Phase Locking (2PL)}
|
||||
This type of locking is widely used since many decades and is the canonical implementation of concurrency control.
|
||||
Of course, alternatives exist and the locking implementation has a huge impact on performance and scalability.
|
||||
|
||||
Concurrency control is implemented using locking, typically using a locking table.
|
||||
The locks cover tuples, tables, indexes, blocks and more.
|
||||
|
||||
Locks have different semantics that indicate whether they can be shared or are exclusive.
|
||||
|
||||
Compatibility matrix:
|
||||
\begin{tables}{cccc}{ & NL & S & X}
|
||||
S & \checkmark & \checkmark & - \\
|
||||
X & \checkmark & - & - \\
|
||||
\end{tables}
|
||||
|
||||
\subsubsection{The protocol}
|
||||
\begin{enumerate}
|
||||
\item Before accessing an object, a transaction must acquire lock
|
||||
\item A transaction acquires a lock only once. Lock upgrades are possible
|
||||
\item A transaction is blocked if the lock request cannot be granted according to the compatibility matrix
|
||||
\item A transaction goes through two phases:
|
||||
\begin{enumerate}
|
||||
\item Growth: Acquire locks, but never release locks
|
||||
\item Shrink: Release locks, but never acquire locks
|
||||
\end{enumerate}
|
||||
\item At EOT (commit or abort), all locks must be released.
|
||||
\end{enumerate}
|
||||
|
||||
The reason for the two phases is that 2PL prevents histories where a transaction must be aborted due to concurrency control issues.
|
||||
An operation is thus not allowed unless we are sure it is safe.
|
||||
|
||||
Without the two phases, the following can happen:
|
||||
$L_1[x] w_1[x] U_1[x] L_2[x] w_2[x] U_2[y] L_2[y] w_2[y]$, etc, thus meaning that if $T_1$ now access $y$ or $x$ for read or write, the history becomes non-serializable.
|
||||
|
||||
The basic 2PL doesn't say anything about aborting or committing transactions.
|
||||
The issue is that if we are to undo the changes of $T_1$, we would also undo some of the changes from $T_2$, which would then in turn not be restorable anymore.
|
||||
|
||||
\subsubsection{Strict 2PL}
|
||||
Strict 2PL combines 2PL with strict histories. This means that we enforce 2PL, but we also keep the read and write locks until the transaction commits or aborts.
|
||||
This results in serializable and strict histories.
|
||||
All DB engines implement strict 2PL.
|
||||
|
||||
|
||||
\subsubsection{Deadlocks Detection}
|
||||
In theory, we could use a wait-for graph, which is however expensive.
|
||||
For example, for $T_1 \rightarrow T_2 \rightarrow T_3 \rightarrow T_4 \rightarrow T_1$, we could abort $T_3$ to resolve the cycle.
|
||||
|
||||
In practice, no wait-for graphs are built. Transactions and queries have timers. If the timer expires, the transaction or query is aborted.
|
||||
Deadlocks are considered rare enough to prefer making mistakes every now and then (aborting a transaction that was fine),
|
||||
rather than implementing something as expensive as a wait-for graph.
|
||||
|
||||
Implications from this are for example that long transactions may block queries and that those queries may abort because the timer expired.
|
||||
|
||||
However, this is only common in poorly written applications.
|
||||
@@ -0,0 +1,17 @@
|
||||
\subsection{Snapshot isolation}
|
||||
This was initially used by Oracle and is a form of Multi-Version Concurrency Control (MVCC).
|
||||
|
||||
MVCC applies serializability, but writes generate a new version and reads read from a concrete version of the DB.
|
||||
This means that the whole DB can be implemented (almost) entirely without locks.
|
||||
|
||||
When a transaction starts, it receives a timestamp $T$.
|
||||
Read operations only have access to items committed before $T$.
|
||||
All write operations are carried out in a separate buffer (shadow paging) and only become visible after a commit.
|
||||
When a transaction commits, it checks for conflicts. If there are, the first-committer-wins rule is applied, i.e. if we try to commit and there are conflicts,
|
||||
this transaction has to rerun (or be aborted).
|
||||
|
||||
However, there are scenarios, such as $r_1[x] r_1[y] r_2[x] r_2[y] w_1[x] w_2[x] c_1 c_2$, where the history is not serializable:
|
||||
$T_2 \rightarrow T_1 \rightarrow T_2$.
|
||||
Depending on the application, this can still be correct (if this is for a ticketing system), but can also be disastrous (for e.g. a bank).
|
||||
|
||||
If done properly however, it has significant performance advantages, as well as in distributed settings with replication.
|
||||
@@ -0,0 +1,77 @@
|
||||
\subsection{Recovery}
|
||||
Recovery has to make sure that, even in the case of failures, the changes from transactions that have not completed are not visible
|
||||
and those of committed transactions are recorded and visible.
|
||||
We define undesirable situations and classify histories according to these situations.
|
||||
Furthermore, we want to define how histories can avoid such situations.
|
||||
|
||||
|
||||
\subsubsection{Recovery procedures}
|
||||
\begin{enumerate}[label=R\arabic*]
|
||||
\item Undo all changes from transactions
|
||||
\item Redo the changes from committed transactions
|
||||
\item Undo the changes that remain in the system from active transactions
|
||||
\item Read consistent snapshot from backup, and, if available, apply log
|
||||
\end{enumerate}
|
||||
R1 is used for abort/rollback of single transactions, R2 and R3 are used on system crash (where we lose the memory, but disks survive)
|
||||
and R4 is used for recovery from a system crash with disk loss.
|
||||
|
||||
This means that we can undo the changes of an active transaction (we keep a copy of the value before it was modified; Shadow pages is a mechanism that would support this),
|
||||
we can redo changes made by committed transactions (we keep a persistent record of all the changes made by committed transactions, this why there is a redo log)
|
||||
and we have a consistent snapshot to start with (either move forward by taking a snapshot and apply committed transactions,
|
||||
or move backward by taking the current state and go back to a consistent state by undoing changes from transactions that should be there).
|
||||
|
||||
|
||||
\subsubsection{Undo - Redo}
|
||||
The changes of an aborted transaction are undone typically by restoring the \bi{before image} of the modified value.
|
||||
|
||||
Changes are redone by restoring the \bi{after image} of the modified value.
|
||||
|
||||
Databases typically log transactions by keeping before and after images of their changes.
|
||||
|
||||
|
||||
\subsubsection{Recovery on histories}
|
||||
A transaction $T_1$ reads from another transaction $T_2$ if $T_1$ reads a value written by $T_2$ at a time when $T_2$ was not aborted.
|
||||
|
||||
We have:
|
||||
\begin{itemize}
|
||||
\item \bi{Recoverable} (RC) history: If $T_i$ reads from $T_j$ and commits, then $c_j < c_i$
|
||||
\item \bi{Avoids Cascading Aborts} (ACA) history: If $T_i$ reads $x$ from $T_j$, then $c_j < r_i[x]$
|
||||
\item \bi{Strict} (ST) history: If $T_i$ reads from, or overwrites, a value written by $T_j$ then $c_j < r_i[x] / w_i[x]$ or $a_j < r_i[x] / w_i[x]$
|
||||
\end{itemize}
|
||||
|
||||
|
||||
\paragraph{Recoverable}
|
||||
There is no need to undo a committed transaction because it read the wrong data and they commit in their serialization order.
|
||||
|
||||
Suppose we have the following sequence: $\ldots w_2[x] r_1[x] c_1 \ldots$.
|
||||
If $T_2$ aborts, then $T_1$ has read invalid data and since $T_1$ has committed, the data has returned to the user.
|
||||
Recovery now involves correcting the application that has processed the data and might have acted upon it, which typically is a very expensive procedure.
|
||||
This is avoided by making sure that $T_1$ does not commit until $T_2$ commits and if $T_2$ aborts, then we can safely abort $T_1$,
|
||||
since the results have not been returned to the user yet.
|
||||
|
||||
|
||||
\paragraph{ACA}
|
||||
Aborting a transaction does not cause aborting others and transactions only read from committed transactions
|
||||
|
||||
Suppose we have the following sequence: $\ldots w_2[x] r_1[x] a_2 \ldots$.
|
||||
When $T_2$ aborts, we have to abort $T_1$ because it has read invalid data (the change made by $T_2$, which will be undone as part of rollback of $T_2$).
|
||||
If those cascading aborts happen regularly, the performance will degrade and possibly further transactions will be aborted.
|
||||
This is avoided by making sure that uncommitted data is never read (\textit{read committed}).
|
||||
|
||||
|
||||
\paragraph{Strict}
|
||||
Undoing a transaction does not undo the changes of other transactions. Transactions further don't read or overwrite updates of uncommitted transactions.
|
||||
|
||||
Suppose we have the following sequence: $\ldots w_2[x] w_1[x] a_2 \ldots$.
|
||||
If we undo $w_2[x]$, we would also remove the changes made by $T_1$. Thus, undoing $T_2$ implies aborting $T_1$.
|
||||
Like in ACA, non-strict execution leads to cascading aborts of transactions.
|
||||
This is avoided by not letting a value be read or updated unless it is committed.
|
||||
|
||||
|
||||
\paragraph{Why recoverability matters}
|
||||
Fixing errors with recovery typically are very expensive:
|
||||
\begin{itemize}
|
||||
\item Not RC: running my program issues a report, but the data that I got was later removed because a different program aborted $\rightarrow$ report is invalid
|
||||
\item Not ACA: Thrashing behaviour when transactions keep aborting each other (my program slows down, or makes no progress at all because of other programs)
|
||||
\item Not Strict: recovery after a failure becomes very complex (or impossible)
|
||||
\end{itemize}
|
||||
@@ -0,0 +1,127 @@
|
||||
\subsection{Recovery Procedures}
|
||||
\inlinedefinition[Recovery Manager] A recovery manager implements the recovery procedure, which depends on how the system functions.
|
||||
It may or may not require both, one or none of UNDO and REDO.
|
||||
|
||||
\subsubsection{From the buffer cache's perspective}
|
||||
\begin{itemize}
|
||||
\item \bi{Steal Policy}: An uncommitted transaction is allowed to overwrite the changes of a committed transaction in persistent storage.
|
||||
This logically means that we need to be able to undo transactions.
|
||||
\item \bi{Force Policy}: All changes made by a transaction must be in persistent storage before the transaction commits.
|
||||
This requires flushing all blocks with updates from the transaction and if not in-place, it will require being able to redo the transaction.
|
||||
\item \bi{Steal/No-Force}: UNDO/REDO, is the most common approach
|
||||
\end{itemize}
|
||||
If for updates tuples are locked, blocks may still be updatable.
|
||||
Thus if the block is copied to storage, it is possible that some changes are committed while others are not.
|
||||
If failures occur, there is no guarantee that the data in storage is 100\% consistent, thus requiring undo and redo.
|
||||
|
||||
\subsubsection{Recovery Procedure for UNDO/REDO}
|
||||
\begin{itemize}
|
||||
\item READ: simply read the value from the block in the buffer cache
|
||||
\item WRITE: Create log entry (before and after image), append it to persistent log and write the after image to the block on the buffer cache.
|
||||
\item COMMIT: We write a persistent log entry indicating that the transaction has committed
|
||||
\item ABORT: For all updates, we restore the before image using the log entry
|
||||
\end{itemize}
|
||||
|
||||
The Procedure:
|
||||
\begin{enumerate}
|
||||
\item Start from the end of the log and work backwards
|
||||
\item Keep a list of UNDONE items and another for REDONE items
|
||||
\item Procedure terminates when all items are in either the UNDONE or REDONE list or we reach the beginning of the log.
|
||||
\item For each log entry, we look at the data item $x$ being accessed and if $x$ is in none of the two lists:
|
||||
\begin{itemize}
|
||||
\item If the log entry is of a committed transaction, apply the after image, add $x$ to the REDONE list
|
||||
\item If the log entry is of an aborted transaction, apply the before image, add $x$ to the UNDONE list
|
||||
\end{itemize}
|
||||
\end{enumerate}
|
||||
The procedure ignores the data stored on disk as it could correspond to uncommitted transactions, it only takes it as starting point for the recovery.
|
||||
An alternative way of thinking about the procedure is as follows:
|
||||
\begin{algorithm}
|
||||
\caption{Undo-Redo Recovery}
|
||||
\begin{algorithmic}[1]
|
||||
\Procedure{UndoRedoRecovery}{}
|
||||
\For{each item in the DB}
|
||||
\State Find last committed transaction modifying the item and REDO it
|
||||
\If{no committed transaction modified item}
|
||||
\State Find the first aborted transaction that modified the item and \texttt{UNDO} the modification
|
||||
\EndIf
|
||||
\If{If no transaction has touched the item}
|
||||
\State Value is correct (we assume that we start from a consistent state)
|
||||
\EndIf
|
||||
\EndFor
|
||||
\EndProcedure
|
||||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
This is not the way it's done in practice, because there typically are many more items than log entries.
|
||||
|
||||
\shade{ForestGreen}{Advantages} Only forced I/O are log records, buffer cache manager has a lot of freedom
|
||||
(no flushing dirty pages, I/O on data is minimized and only triggered for replacement policies) and we allow writing dirty data to disk, simplifying buffer management.
|
||||
|
||||
However, recovery is more complicated and takes more time, but normal operations are only minimally affected.
|
||||
Furthermore, queries are not affected since there is no forced I/O of data.
|
||||
Transactions are affected, because we need to write every operation to the log.
|
||||
|
||||
|
||||
\subsubsection{UNDO, no REDO}
|
||||
\begin{itemize}
|
||||
\item READ: simply read the value from the block in the buffer cache
|
||||
\item WRITE: Create log entry (before image), append it to persistent log and write the after image to the block on the buffer cache.
|
||||
\item COMMIT: Flush all dirty values modified by transaction if still in cache. We write a persistent log entry indicating that the transaction has committed
|
||||
\item ABORT: For all updates, we restore the before image using the log entry
|
||||
\end{itemize}
|
||||
The Procedure:
|
||||
\begin{enumerate}
|
||||
\item Start from the end of the log and work backwards
|
||||
\item Keep a list of UNDONE items
|
||||
\item Procedure terminates when all items are in either the UNDONE list or we reach the beginning of the log.
|
||||
\item For each log entry, we look at the data item $x$ being accessed and if $x$ is not in the UNDONE list and the transaction is aborted,
|
||||
we UNDO the changes by using the before image, adding $x$ to the UNDONE list
|
||||
\end{enumerate}
|
||||
It relies on the fact that all committed values are in persistent storage and therefore, they have not been lost.
|
||||
|
||||
However, it has forced I/O on all dirty blocks touched by the transaction when it commits,
|
||||
log records no longer need to include after images, reducing log record size and the recovery procedure is shorter because it only involves undoing aborted transactions.
|
||||
|
||||
However, this trade-off does not pay off in practice, as we still need to write the log with every update and while it requires smaller log records,
|
||||
it forces I/O on data blocks, which are often much larger.
|
||||
And finally, flushing the buffer cache interferes with its operations.
|
||||
|
||||
|
||||
\subsubsection{no UNDO, REDO}
|
||||
\begin{itemize}
|
||||
\item READ: If the transaction did not write the item before, read the value from the block in the buffer cache. Otherwise, read the value from the temporary buffer
|
||||
\item WRITE: Create log entry (after image), append it to persistent log and write the after image to the block on the buffer cache.
|
||||
\item COMMIT: Apply all updates in the temporary buffer to the actual blocks. We write a persistent log entry indicating that the transaction has committed
|
||||
\item ABORT: Discard the temporary buffer
|
||||
\end{itemize}
|
||||
The Procedure:
|
||||
\begin{enumerate}
|
||||
\item Start from the end of the log and work backwards
|
||||
\item Keep a list of REDONE items
|
||||
\item Procedure terminates when all items are in either the REDONE list or we reach the beginning of the log.
|
||||
\item For each log entry, we look at the data item $x$ being accessed and if $x$ is not in the REDONE list and the transaction is committed,
|
||||
we REDO the changes by using the after image, adding $x$ to the REDONE list
|
||||
\end{enumerate}
|
||||
It relies on the fact that there are never dirty blocks in the buffer cache. All data there is committed and is the last committed version.
|
||||
|
||||
It has forced I/O only on the log records and they are also smaller.
|
||||
Furthermore, the recovery procedure is shorter, only involving redoing the last committed transaction that touched the item.
|
||||
It is similar to snapshot isolation.
|
||||
|
||||
|
||||
\subsubsection{No UNDO, no REDO}
|
||||
This procedure is rarely used in conventional DBs. It does not require a log.
|
||||
However, it requires to be able to write all changes made by a transaction to persistent storage in \bi{a single atomic action}.
|
||||
It relies on directories to keep track of the changes.
|
||||
\begin{itemize}
|
||||
\item READ: If the transaction did not write the item before, use current directory to find the latest committed copy.
|
||||
Otherwise, read the value from the shadow directory of that transaction to find the updated copy
|
||||
\item WRITE: Write to a buffer and add a pointer in the shadow directory for the transactions
|
||||
\item COMMIT: Create a full directory by merging the current one and the shadow directory of the transaction.
|
||||
Then, we swap the pointer indicating the latest committed directory.
|
||||
\item ABORT: Discard the buffer(s) and the shadow directory
|
||||
\end{itemize}
|
||||
There is no recovery procedure here, which is the exact point.
|
||||
|
||||
It is not used in practice, although some ideas can be applied partially.
|
||||
One of the reasons for that is expensive pointer indirections for accesses and requires garbage collection and moves data all the time,
|
||||
which poses a challenge for clustered indexes or hash clustered tables.
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
\subsection{Implementation of Recovery}
|
||||
The most common implementation of these ideas is Write Ahead Logging (WAL).
|
||||
There, we separate the storage used for the log and data. The log contains enough information for whatever policy is chosen and log records corresponding to a change in the DB
|
||||
must be written to the log before changes to the data in the buffer cache are flushed to permanent storage.
|
||||
The COMMIT record in the log is us ed to mark the end of a transaction.
|
||||
|
||||
This is typically used to implement UNDO/REDO (on 2PL-based systems) or no-UNDO/REDO (on Snapshot Isolation (SI)-based systems)
|
||||
|
||||
|
||||
\subsubsection{Checkpoints}
|
||||
A log would grow forever if we don't do anything. Storage would then become an issue and recovery time would increase.
|
||||
Instead, Checkpoints are used. At a checkpoint, we do the following:
|
||||
\begin{itemize}
|
||||
\item Push all dirty blocks to the disk
|
||||
\item Push all the logs in the log buffer to disk
|
||||
\item We now have an active translation table and dirty page table
|
||||
\item We mark the log with a checkpoint label and flush it to the log
|
||||
\end{itemize}
|
||||
Then, recovery happens from a checkpoint instead from the beginning. However note that a checkpoint is not necessarily a consistent copy of the DB.
|
||||
|
||||
|
||||
\subsubsection{Recovery with WAL and checkpoints}
|
||||
Also called ARIES style recovery, if we use WAL, the recovery with a checkpoint works as follows:
|
||||
|
||||
\begin{algorithm}
|
||||
\caption{WAL checkpoint recovery}
|
||||
\begin{algorithmic}[1]
|
||||
\Procedure{WALCheckpointRecovery}{}
|
||||
\State Find latest completed checkpoint in the log
|
||||
\While{Traversing the log to the end}
|
||||
\State identify active transactions at time of crash and dirty pages that may not have made it to disk at time of crash for each element
|
||||
\EndWhile
|
||||
\State Apply all updates (REDO) starting from the log entry matching the lowest SCN in the dirty pages
|
||||
\State UNDO all transactions that were active at the time of the crash
|
||||
\EndProcedure
|
||||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
\newpage
|
||||
\section{Concurrency Control and Recovery}
|
||||
\input{parts/05_concurrency-control-recovery/00_transactions.tex}
|
||||
\input{parts/05_concurrency-control-recovery/01_acid.tex}
|
||||
\input{parts/05_concurrency-control-recovery/02_transaction-model.tex}
|
||||
\input{parts/05_concurrency-control-recovery/03_concurrency-control.tex}
|
||||
\input{parts/05_concurrency-control-recovery/04_two-phase-locking.tex}
|
||||
\input{parts/05_concurrency-control-recovery/05_snapshot-isolation.tex}
|
||||
\input{parts/05_concurrency-control-recovery/06_recovery.tex}
|
||||
\input{parts/05_concurrency-control-recovery/07_recovery-procedures.tex}
|
||||
\input{parts/05_concurrency-control-recovery/08_implementation-of-recovery.tex}
|
||||
% \input{parts/05_concurrency-control-recovery/}
|
||||
@@ -1,6 +0,0 @@
|
||||
\section{Vector Search}
|
||||
\input{parts/05_vector-search/00_intro-trees-clustering.tex}
|
||||
\input{parts/05_vector-search/01_hsnw-index.tex}
|
||||
\input{parts/05_vector-search/02_hash-index.tex}
|
||||
\input{parts/05_vector-search/03_filtered-vector-search.tex}
|
||||
% \input{parts/05_vector-search/}
|
||||
@@ -1,13 +0,0 @@
|
||||
\section{Data}
|
||||
\subsection{Formats}
|
||||
\input{parts/06_data/00_formats/00_intro.tex}
|
||||
\input{parts/06_data/00_formats/01_encodings.tex}
|
||||
% \input{parts/06_data/00_formats/}
|
||||
|
||||
\subsection{Cloud systems}
|
||||
\input{parts/06_data/01_cloud-systems/00_intro.tex}
|
||||
\input{parts/06_data/01_cloud-systems/01_key-value-store.tex}
|
||||
\input{parts/06_data/01_cloud-systems/02_cloud-native-databases.tex}
|
||||
\input{parts/06_data/01_cloud-systems/03_future.tex}
|
||||
% \input{parts/06_data/01_cloud-systems/}
|
||||
% \input{parts/06_data/}
|
||||
@@ -0,0 +1,6 @@
|
||||
\section{Vector Search}
|
||||
\input{parts/06_vector-search/00_intro-trees-clustering.tex}
|
||||
\input{parts/06_vector-search/01_hsnw-index.tex}
|
||||
\input{parts/06_vector-search/02_hash-index.tex}
|
||||
\input{parts/06_vector-search/03_filtered-vector-search.tex}
|
||||
% \input{parts/06_vector-search/}
|
||||
@@ -0,0 +1,13 @@
|
||||
\section{Data}
|
||||
\subsection{Formats}
|
||||
\input{parts/07_data/00_formats/00_intro.tex}
|
||||
\input{parts/07_data/00_formats/01_encodings.tex}
|
||||
% \input{parts/07_data/00_formats/}
|
||||
|
||||
\subsection{Cloud systems}
|
||||
\input{parts/07_data/01_cloud-systems/00_intro.tex}
|
||||
\input{parts/07_data/01_cloud-systems/01_key-value-store.tex}
|
||||
\input{parts/07_data/01_cloud-systems/02_cloud-native-databases.tex}
|
||||
\input{parts/07_data/01_cloud-systems/03_future.tex}
|
||||
% \input{parts/07_data/01_cloud-systems/}
|
||||
% \input{parts/07_data/}
|
||||
Reference in New Issue
Block a user