[DMDB] B-trees

semester4/dmdb/data-modelling-databases-summary.tex

@@ -7,11 +7,11 @@
 \input{glossary/main.tex}
 \setGlossaryMarkup
 
-
 \usepackage{lmodern}
 \setFontType{sans}
 
 \newcommand{\sql}{\acrshort{sql}}
+\renewcommand{\descriptorNameDisplay}[1]{\textbf{#1}}
 
 \setup{Data Modelling \& Databases}
 

semester4/dmdb/parts/03_systems/01_database-engines/00_memory/03_data-in-blocks-and-pages.tex

@@ -18,6 +18,7 @@ There are some ways to improve the performance:
 
 
 \paragraph{Slotted Pages}
+\label{sec:slotted-pages}
 To find tuples within pages, slotted pages are used often.
 In slotted pages, each page has a header containing checksum, version, transaction visibility, compression information, utilization, etc.
 

semester4/dmdb/parts/03_systems/02_indexing/00_access-methods.tex

@@ -0,0 +1,13 @@
+\subsubsection{Access methods}
+\inlinedefinition[Access Methods] typically refers to the operations and data structures used for reading data from the buffer cache into the private space of the worker thread
+
+The basic access method is \bi{sequential scan}, where all pages of the table are read from beginning to end.
+Indexing is used to avoid having to scan the whole table.
+
+There are many indexing techniques, but we will focus on three indexing techniques in this section, hashing, B+ trees and bitmaps.
+
+Hashing is primarily used to randomize access to the data, finding data based on a key (hash and access concept)
+and both as an index for tables and used to implement operators.
+
+B+ trees on the other hand are used to order data along the values of one attribute, but require traversal, unless we are looking for consecutive values.
+They are thus only used as an indexing mechanism.

semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex

@@ -0,0 +1,6 @@
+\subsubsection{B+ Trees}
+\paragraph{B-Tree}
+In B-Trees of order $k$, each node (except the root) has between $k \div 2$ and $k$ child nodes.
+The root node has at least two children, unless it is also a leaf and any non-leaf node contains at exactly $k - 1$ keys.
+
+Even if database systems say they use B-Trees, they do in fact use B+ trees.

semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/01_basics.tex

@@ -0,0 +1,59 @@
+\paragraph{Basics}
+B+ Trees are B-Trees, but the data (or the pointers to it) is at the leaves only, which are organized as a linked list.
+They are balanced trees, their inner nodes correspond to blocks and the leaf nodes correspond to blocks.
+The blocks are organized like slotted pages for variable length data (see Section \ref{sec:slotted-pages})
+
+The keys in the inner nodes may not correspond to the actual data and are instead used as separators.
+Typically, leaf nodes contain pointers to the tuples \texttt{(value, key)}, where \texttt{value} is the actually indexed value
+and \texttt{key} is the pointer to the full data tuple, typically as a row id or tuple id.
+There are some systems that allow storing the data directly on the leaves.
+
+Some systems create a B+ tree index by default for all tables and index the key. If there is no key, the engine assigns random keys and indexes them.
+
+
+\paragraph{Clustered Indexes}
+An index orders the table by the attribute it is indexing, but the tuples in the table might not be ordered.
+This is where \bi{clustered indexes} come into play: They force the tuples to be stored in the same order as the index indicates,
+thus, the table is physically stored in a sorted manner. This is typically done only for the primary key,
+and automatically done in systems that store data in the leaf nodes directly.
+
+This eliminates random memory accesses caused by looking up ranges, since the tuples are stored in consecutive order, according to the index.
+
+This is most useful for tables that are infrequently updated, since they require maintenance.
+
+
+\paragraph{What to index?}
+This is a crucial question, as it can massively speed things up, but equally significantly slow things down.
+
+B+ trees can use one or \textit{more} attributes as the key to the index.
+If we have just one attribute, those are the values.
+If there are several attributes, it builds a composite key, which is useful when we are looking for combinations of values on certain attributes,
+or a table is searched by several attributes.
+
+We then compare the keys lexicographically:
+\[
+    (a_1, a_2) < (b_1, b_2) \Leftrightarrow (a_1 < b_1) \lor (a_1 = b_1 \land a_2 < b_2)
+\]
+Some values can also be left unspecified, but this is typically only done to the ones at the beginning of the key.
+
+\newpage
+\paragraph{Composite Index}
+For a \acrshort{sql} statement such as
+\begin{code}{sql}
+    SELECT department_id, last_name, salary
+    FROM employees WHERE salary > 5000
+    ORDER BY department_id, last_name;
+\end{code}
+if we assume a composite index on \texttt{department\_id, last\_name, salary}, then the data is sorted by the three attributes in that order.
+
+We can then perform a \bi{full index scan}, where read the data from the leaves in sorted order and filter on salary.
+This avoids having to scan the entire table and the results is also already sorted.
+
+
+\paragraph{Non-unique values}
+A B+ tree index can also be built on attributes containing also non-unique values.
+
+This is a problem, because we cannot find all the duplicates with the basic design. To resolve this, we can repeat the key at the leaf nodes for every duplicated entry,
+or we can store the key once, but point to a linked list of all matching entries.
+
+If the data is stored in the leaves, we append the tuple ID to know what tuple the entry refers to, because otherwise, they are all the same.

semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/02_operations.tex

@@ -0,0 +1,55 @@
+\paragraph{Operations}
+\subparagraph{Direct Lookup}
+We traverse the tree from the root, then within each node, we use binary search to look for the correct entry.
+When we reach a leaf node, we return the corresponding pointer and the position.
+
+
+\subparagraph{Range Lookup}
+We return all tuples that match, by first finding the first tuple matching, the continue traversal until the last match is found.
+We can determine the last match easily because the tree's leafs are lexicographically sorted.
+Since the leafs are linked as linked lists, we can simply move along the linked lists, cutting down on processing time.
+
+
+\subparagraph{Inserting into the index}
+We first look up the corresponding leaf. Then, if there is space, we simply insert the value.
+If there is no space in the leaf, we split the leaf into two new leafs and insert the new item on the corresponding leaf.
+
+All that remains to do now is to insert the separator on the parent node.
+If that node is full, we split the node in two and insert the separator into the parent node and propagate up to the root, if necessary.
+
+
+\subparagraph{Deleting from the index}
+We proceed by looking up the corresponding leaf, then we remove the entry from the leaf.
+If the leaf now is less than half full, we check the neighboring leaf
+\begin{itemize}
+    \item If it is more than half full, we balance the two leafs
+    \item If it is half full, we merge both leaves
+\end{itemize}
+For both operations, we update the separator in the parent node.
+
+
+\paragraph{Concurrent Access}
+Indexes are heavily used while they are being maintained, thus creating conflicting, concurrent accesses.
+
+A way to prevent this is using \bi{lock coupling}, where we lock a page and its parent and move down the tree.
+
+This prevents problems when the page must be split, however, this is not enough, if we have to go further up to apply the changes.
+
+To prevent this, on every node we check if there is space for one more entry. If not, we split the node.
+This way, the split of a page never leads to propagating changes.
+
+Alternatively, we can try lock coupling, and if we have to do a change that propagates up, we abort, release everything and start again from the top,
+but now lock the entire path.
+
+
+\paragraph{Bulk Inserts}
+To create a B+ Tree index, we proceed as follows (the tree is created bottom up (from the leaves up))
+\begin{enumerate}
+    \item Sort the data
+    \item Use the sorted data to fill one block after the other
+    \item Remember the largest value in each block
+    \item create the inner nodes by using the largest value in each block as separator
+    \item iterate upwards to the next level, until there is only one block left (the root)
+\end{enumerate}
+If the blocks are filled completely, we get a clustered and compact tree.
+However, since we usually expect that data has to be updated, it is advised to leave some space in the each block to accommodate that without expensive propagating changes.

semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/03_optimizations.tex

@@ -0,0 +1,27 @@
+\paragraph{Optimizations}
+\shade{ForestGreen}{Reverse Index}
+If the indexed attribute is a sequence and new items are constantly produced, we may encounter an issue where values next to each other go into the same block,
+leading to concurrent updates fighting to insert. Depending on how updates are handled, many copies could be produced.
+
+Reverse indexing solves that issue by reversing the indices, meaning that consecutive indices go to different pages.
+
+
+\shade{ForestGreen}{Many Keys} Depending on attributes, many keys can become expensive to process.
+There are several possible optimizations:
+\begin{itemize}
+    \item Replace separators that correspond to actual keys with shorter separators with the same effect (can be hard)
+    \item Factor common prefix (this makes a lot of sense and is usually easy, too)
+\end{itemize}
+
+
+\shade{ForestGreen}{Ignoring rules} We can also ignore the rules of the B+ trees and e.g. not merge nodes when they do not have enough data,
+delaying a merge (and instead rebuilding the tree periodically)
+
+
+\shade{ForestGreen}{Variable Length Keys} This is a bit of a problem. We can mitigate it by no longer forcing the number of entries to lay between $k \div 2$ and $k$,
+and using smaller values as separators, placing long values in leafs.
+
+\shade{ForestGreen}{Making sure there is space} We don't ever fill blocks to the max and make sure there is space on creation.
+
+
+\shade{ForestGreen}{Depth vs Breadth} For slow devices, prefer larger node sizes, thus shallower trees; For fast devices, prefer deeper trees.

semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/04_uses.tex

@@ -0,0 +1,37 @@
+\paragraph{When to use B+ Trees}
+\begin{itemize}
+    \item For exact match queries (using \texttt{=} or \texttt{in}). This avoids having to compare against the entire table
+    \item Range queries (using \texttt{<} and/or \texttt{<}).
+    \item Full scans for sorted output (as the tuples are inherently sorted in the tree)
+\end{itemize}
+
+
+\paragraph{When not to use B+ Trees}
+\begin{itemize}
+    \item The index stores pointers to tuples
+    \item For larger number of accesses, more random memory accesses happen, which is slower.
+\end{itemize}
+
+
+\paragraph{Columns vs Rows}
+B+ Trees can be used to index columns, but are primarily used in row store systems,
+because Column Stores optimize sequential reads and can use vector operations, as well as compressions.
+
+A B+ Tree would point to an entry, but only after a lot of pointer chasing.
+
+And then there is also the point that they require extra storage, which for column stores just doesn't make sense.
+
+
+
+\paragraph{Filtered Indexes}
+As the name implies, this is a B+ Tree that indexes only parts of a table, which commonly are the most used ranges.
+This of course means that the index is smaller and thus faster to traverse.
+It defines a range predicate over the indexed key.
+
+
+\paragraph{How are they used?}
+\begin{itemize}
+    \item JOIN: if the inner table is indexed, a nested loop join will look up the tuple on the index, reducing comparisons
+    \item ORDER BY: If the sorting is on the indexed key, we can easily retrieve tuples in order
+    \item IN or NOT IN: In nested queries, the index can be used to check if a tuple in the outer query is in the result of the inner query.
+\end{itemize}

semester4/dmdb/parts/03_systems/main.tex

@@ -19,4 +19,13 @@
 % \input{parts/03_systems/01_database-engines/00_memory/}
 % \input{parts/03_systems/01_database-engines/}
 
+\newpage
+\subsection{Indexing}
+\input{parts/03_systems/02_indexing/00_access-methods.tex}
+\input{parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex}
+\input{parts/03_systems/02_indexing/00_b-trees/01_basics.tex}
+\input{parts/03_systems/02_indexing/00_b-trees/02_operations.tex}
+\input{parts/03_systems/02_indexing/00_b-trees/03_optimizations.tex}
+% \input{parts/03_systems/02_indexing/00_b-trees/}
+% \input{parts/03_systems/02_indexing/}
 % \input{parts/03_systems/}