diff --git a/semester4/dmdb/data-modelling-databases-summary.pdf b/semester4/dmdb/data-modelling-databases-summary.pdf index ac2a302..c505a37 100644 Binary files a/semester4/dmdb/data-modelling-databases-summary.pdf and b/semester4/dmdb/data-modelling-databases-summary.pdf differ diff --git a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/00_b-trees.tex similarity index 92% rename from semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex rename to semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/00_b-trees.tex index 2e6bf68..fba8803 100644 --- a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex +++ b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/00_b-trees.tex @@ -1,4 +1,3 @@ -\subsubsection{B+ Trees} \paragraph{B-Tree} In B-Trees of order $k$, each node (except the root) has between $k \div 2$ and $k$ child nodes. The root node has at least two children, unless it is also a leaf and any non-leaf node contains at exactly $k - 1$ keys. diff --git a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/01_basics.tex b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/01_basics.tex similarity index 100% rename from semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/01_basics.tex rename to semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/01_basics.tex diff --git a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/02_operations.tex b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/02_operations.tex similarity index 100% rename from semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/02_operations.tex rename to semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/02_operations.tex diff --git a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/03_optimizations.tex b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/03_optimizations.tex similarity index 100% rename from semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/03_optimizations.tex rename to semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/03_optimizations.tex diff --git a/semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/04_uses.tex b/semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/04_uses.tex similarity index 100% rename from semester4/dmdb/parts/03_systems/02_indexing/00_b-trees/04_uses.tex rename to semester4/dmdb/parts/03_systems/02_indexing/01_b-trees/04_uses.tex diff --git a/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/00_hash-tables.tex b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/00_hash-tables.tex new file mode 100644 index 0000000..3e95e98 --- /dev/null +++ b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/00_hash-tables.tex @@ -0,0 +1,54 @@ +\paragraph{Hashing in Databases} +Hashing is a very common operation in many systems, including databases. +Many internal data structures, such as buffer caches and lock tables, are implemented as hash tables. +Some operators also use hashing to speed things up, such as hash join and group by operators. + +It can also be used as an indexing and partitioning strategy. + +Hashing is on the other side compared to B+ trees when it comes to the trade-off between storage and compute, +by using more compute, it can save on storage space requirements. + + +\paragraph{Hash Tables} +There are many hashing functions with strong properties, but in databases, the hash function has to be very cheap, as it is used very often. + +Perfect hash functions do exist, but they need a hash table that is as big as the cardinality of the attributes, +so for 4 byte keys, it needs a 4 GB table. +Thus, to save on storage, we use a hash table that is smaller than that. +If we were to choose it too small, there would be too many collisions, if it is chosen too large, then we waste lots of space. + +Collisions may also occur when the index key is not \texttt{UNIQUE}, and thus it could be that multiple tuples have the same key. + +That may lead you to think, why not grow it if we have a collision. That is a valid idea, but growing hash tables is expensive, thus should be avoided. + +Another shortcoming of hash indexes is that it only supports \bi{point queries}, i.e. it only works for the primary key, but barely anything else. + + +\paragraph{Collisions} +A way to handle collisions safely is to use \bi{chaining}. When a collision occurs, we add another entry in a linked list. +If these lists are reasonably short, then it is reasonably efficient. We can also already reserve space for the linked lists to speed things up, +if we expect (many) collisions to occur. + +Another option is to use \bi{open addressing}. Here, when a collision occurs, we look for an empty slot in the hash table using some rule. +Here, two ways of achieving this are covered in the course: +\begin{itemize} + \item \bi{Linear probing}, where we go to the next slot(s) and place the attribute there + \item \bi{Cuckoo hashing}, where we use several hash functions and move to the next one, if we have a collision with the first, and so on +\end{itemize} + + +\paragraph{Extensible Hashing} +When the hash table is full, in a basic system, we would need to create a new, larger hash table with more buckets (typically double the number) and then rehash all existing items. +This of course is very expensive both computationally and in terms of memory accesses, as we do lots of random accesses, which leads to cache misses. + +This is where extensible comes in. It is based on sharing buckets and unsharing them when needed. +These hash table buckets are mapped into blocks and initially several buckets share the same block + +Then, when a bucket gets full, we split the bucket and move entries as needed into a new bucket. + +In \bi{local sharing}, the size of a table can be doubled without immediately adding more space, allowing for more splitting. +We simply increase the number of buckets and the degree of sharing and split the buckets as they become full. + +Again, if we initially provide more space than needed, we can allow for the first growth to happen without disruption. + +We want (ideally) to have two page lookups to access an item, so we typically have a lookup in the bucket directory and the data block and they both can grow independently. diff --git a/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/01_linear-hashing.tex b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/01_linear-hashing.tex new file mode 100644 index 0000000..c9b6472 --- /dev/null +++ b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/01_linear-hashing.tex @@ -0,0 +1,11 @@ +\paragraph{Linear hashing} +Here, a split pointer is used to indicate which bucket will be split in case of overflow. +When it occurs, we chain the block that overflows, split the bucket indicated by the pointer and move the pointer. + +This allows for a gradual increase in the size of the table and the idea is to gradually redistribute the data. +We always split on the pointer, even if the overflow is elsewhere. The pointer will eventually reach a bucket with chains +and when the bucket is split, the data is reorganized. + +This has the advantage that the directory (i.e. the list of buckets) grows page by page, instead of doubling. + +When all buckets have been split, start fresh. diff --git a/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/02_in-postgres.tex b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/02_in-postgres.tex new file mode 100644 index 0000000..8142564 --- /dev/null +++ b/semester4/dmdb/parts/03_systems/02_indexing/02_hashing/02_in-postgres.tex @@ -0,0 +1,17 @@ +\paragraph{In Postgres} +PostgreSQL uses a variation of linear (overflow) hashing and extensible hashing (with doubling). +Buckets are initially allocated in groups with powers of two. Overflows are captured in overflow pages (like in linear hashing). +When a split occurs, we do not allocate all the pages, but a fraction of them. + +These ideas can be combined in many different ways. + +\paragraph{When to use} +For memory structures: +\begin{itemize} + \item For shared resources (System Global Area) + \item Database Buffer Cache for I/O + \item Redo log buffer for recovery + \item For large memory pools +\end{itemize} + +Then of course, it can be used for a hash join, for the \texttt{SELECT DISTINCT} function and for aggregation (\texttt{GROUP BY}). diff --git a/semester4/dmdb/parts/03_systems/main.tex b/semester4/dmdb/parts/03_systems/main.tex index 47d2556..96026e3 100644 --- a/semester4/dmdb/parts/03_systems/main.tex +++ b/semester4/dmdb/parts/03_systems/main.tex @@ -22,10 +22,18 @@ \newpage \subsection{Indexing} \input{parts/03_systems/02_indexing/00_access-methods.tex} -\input{parts/03_systems/02_indexing/00_b-trees/00_b-trees.tex} -\input{parts/03_systems/02_indexing/00_b-trees/01_basics.tex} -\input{parts/03_systems/02_indexing/00_b-trees/02_operations.tex} -\input{parts/03_systems/02_indexing/00_b-trees/03_optimizations.tex} -% \input{parts/03_systems/02_indexing/00_b-trees/} + +\subsubsection{B+ Trees} +\input{parts/03_systems/02_indexing/01_b-trees/00_b-trees.tex} +\input{parts/03_systems/02_indexing/01_b-trees/01_basics.tex} +\input{parts/03_systems/02_indexing/01_b-trees/02_operations.tex} +\input{parts/03_systems/02_indexing/01_b-trees/03_optimizations.tex} +% \input{parts/03_systems/02_indexing/01_b-trees/} + +\subsubsection{Hashing} +\input{parts/03_systems/02_indexing/02_hashing/00_hash-tables.tex} +\input{parts/03_systems/02_indexing/02_hashing/01_linear-hashing.tex} +\input{parts/03_systems/02_indexing/02_hashing/02_in-postgres.tex} +% \input{parts/03_systems/02_indexing/02_hashing/} % \input{parts/03_systems/02_indexing/} % \input{parts/03_systems/}