[DMDB] Prepare parts; bitmaps, bloom filters, zone maps complete

semester4/dmdb/parts/03_systems/02_indexing/03_bitmaps/00_intro.tex

@@ -0,0 +1,13 @@
+Bitmap indexes are used for attributes with low cardinality (low number of distinct values), attributes that are retrieved by category and queries with low selectivity.
+
+In addition, they are used to perform aggregation functions, such as \texttt{SUM} and to perform selection and operators over column stores.
+
+Bitmap indexes are an alternative to B-trees and very space efficient.
+
+Bitmaps work as follows:
+\begin{itemize}
+    \item List all distinct values of an attribute
+    \item For every distinct value (and for NULLs), create an array with 1 bit entries (i.e. 0 or 1)
+    \item In the array corresponding to a given value of the attribute, set the positions in the array to one that correspond to tuples that have that value in that attribute.
+    \item Now, each array indicates the tuples that have the value corresponding to that array.
+\end{itemize}

semester4/dmdb/parts/03_systems/02_indexing/03_bitmaps/01_as-indexes.tex

@@ -0,0 +1,11 @@
+\paragraph{Bitmaps as indexes}
+While a B-tree or hash index stores the row ID aor tuple ID, a bitmap index only stores a single bit at a given position,
+which is indicating the of the tuple from the beginning of the table.
+
+This means that to map a 1 in the beatmap to a tuple address requires applying a function that calcualtes the address.
+One way to achieve this is to layout the bitmap in page ID order. Then to map a bit position to a record id,
+we fix the number of records per page, lay the pages out such that the page numbers are sequential and increasing and then construct the row ID, page ID and slot number.
+
+As a result, $\texttt{pageID} = \texttt{bit-pos} \div \#\texttt{records per page}$ and $\texttt{slot-number} = \texttt{bit-pos} \% \#\texttt{records per page}$
+
+We can also compress bitmaps, in Postgres, this is done using run-length encoding.

semester4/dmdb/parts/03_systems/02_indexing/03_bitmaps/02_usage.tex

@@ -0,0 +1,20 @@
+\paragraph{Using bitmaps}
+Bitmaps are used in database systems to solve simple predicate selections, such as
+\mint{sql}|SELECT NAME FROM STUDENTS WHERE Students.Department = D-INFK|
+or for multiple operations, we can perform operations directly on the bitmaps
+\mint{sql}|SELECT NAME FROM STUDENTS WHERE Students.Department = D-INFK and Students.Degree = Bachelor|
+
+In addition, they are usually \textit{very} sparse and long lists with just a few 1s compress very well, using run length encoding.
+
+They are also often used for large tables in analytics as it is faster than other indexes for low selectivity queries
+and they are especially useful for expensive comparisons, such as for strings.
+
+They can also be used to implement operators:
+\begin{itemize}
+    \item \texttt{NOT IN}: get a bitmap with the results of the inner query, match it against a bitmap for the outer query,
+          then retrieve the tuples
+    \item \texttt{COUNT}: Count how many 1s appear in the bitmap
+    \item \texttt{JOIN}: Match (\texttt{AND} operation) the bitmaps of the values of the two attributes
+    \item \texttt{RANGE} predicates: Combine the bitmaps (\texttt{OR} operations) of all the values in the range
+    \item \dots
+\end{itemize}

semester4/dmdb/parts/03_systems/02_indexing/04_bloom-filters.tex

@@ -0,0 +1,12 @@
+A bloom filter is an index that says whether a tuple with a given value is \bi{NOT} in the original table.
+It acts as a filter, eliminating unnecessary accesses to the data.
+However, it can have false positives, i.e. it may state that a given tuple is there, but it is not in fact.
+Most importantly, it doesn't have false negatives, which is good, because this means we don't accidentally miss any tuples.
+
+A bloom filter works by creating an array of size $m$, with values hashed using $k$ hash functions.
+Each hash point points to a position in the array that is set to $1$.
+Then, to check if an item is in the data, we check all the $k$ hash positions for the value
+and if they all are all 1, then the item is in the data and if one (or more) is $0$, then the element \textit{definitely}
+isn't in the data.
+
+The bloom filter's error rate depends on the number of hashes ($k$), the size of the array ($m$) and the number of items being hashed ($n$)

semester4/dmdb/parts/03_systems/02_indexing/05_zone-maps.tex

@@ -0,0 +1,6 @@
+There are some systems that do not use indexes, or very sparingly.
+Instead, they keep metadata about the blocks and store them in memory and use it to check if the data in the block is needed.
+
+This is common in column-store engines such as DuckDB and Snowflake.
+This metadata can be as simple as min/max for each column,
+or complete stats, bitmaps, precomputed aggregates, or even bloom filters.