[DMDB] Notes on query processing, better summary for those, some exam notes

semester4/dmdb/parts/08_quick-overview/00_sql.tex

@@ -0,0 +1,12 @@
+\subsection{SQL}
+DuckDB (and PostgreSQL) features an \texttt{EXPLAIN} and \texttt{EXPLAIN ANALYZE} query,
+pretty-printing the query plan and pretty-printing the query plan and profiling the query execution, respectively
+
+A few useful commands for debugging and writing queries (for DuckDB and PostgreSQL via \texttt{psql} CLI):
+\begin{tables}{p{4cm}p{4cm}p{8cm}}{DuckDB & PostgreSQL          & Description}
+              \texttt{SHOW TABLES;}       & \verb|\dt|          & Show all tables                         \\
+              \texttt{SHOW tablename;}    & \verb|\d tablename| & Show schema of table \texttt{tablename} \\
+\end{tables}
+Note that \texttt{SHOW} and \texttt{DESCRIBE} are aliases in DuckDB. They can also be used on queries!.
+
+To see all \texttt{psql} commands, run \verb|\?|.

semester4/dmdb/parts/08_quick-overview/02_system.tex

@@ -0,0 +1,21 @@
+\subsection{System}
+For computing the usable space, deduct the overhead \textit{after} multiplying with the free fraction,
+e.g. for a block size of 4kB (with kB actually being KiB), header and directories having a combined size of 100 bytes and \texttt{PCTFREE} set to 20\%,
+the available space is $\ceil{4096 \cdot 0.8 - 100} = 3177$.
+
+To compute the maximum number of tuples in a slotted page, remember that it also has a header with a certain number of bytes for the slotting.
+Add that to the number of bytes per tuple when computing, even though it doesn't count to the tuple size!
+
+Row stores are better if we need to access many columns, column stores otherwise. This is (primarily) due to indexing.
+
+
+\subsubsection{Indexing}
+The global depth is the maximum Local Depth of all buckets.
+
+The hash key is the actual value of the column to be hashed typically.
+Thus, an assignment may be very misleading in that the block ID and hash key are asked.
+The hash key is the value before hashing, the block ID afterwards is the block the hashing algorithm assigned it to.
+
+When splitting a block, its $N$ entries must be repartitioned. There are $2^N$ possible ways in which to do this.
+So, there is a $\frac{2}{2^N}$ probability for there to be a \bi{recursive overflow}, an overflow in which all keys end up in the same bucket again
+and need further splitting.

semester4/dmdb/parts/08_quick-overview/03_qp.tex

@@ -0,0 +1,43 @@
+\subsection{Query Processing}
+\subsubsection{Sorting}
+Given $B$ frames of memory and $N$ records, we ahve
+\begin{itemize}
+    \item \bi{Merge Sort}: $2N \cdot P$, with $P = (1 + \ceil{\log_{B - 1}\ceil{N \div B}})$ the number of passes.
+          After the first pass, $\ceil{N \div B}$ number of sorted runs were created (typically)
+    \item \bi{Replacement Sort / Heap Sort}: $2N \cdot P$, with $P = (1 + \ceil{\log_{B - 1} \ceil{N \div E}})$ the number of passes, with $E$ the number of records per run,
+          given by $E = 2B$ in the average case, $E = N$ in the best case (thus single run, data already sorted), $B - 1$ (worst case, reversed data)
+\end{itemize}
+
+
+\subsubsection{Selection}
+\begin{itemize}
+    \item \bi{File Scan}: $N \div P_F$, with $P_F$ the number of pages, because we need to run through all pages
+    \item \bi{B+ Tree}: $\texttt{height}(T) + \texttt{cnt}(P_L) + \texttt{cnt}(P_F)$, $P_L$ the leaf pages and $P_F$ the file pages.
+          \begin{itemize}
+              \item For a clustered index, $\texttt{cnt}(P_F) = \texttt{cnt}(R') \div (F_L \cdot R_L)$, with $R_L$ the number of records per leaf page and $F_L$ the fill factor.
+              \item For an unclustered index, $\texttt{cnt}(P_F) = \texttt{cnt}(R')$, with $\texttt{cnt}(R') = \texttt{cnt}(R) \cdot S$, with $S$ the selectivity of the predicate.
+          \end{itemize}
+    \item \bi{Hash Index}: Cost is $1 + \texttt{cnt}(R') \div \texttt{cnt}(P_L) + \texttt{cnt}(R')$, where the $1$ is to do the lookup and
+          $\texttt{cnt}(R') \div P_L$, with $P_L$ the number of records per Leaf.
+\end{itemize}
+
+
+\subsubsection{Projection}
+\begin{itemize}
+    \item \bi{Partitioning}: $\texttt{Cost}_\texttt{part}(R) = \texttt{cnt}(R) + \texttt{cnt}(R')$ with
+          $\texttt{cnt}(R') = \texttt{cnt}(R) Q$, with $Q$ the fraction of selected attributes divided by total attributes
+    \item \bi{Sort-Based}: The number of sorted runs are $M = \ceil{\texttt{cnt}(R') \div B}$, with the merge passes $P = \ceil{\log_{B - 1}(M)}$, total cost:
+          $2 \cdot N \cdot P + \texttt{Cost}_\texttt{part}(R)$
+    \item \bi{Hash-Based}:
+          \begin{itemize}
+              \item \bi{Minimum $B$ for $M$-pass}: $B > \sqrt{\texttt{cnt}(R') \cdot f}$, with $f$ the Minister of Magic Factor (for Books $\leq$ 5),
+                    aka. Fudge Factor, typically $f \approx 1.2$.
+              \item \bi{Below threshold}: To check if we need recursive partitioning, we use the un-approximated version of the above: $\frac{f \cdot \texttt{cnt}(R')}{B - 1} < B$
+              \item \bi{Cost}: The base cost is $\texttt{Cost}_\texttt{part}(R) + \texttt{cnt}(R')$ for partitioning and duplicate elimination later on
+                    \begin{itemize}
+                        \item If below threshold, that's our cost
+                        \item If above threshold, then we need to repartition, which costs $2 \cdot \texttt{cnt}(R')$ (once each for write and read) for each time we do this action.
+                    \end{itemize}
+          \end{itemize}
+\end{itemize}
+Note that both the sort- and hash-based approaches perform approximately equally well at larger $B$, since both algorithms need two passes over the data.

semester4/dmdb/parts/08_quick-overview/main.tex

@@ -0,0 +1,7 @@
+\section{Quick Overview}
+This section aims to give you the most important things to remember in a very concise manner.
+
+\input{parts/08_quick-overview/00_sql.tex}
+\input{parts/08_quick-overview/01_theory.tex}
+\input{parts/08_quick-overview/02_system.tex}
+\input{parts/08_quick-overview/03_qp.tex}