diff --git a/semester4/dmdb/data-modelling-databases-summary.pdf b/semester4/dmdb/data-modelling-databases-summary.pdf index a0fb95e..5767819 100644 Binary files a/semester4/dmdb/data-modelling-databases-summary.pdf and b/semester4/dmdb/data-modelling-databases-summary.pdf differ diff --git a/semester4/dmdb/parts/06_vector-search/03_filtered-vector-search.tex b/semester4/dmdb/parts/06_vector-search/03_filtered-vector-search.tex index f2c70ea..193b3b3 100644 --- a/semester4/dmdb/parts/06_vector-search/03_filtered-vector-search.tex +++ b/semester4/dmdb/parts/06_vector-search/03_filtered-vector-search.tex @@ -54,6 +54,7 @@ Not all dataset and query combinations are equally easy, and filters also change \subsubsection{Quantization} +\label{sec:vector-search-quantization} The total storage usage of course is always $N \cdot c$, where $N$ is the number of vectors and $c$ is the cost per vector, computed as follows: \begin{itemize} \item \bi{Scalar-Quantization} (SQ): $c = i + d \cdot m$, where $d$ is the number of dimensions of the vector and $m$ is the byte count per dim. diff --git a/semester4/dmdb/parts/08_quick-overview/01_theory.tex b/semester4/dmdb/parts/08_quick-overview/01_theory.tex index 12d8207..723d4ca 100644 --- a/semester4/dmdb/parts/08_quick-overview/01_theory.tex +++ b/semester4/dmdb/parts/08_quick-overview/01_theory.tex @@ -1,7 +1,7 @@ \subsection{Functional Dependencies \& Normal Forms} Also see Section~\ref{sec:functional-dependency} for more detailed and formal definitions -\subsection{Functional Dependencies} +\subsubsection{Functional Dependencies} \inlinedefinition[Transitive Closure] This is the set of all attributes that can be derived from the key, obtained by recursively evaluating the key using FDs, on each iteration also including the attributes previously obtained using FDs. @@ -23,23 +23,26 @@ The last task is achieved by checking for each attribute in the LHS, if removing Finally, we remove all redundant FDs. -\subsection{Normal Forms} +\subsubsection{Normal Forms} Below a short explanation as to how to check if a given relation is in any of the normal forms. A relation being in a NF further down the list implies it also is in the above, whereas a relation \textit{not} being in a higher up the list NF implies it isn't in the further below. \begin{itemize} \item \bi{1NF}: Each attribute cannot contain further tables or arrays (this is very commonly given in the tasks) \item \bi{2NF}: Determine the candidate keys. Then for each non-prime attribute (i.e. attribute not part of any candidate key) determine if it depends on - only a strict subset of a candidate key. If so, the relation is not in 2NF - \item \bi{3NF}: A relation is in 3NF if there are no ``transitive dependencies'', such as $A \rightarrow B, B \rightarrow C$. - More formally, if for all FD $\alpha \rightarrow B$, either the FD is trivial, $B$ is an attribute of at least one key or $\alpha$ is a superkey of $\cR$. + only a strict subset of a candidate key. If so, the relation is not in 2NF, + thus a relation is in 2NF if each non-prime attribute depends on the entirety of \bi{every} candidate key (or superkey) + \item \bi{3NF}: A relation is in 3NF if there are no ``transitive dependencies'' for non-prime attributes, such as $A \rightarrow B, B \rightarrow C$. + More formally, if for all FD $\alpha \rightarrow B$, either the FD is trivial, $B$ is an attribute of at least one key + (i.e. every attribute of $B$ must be part of a candidate key, i.e. there are no non-prime attributes in $B$) or $\alpha$ is a superkey of $\cR$. This means that the attributes only depend on candidate keys. - \item \bi{BCNF}: Here, we drop the possibility of $B$ being an attribute of at least one key. This means that a relation in BCNF is free of redundancies introduced by FDs. + \item \bi{BCNF}: Here, we drop the possibility of $B$ being an attribute of at least one key, so we need to check only if $\alpha$ is a superkey of $\cR$ or the FD being trivial. + This means that a relation in BCNF is free of redundancies introduced by FDs. \end{itemize} This means that for determining if a relation is in 3NF or BCNF, we need to be able to quickly identify superkeys and candidate keys. -\subsubsection{3NF Syntehsis algorithm} +\paragraph{3NF Syntehsis algorithm} This algorithm works as follows: \begin{enumerate} \item Compute the minimal cover @@ -50,7 +53,7 @@ This algorithm works as follows: \end{enumerate} -\subsubsection{BCNF Decomposition algorithm} +\paragraph{BCNF Decomposition algorithm} The order in which the FDs are picked for decomposition matters very much. To determine e.g. which FDs are lost, we need to apply the algorithm by starting with each FD. The algorithm works as follows ($\cR_i$ is the current state of relation we are evaluating): diff --git a/semester4/dmdb/parts/08_quick-overview/03_qp.tex b/semester4/dmdb/parts/08_quick-overview/03_qp.tex index f5d4b38..096be61 100644 --- a/semester4/dmdb/parts/08_quick-overview/03_qp.tex +++ b/semester4/dmdb/parts/08_quick-overview/03_qp.tex @@ -1,3 +1,4 @@ +\newpage \subsection{Query Processing} \subsubsection{Sorting} Given $B$ frames of memory and $N$ records, we have (typically I/Os in pages to be read) @@ -12,15 +13,15 @@ Given $B$ frames of memory and $N$ records, we have (typically I/Os in pages to \subsubsection{Selection} \begin{itemize} \item \bi{File Scan}: $N \div P_F$, with $P_F$ the number of pages, because we need to run through all pages - \item \bi{B+ Tree}: $\texttt{height}(T) - 1 + \texttt{cnt}(P_L) + \texttt{cnt}(P_F)$, $P_L$ the leaf pages and $P_F$ the file pages, + \item \bi{B+ Tree}: $\texttt{height}(T) - 1 + \texttt{cnt}(P_L) + \texttt{cnt}(P_F)$, $P_L$ the affected leaf pages and $P_F$ the file pages, the minus one comes from the fact that we count the leaf page already using $P_L$. \begin{itemize} \item For a clustered index, $\texttt{cnt}(P_F) = \texttt{cnt}(R') \div (F_L \cdot R_L)$, with $R_L$ the number of records per leaf page and $F_L$ the fill factor. \item For an unclustered index, $\texttt{cnt}(P_F) = \texttt{cnt}(R')$, with $\texttt{cnt}(R') = \texttt{cnt}(R) \cdot S$, with $S$ the selectivity of the predicate. \end{itemize} - \item \bi{Hash Index}: Cost is $1 + \texttt{cnt}(R') \div \texttt{cnt}(P_L) + \texttt{cnt}(R')$, where the $1$ is to do the lookup, - $\texttt{cnt}(R') \div P_L$ to fetch the row IDs and $\texttt{cnt}(R')$ to fetch the records - (this is an up-to, it is $\texttt{cnt}(R') \div P_F$ as minimum, for clustered index), with $P_L$ the number of records per Leaf and $P_F$ the number of records per file. + \item \bi{Hash Index}: Cost is $1 + \texttt{cnt}(R') \div L_L + \texttt{cnt}(R')$, where the $1$ is to do the lookup, + $\texttt{cnt}(R') \div R_L$ to fetch the row IDs and $\texttt{cnt}(R')$ to fetch the records + (this is an up-to, it is $\texttt{cnt}(R') \div R_F$ as minimum, for clustered index), with $R_L$ the number of records per Leaf and $R_F$ the number of records per file. \end{itemize} diff --git a/semester4/dmdb/parts/08_quick-overview/04_checklist.tex b/semester4/dmdb/parts/08_quick-overview/04_checklist.tex index 3576605..5594aef 100644 --- a/semester4/dmdb/parts/08_quick-overview/04_checklist.tex +++ b/semester4/dmdb/parts/08_quick-overview/04_checklist.tex @@ -1,3 +1,4 @@ +\newpage \subsection{Checklist} The following things are typically important to know very well (not exhaustive) \begin{todolist} @@ -8,7 +9,7 @@ The following things are typically important to know very well (not exhaustive) \item Time complexities, I/Os and sorted runs for Query Processing, see summarized above, or Section~\ref{sec:query-processing} \item Concepts behind the query processing operators, see Section~\ref{sec:query-processing} \item Conflict Serializability, see Section~\ref{sec:tm-history} - \item Core concepts of Vector Search, see Section~\ref{sec:vector-search} + \item Core concepts of Vector Search, see Section~\ref{sec:vector-search}, especially quantization costs, see Section~\ref{sec:vector-search-quantization} \item Recoverability (both the normal techniques, plus Snapshot Isolation and 2-Phase Locking (and strict variant thereof)), see Section~\ref{sec:recovery-of-histories} \item Rewriting rules, see Section~\ref{sec:rewriting-rules} \end{todolist}