[DMDB] Finished vector search

semester4/dmdb/glossary/acronyms.tex

@@ -3,3 +3,5 @@
 \newacronym{simd}{SIMD}{Single Instruction Multiple Data}
 \newacronym{avxa}{AVX}{Advanced Vector Extensions, also see \gls{avx}}
 \newacronym{ra}{RA}{Relational Algebra, see Section~\ref{sec:relational-algebra}}
+\newacronym{ann}{ANN}{Approximate Nearest Neighbour Search}
+\newacronym{knn}{KNN}{$K$-Nearest Neighbour Search}

semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex

@@ -1,4 +1,6 @@
 \subsection{Hierarchical Navigable Small World Index (HNSW Index)}
+The name used in the lecture was HSNW (Hierarchical Small Navigable World) Index.
+
 This is a layered graph concept, at insert, first all nodes are inserted into layer 0 and each is connected to $M \approx 3 - 4$ neighbours.
 Then, the edge count is $\tco{N \cdot M}$. With $N = 100$ million and $M = 16$, we have $1.6B$ edges, with a typical memory footprint of $4-8$GB for $100M$ 32-bit Edge IDs.
 Then, some nodes (typically about $N_{L_k} = N / M^k$ to layer $k$) are promoted to layer 1, connected to $M$ nearest neighbours in $L1$ only.
@@ -12,3 +14,36 @@ This is done using a greedy algorithm.
 
 
 \subsubsection{Search}
+We note that greedy search always makes progress in this index because the graph stays navigable.
+
+HNSW index search works by starting at a fixed entry, then doing \textit{one} greedy hop only.
+The hop goes towards the neighbour on this layer that is closer to the query.
+This is often done using Beam Search (see below) to find the $K$ closest vectors.
+
+After some number of hops, there are no more neighbours in layer $LN$ that are closer to check, so descend one layer and repeat.
+Traversal of the layer costs $\tco{M_L}$, with $M_L$ the number of edges on layer $L$.
+Higher up layers have fewer edges, thus making traversal cheap.
+
+The parameter \texttt{ef\_search} controls the recall vs latency.
+Set to $1$ it is approximately greedy, for $\texttt{ef\_search} \gg 1$ means more effort, i.e. more vectors visited.
+
+\paragraph{Beam Search}
+This is another greedy search algorithm, which chooses the next node to visit from a beam of size $N$.
+The goal is to find the closest node of the query, with $d$ the distance to the query.
+
+The procedure is simple:
+\begin{algorithm}
+    \caption{Beam Search}
+    \begin{algorithmic}[1]
+        \Procedure{BeamSearch}{$G$, start\_node}
+            \State $\texttt{Beam} \gets$ new beam data structure
+            \State \texttt{Beam.add(start\_node)}
+            \While{\texttt{beam} in \texttt{Beam}}
+                \State Check distances of neighbours and add them to \texttt{Beam}
+                \State Truncate \texttt{Beam} to size \texttt{max\_beam} \Comment{This is a tunable parameter}
+            \EndWhile
+            \State \Return \texttt{closest}
+        \EndProcedure
+    \end{algorithmic}
+\end{algorithm}
+

semester4/dmdb/parts/05_vector-search/02_hash-index.tex

@@ -0,0 +1,16 @@
+\subsection{Hash Index}
+To measure the quality of \acrshort{ann} in comparison to \acrshort{knn}, we use $\texttt{recall@k} = |\texttt{ANN\_result} \cap \texttt{KNN\_result}| \div k$.
+This however is not always a good metric, as two vastly different results, one objectively worse can score equally well.
+
+Alternative methods mentioned are \texttt{RDE@k} and \texttt{TDK@k}, but not elaborated any further.
+
+
+\subsubsection{Performance vs Recall Trade-off}
+This is a challenging task, primarily focusing on how many vectors we need to visit to achieve a user-specified target recall.
+
+A few explored strategies are:
+\begin{itemize}
+    \item \bi{Uniform autotuning for all queries}: Using learned offline models (e.g. Google CloudSQL VectorAssist)
+    \item \bi{Different for each query}: A model predicts the search effort parameters for each query or index
+    \item \bi{Adaptive}: Decide to continue or stop early based on the current search state
+\end{itemize}

semester4/dmdb/parts/05_vector-search/03_filtered-vector-search.tex

@@ -0,0 +1,78 @@
+\subsection{In Databases}
+The benefit of supporting vector search in DBMS is that this makes it easy to use for the end user and can be optimized for large datasets.
+
+\subsection{Filtered Vector Search (FVS)}
+This allows us to query both structured and unstructured data. In concept, this is SQL + Vectors, sometimes plus joins, subquery expressions,
+multiple vector searches (on both sides of the join). Sometimes, the dataset isn't even a vector + tag stored in main memory.
+
+Indices are typically built on unfiltered data. Thus, we need to think about the proper execution method. Suppose we are to retrieve $K$ rows with filtering:
+\begin{itemize}
+    \item \bi{Pre-Filter}: Filter first, then run \texttt{NNS}, and we get the $K$ rows. This is expensive if the filter is not selective.
+    \item \bi{Post-Filter}: We first run \acrshort{ann} search, from which we get $K'$ rows which we then filter to get our $K$ rows.
+          This is a recall and performance challenge, because if $K' \gg K$, then we have a lot of wasted effort.
+          If on the other hand $K' > K$ and thus $\sigma(K') < K$, then we have low recall.
+\end{itemize}
+
+One way around this issue is to apply an \textit{inline filter}, i.e. during \acrshort{ann} search, we do vector search, evaluate the stopping condition and apply the filter.
+This gives us $K$ rows directly and a handy performance uplift to go along with it.
+
+Sounds too good to be true? That's because it is. There are still some issues, namely if filtered out nodes don't participate in the navigation.
+In that case, the search stops if we reach such a filtered node, but there would be nodes that are closer to the query behind this node,
+which due to the removal of the filtered out node are now no longer accessible.
+
+This means that we need to have the filtered nodes participate in the navigation, which increases the cost by increasing the number of needed distance computations.
+
+Thus, \textit{filters increase the search effort and difficulty of tuning search parameters}.
+Selective filters make it harder to find valid nodes.
+
+This is where the \texttt{Value-Vector-Correlation} comes in. It captures the relationship between the \bi{probability of satisfying the filters}
+and the \bi{distance from the query vector}.
+It doesn't however always correlate positively and the correlation depends on the query.
+For instance, a query like ``Item like a blue T-Shirt and price > \$10K'' is negatively correlated.
+
+
+\subsubsection{Hash-Tree Index}
+If we use a Hash-Tree instead of the HNSW index, with filtering the internal node navigation does not change, as the data vectors are only in the leaves,
+so we simply filter there.
+
+
+\subsubsection{Performance Challenges}
+The cost of a vector search is given by $f(n_d \cdot c_d, n_f \cdot c_f)$, where $n_d$ is the number of distance computations, $n_f$ is the number of filter evaluations
+and $c_i$ and $c_f$ are the associated costs.
+
+Depending on the query, the cost very much differs, in that for 20\% selectivity, filtering first is faster, while for 50\% selectivity, doing the NNS first is faster.
+
+Since the distance computation and associated access costs are relative to the vector size,
+we could use Principal Component Analysis (PCA) or others to reduce the dimension of the vectors.
+Alternatively, we can quantize some values, with the drawback of reducing precision.
+Trees offer more residualization, so the best of both worlds is to score fast with quantized vectors, then re-score with the full precision on just a subset.
+
+A further optimization is to change how much of the set we re-score vs how much of it we score.
+
+Not all dataset and query combinations are equally easy, and filters also change the difficulty.
+
+
+\subsubsection{Filter agnosticism}
+Filter agnosticism isn't just a single type, it's a spectrum.
+
+\begin{figure}[h!]
+    \begin{center}
+        \includegraphics[width=0.8\textwidth]{assets/filter-agnosticism.png}
+    \end{center}
+    \caption{Spectrum of filter agnosticism
+        (Figure from lecture slides 18, slide 65)}
+\end{figure}
+
+Ideally, we would build an index per filter, but there are issues:
+\begin{itemize}
+    \item Footprint is too high because of duplicated nodes
+    \item Not enough data to build an index on the small partitions
+\end{itemize}
+We can emulate this partitioning with efficient footprint using a monolithic graph with pruning.
+
+\inlinedefinition[Value-induced neighbourhood] We can build a new index based on attributes and embedding similarity (embedding distance)
+
+\inlinedefinition[Predicate Traversal] Alternatively we can reuse unfiltered indexes and use inline filtering to discover filter-passing nodes,
+then light up the right neighbourhood at search time.
+
+\inlinedefinition[Densified Predicate Traversal] We also add edges to alleviate connectivity issues and proceed as with the predicate traversal

semester4/dmdb/parts/05_vector-search/main.tex

@@ -1,4 +1,6 @@
 \section{Vector Search}
 \input{parts/05_vector-search/00_intro-trees-clustering.tex}
 \input{parts/05_vector-search/01_hsnw-index.tex}
+\input{parts/05_vector-search/02_hash-index.tex}
+\input{parts/05_vector-search/03_filtered-vector-search.tex}
 % \input{parts/05_vector-search/}