[DMDB] Finished vector search

semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex

@@ -1,4 +1,6 @@
 \subsection{Hierarchical Navigable Small World Index (HNSW Index)}
+The name used in the lecture was HSNW (Hierarchical Small Navigable World) Index.
+
 This is a layered graph concept, at insert, first all nodes are inserted into layer 0 and each is connected to $M \approx 3 - 4$ neighbours.
 Then, the edge count is $\tco{N \cdot M}$. With $N = 100$ million and $M = 16$, we have $1.6B$ edges, with a typical memory footprint of $4-8$GB for $100M$ 32-bit Edge IDs.
 Then, some nodes (typically about $N_{L_k} = N / M^k$ to layer $k$) are promoted to layer 1, connected to $M$ nearest neighbours in $L1$ only.
@@ -12,3 +14,36 @@ This is done using a greedy algorithm.
 
 
 \subsubsection{Search}
+We note that greedy search always makes progress in this index because the graph stays navigable.
+
+HNSW index search works by starting at a fixed entry, then doing \textit{one} greedy hop only.
+The hop goes towards the neighbour on this layer that is closer to the query.
+This is often done using Beam Search (see below) to find the $K$ closest vectors.
+
+After some number of hops, there are no more neighbours in layer $LN$ that are closer to check, so descend one layer and repeat.
+Traversal of the layer costs $\tco{M_L}$, with $M_L$ the number of edges on layer $L$.
+Higher up layers have fewer edges, thus making traversal cheap.
+
+The parameter \texttt{ef\_search} controls the recall vs latency.
+Set to $1$ it is approximately greedy, for $\texttt{ef\_search} \gg 1$ means more effort, i.e. more vectors visited.
+
+\paragraph{Beam Search}
+This is another greedy search algorithm, which chooses the next node to visit from a beam of size $N$.
+The goal is to find the closest node of the query, with $d$ the distance to the query.
+
+The procedure is simple:
+\begin{algorithm}
+    \caption{Beam Search}
+    \begin{algorithmic}[1]
+        \Procedure{BeamSearch}{$G$, start\_node}
+            \State $\texttt{Beam} \gets$ new beam data structure
+            \State \texttt{Beam.add(start\_node)}
+            \While{\texttt{beam} in \texttt{Beam}}
+                \State Check distances of neighbours and add them to \texttt{Beam}
+                \State Truncate \texttt{Beam} to size \texttt{max\_beam} \Comment{This is a tunable parameter}
+            \EndWhile
+            \State \Return \texttt{closest}
+        \EndProcedure
+    \end{algorithmic}
+\end{algorithm}
+