\subsection{Hierarchical Navigable Small World Index (HNSW Index)}
The name used in the lecture was HSNW (Hierarchical Small Navigable World) Index.

This is a layered graph concept, at insert, first all nodes are inserted into layer 0 and each is connected to $M \approx 3 - 4$ neighbours.
Then, the edge count is $\tco{N \cdot M}$. With $N = 100$ million and $M = 16$, we have $1.6B$ edges, with a typical memory footprint of $4-8$GB for $100M$ 32-bit Edge IDs.
Then, some nodes (typically about $N_{L_k} = N / M^k$ to layer $k$) are promoted to layer 1, connected to $M$ nearest neighbours in $L1$ only.
This process is repeated a few times until the number of nodes in the layer is low enough.
This number depends on a number of factors, primarily how coarse grained we want the initial search to be.

Then, after building, we prune edges, in a process called RNG pruning.
Here we prune an edge $(u, v)$ if there exist edges $(u, w)$ and $(w, v)$, such that $d(u, w) < d(u, v)$ and $d(w, v) < d(u, v)$.
Important is that both conditions have to be met, as the goal is to remove the longest redundant edge from the graph.
This is done using a greedy algorithm.

Higher values of \texttt{ef\_construction} generally improves the quality of the index at the cost of slower index construction.


\subsubsection{Search}
We note that greedy search always makes progress in this index because the graph stays navigable.

HNSW index search works by starting at a fixed entry, then doing \textit{one} greedy hop only.
The hop goes towards the neighbour on this layer that is closer to the query.
This is often done using Beam Search (see below) to find the $K$ closest vectors.

After some number of hops, there are no more neighbours in layer $LN$ that are closer to check, so descend one layer and repeat.
Traversal of the layer costs $\tco{M_L}$, with $M_L$ the number of edges on layer $L$.
Higher up layers have fewer edges, thus making traversal cheap.

The parameter \texttt{ef\_search} controls the recall vs latency.
Set to $1$ it is approximately greedy, for $\texttt{ef\_search} \gg 1$ means more effort, i.e. more vectors visited,
but also increases recall.


\paragraph{Beam Search}
This is another greedy search algorithm, which chooses the next node to visit from a beam of size $N$.
The goal is to find the closest node of the query, with $d$ the distance to the query.

The procedure is simple:
\begin{algorithm}
    \caption{Beam Search}
    \begin{algorithmic}[1]
        \Procedure{BeamSearch}{$G$, start\_node}
            \State $\texttt{Beam} \gets$ new beam data structure
            \State \texttt{Beam.add(start\_node)}
            \While{\texttt{beam} in \texttt{Beam}}
                \State Check distances of neighbours and add them to \texttt{Beam}
                \State Truncate \texttt{Beam} to size \texttt{max\_beam} \Comment{This is a tunable parameter}
            \EndWhile
            \State \Return \texttt{closest}
        \EndProcedure
    \end{algorithmic}
\end{algorithm}