mirror of
https://github.com/janishutz/eth-summaries.git
synced 2026-07-27 15:24:37 +02:00
54 lines
2.9 KiB
TeX
54 lines
2.9 KiB
TeX
\subsection{Hierarchical Navigable Small World Index (HNSW Index)}
|
|
The name used in the lecture was HSNW (Hierarchical Small Navigable World) Index.
|
|
|
|
This is a layered graph concept, at insert, first all nodes are inserted into layer 0 and each is connected to $M \approx 3 - 4$ neighbours.
|
|
Then, the edge count is $\tco{N \cdot M}$. With $N = 100$ million and $M = 16$, we have $1.6B$ edges, with a typical memory footprint of $4-8$GB for $100M$ 32-bit Edge IDs.
|
|
Then, some nodes (typically about $N_{L_k} = N / M^k$ to layer $k$) are promoted to layer 1, connected to $M$ nearest neighbours in $L1$ only.
|
|
This process is repeated a few times until the number of nodes in the layer is low enough.
|
|
This number depends on a number of factors, primarily how coarse grained we want the initial search to be.
|
|
|
|
Then, after building, we prune edges, in a process called RNG pruning.
|
|
Here we prune an edge $(u, v)$ if there exist edges $(u, w)$ and $(w, v)$, such that $d(u, w) < d(u, v)$ and $d(w, v) < d(u, v)$.
|
|
Important is that both conditions have to be met, as the goal is to remove the longest redundant edge from the graph.
|
|
This is done using a greedy algorithm.
|
|
|
|
Higher values of \texttt{ef\_construction} generally improves the quality of the index at the cost of slower index construction.
|
|
|
|
|
|
\subsubsection{Search}
|
|
We note that greedy search always makes progress in this index because the graph stays navigable.
|
|
|
|
HNSW index search works by starting at a fixed entry, then doing \textit{one} greedy hop only.
|
|
The hop goes towards the neighbour on this layer that is closer to the query.
|
|
This is often done using Beam Search (see below) to find the $K$ closest vectors.
|
|
|
|
After some number of hops, there are no more neighbours in layer $LN$ that are closer to check, so descend one layer and repeat.
|
|
Traversal of the layer costs $\tco{M_L}$, with $M_L$ the number of edges on layer $L$.
|
|
Higher up layers have fewer edges, thus making traversal cheap.
|
|
|
|
The parameter \texttt{ef\_search} controls the recall vs latency.
|
|
Set to $1$ it is approximately greedy, for $\texttt{ef\_search} \gg 1$ means more effort, i.e. more vectors visited,
|
|
but also increases recall.
|
|
|
|
|
|
\paragraph{Beam Search}
|
|
This is another greedy search algorithm, which chooses the next node to visit from a beam of size $N$.
|
|
The goal is to find the closest node of the query, with $d$ the distance to the query.
|
|
|
|
The procedure is simple:
|
|
\begin{algorithm}
|
|
\caption{Beam Search}
|
|
\begin{algorithmic}[1]
|
|
\Procedure{BeamSearch}{$G$, start\_node}
|
|
\State $\texttt{Beam} \gets$ new beam data structure
|
|
\State \texttt{Beam.add(start\_node)}
|
|
\While{\texttt{beam} in \texttt{Beam}}
|
|
\State Check distances of neighbours and add them to \texttt{Beam}
|
|
\State Truncate \texttt{Beam} to size \texttt{max\_beam} \Comment{This is a tunable parameter}
|
|
\EndWhile
|
|
\State \Return \texttt{closest}
|
|
\EndProcedure
|
|
\end{algorithmic}
|
|
\end{algorithm}
|
|
|