diff --git a/semester4/dmdb/assets/filter-agnosticism.png b/semester4/dmdb/assets/filter-agnosticism.png new file mode 100644 index 0000000..31b39e7 Binary files /dev/null and b/semester4/dmdb/assets/filter-agnosticism.png differ diff --git a/semester4/dmdb/data-modelling-databases-summary.pdf b/semester4/dmdb/data-modelling-databases-summary.pdf index b26dc16..8aeec5a 100644 Binary files a/semester4/dmdb/data-modelling-databases-summary.pdf and b/semester4/dmdb/data-modelling-databases-summary.pdf differ diff --git a/semester4/dmdb/glossary/acronyms.tex b/semester4/dmdb/glossary/acronyms.tex index b316326..721c7b7 100644 --- a/semester4/dmdb/glossary/acronyms.tex +++ b/semester4/dmdb/glossary/acronyms.tex @@ -3,3 +3,5 @@ \newacronym{simd}{SIMD}{Single Instruction Multiple Data} \newacronym{avxa}{AVX}{Advanced Vector Extensions, also see \gls{avx}} \newacronym{ra}{RA}{Relational Algebra, see Section~\ref{sec:relational-algebra}} +\newacronym{ann}{ANN}{Approximate Nearest Neighbour Search} +\newacronym{knn}{KNN}{$K$-Nearest Neighbour Search} diff --git a/semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex b/semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex index 967e681..00b6691 100644 --- a/semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex +++ b/semester4/dmdb/parts/05_vector-search/01_hsnw-index.tex @@ -1,4 +1,6 @@ \subsection{Hierarchical Navigable Small World Index (HNSW Index)} +The name used in the lecture was HSNW (Hierarchical Small Navigable World) Index. + This is a layered graph concept, at insert, first all nodes are inserted into layer 0 and each is connected to $M \approx 3 - 4$ neighbours. Then, the edge count is $\tco{N \cdot M}$. With $N = 100$ million and $M = 16$, we have $1.6B$ edges, with a typical memory footprint of $4-8$GB for $100M$ 32-bit Edge IDs. Then, some nodes (typically about $N_{L_k} = N / M^k$ to layer $k$) are promoted to layer 1, connected to $M$ nearest neighbours in $L1$ only. @@ -12,3 +14,36 @@ This is done using a greedy algorithm. \subsubsection{Search} +We note that greedy search always makes progress in this index because the graph stays navigable. + +HNSW index search works by starting at a fixed entry, then doing \textit{one} greedy hop only. +The hop goes towards the neighbour on this layer that is closer to the query. +This is often done using Beam Search (see below) to find the $K$ closest vectors. + +After some number of hops, there are no more neighbours in layer $LN$ that are closer to check, so descend one layer and repeat. +Traversal of the layer costs $\tco{M_L}$, with $M_L$ the number of edges on layer $L$. +Higher up layers have fewer edges, thus making traversal cheap. + +The parameter \texttt{ef\_search} controls the recall vs latency. +Set to $1$ it is approximately greedy, for $\texttt{ef\_search} \gg 1$ means more effort, i.e. more vectors visited. + +\paragraph{Beam Search} +This is another greedy search algorithm, which chooses the next node to visit from a beam of size $N$. +The goal is to find the closest node of the query, with $d$ the distance to the query. + +The procedure is simple: +\begin{algorithm} + \caption{Beam Search} + \begin{algorithmic}[1] + \Procedure{BeamSearch}{$G$, start\_node} + \State $\texttt{Beam} \gets$ new beam data structure + \State \texttt{Beam.add(start\_node)} + \While{\texttt{beam} in \texttt{Beam}} + \State Check distances of neighbours and add them to \texttt{Beam} + \State Truncate \texttt{Beam} to size \texttt{max\_beam} \Comment{This is a tunable parameter} + \EndWhile + \State \Return \texttt{closest} + \EndProcedure + \end{algorithmic} +\end{algorithm} + diff --git a/semester4/dmdb/parts/05_vector-search/02_hash-index.tex b/semester4/dmdb/parts/05_vector-search/02_hash-index.tex new file mode 100644 index 0000000..640ae08 --- /dev/null +++ b/semester4/dmdb/parts/05_vector-search/02_hash-index.tex @@ -0,0 +1,16 @@ +\subsection{Hash Index} +To measure the quality of \acrshort{ann} in comparison to \acrshort{knn}, we use $\texttt{recall@k} = |\texttt{ANN\_result} \cap \texttt{KNN\_result}| \div k$. +This however is not always a good metric, as two vastly different results, one objectively worse can score equally well. + +Alternative methods mentioned are \texttt{RDE@k} and \texttt{TDK@k}, but not elaborated any further. + + +\subsubsection{Performance vs Recall Trade-off} +This is a challenging task, primarily focusing on how many vectors we need to visit to achieve a user-specified target recall. + +A few explored strategies are: +\begin{itemize} + \item \bi{Uniform autotuning for all queries}: Using learned offline models (e.g. Google CloudSQL VectorAssist) + \item \bi{Different for each query}: A model predicts the search effort parameters for each query or index + \item \bi{Adaptive}: Decide to continue or stop early based on the current search state +\end{itemize} diff --git a/semester4/dmdb/parts/05_vector-search/03_filtered-vector-search.tex b/semester4/dmdb/parts/05_vector-search/03_filtered-vector-search.tex new file mode 100644 index 0000000..7c04d26 --- /dev/null +++ b/semester4/dmdb/parts/05_vector-search/03_filtered-vector-search.tex @@ -0,0 +1,78 @@ +\subsection{In Databases} +The benefit of supporting vector search in DBMS is that this makes it easy to use for the end user and can be optimized for large datasets. + +\subsection{Filtered Vector Search (FVS)} +This allows us to query both structured and unstructured data. In concept, this is SQL + Vectors, sometimes plus joins, subquery expressions, +multiple vector searches (on both sides of the join). Sometimes, the dataset isn't even a vector + tag stored in main memory. + +Indices are typically built on unfiltered data. Thus, we need to think about the proper execution method. Suppose we are to retrieve $K$ rows with filtering: +\begin{itemize} + \item \bi{Pre-Filter}: Filter first, then run \texttt{NNS}, and we get the $K$ rows. This is expensive if the filter is not selective. + \item \bi{Post-Filter}: We first run \acrshort{ann} search, from which we get $K'$ rows which we then filter to get our $K$ rows. + This is a recall and performance challenge, because if $K' \gg K$, then we have a lot of wasted effort. + If on the other hand $K' > K$ and thus $\sigma(K') < K$, then we have low recall. +\end{itemize} + +One way around this issue is to apply an \textit{inline filter}, i.e. during \acrshort{ann} search, we do vector search, evaluate the stopping condition and apply the filter. +This gives us $K$ rows directly and a handy performance uplift to go along with it. + +Sounds too good to be true? That's because it is. There are still some issues, namely if filtered out nodes don't participate in the navigation. +In that case, the search stops if we reach such a filtered node, but there would be nodes that are closer to the query behind this node, +which due to the removal of the filtered out node are now no longer accessible. + +This means that we need to have the filtered nodes participate in the navigation, which increases the cost by increasing the number of needed distance computations. + +Thus, \textit{filters increase the search effort and difficulty of tuning search parameters}. +Selective filters make it harder to find valid nodes. + +This is where the \texttt{Value-Vector-Correlation} comes in. It captures the relationship between the \bi{probability of satisfying the filters} +and the \bi{distance from the query vector}. +It doesn't however always correlate positively and the correlation depends on the query. +For instance, a query like ``Item like a blue T-Shirt and price > \$10K'' is negatively correlated. + + +\subsubsection{Hash-Tree Index} +If we use a Hash-Tree instead of the HNSW index, with filtering the internal node navigation does not change, as the data vectors are only in the leaves, +so we simply filter there. + + +\subsubsection{Performance Challenges} +The cost of a vector search is given by $f(n_d \cdot c_d, n_f \cdot c_f)$, where $n_d$ is the number of distance computations, $n_f$ is the number of filter evaluations +and $c_i$ and $c_f$ are the associated costs. + +Depending on the query, the cost very much differs, in that for 20\% selectivity, filtering first is faster, while for 50\% selectivity, doing the NNS first is faster. + +Since the distance computation and associated access costs are relative to the vector size, +we could use Principal Component Analysis (PCA) or others to reduce the dimension of the vectors. +Alternatively, we can quantize some values, with the drawback of reducing precision. +Trees offer more residualization, so the best of both worlds is to score fast with quantized vectors, then re-score with the full precision on just a subset. + +A further optimization is to change how much of the set we re-score vs how much of it we score. + +Not all dataset and query combinations are equally easy, and filters also change the difficulty. + + +\subsubsection{Filter agnosticism} +Filter agnosticism isn't just a single type, it's a spectrum. + +\begin{figure}[h!] + \begin{center} + \includegraphics[width=0.8\textwidth]{assets/filter-agnosticism.png} + \end{center} + \caption{Spectrum of filter agnosticism + (Figure from lecture slides 18, slide 65)} +\end{figure} + +Ideally, we would build an index per filter, but there are issues: +\begin{itemize} + \item Footprint is too high because of duplicated nodes + \item Not enough data to build an index on the small partitions +\end{itemize} +We can emulate this partitioning with efficient footprint using a monolithic graph with pruning. + +\inlinedefinition[Value-induced neighbourhood] We can build a new index based on attributes and embedding similarity (embedding distance) + +\inlinedefinition[Predicate Traversal] Alternatively we can reuse unfiltered indexes and use inline filtering to discover filter-passing nodes, +then light up the right neighbourhood at search time. + +\inlinedefinition[Densified Predicate Traversal] We also add edges to alleviate connectivity issues and proceed as with the predicate traversal diff --git a/semester4/dmdb/parts/05_vector-search/main.tex b/semester4/dmdb/parts/05_vector-search/main.tex index dedff59..8774e64 100644 --- a/semester4/dmdb/parts/05_vector-search/main.tex +++ b/semester4/dmdb/parts/05_vector-search/main.tex @@ -1,4 +1,6 @@ \section{Vector Search} \input{parts/05_vector-search/00_intro-trees-clustering.tex} \input{parts/05_vector-search/01_hsnw-index.tex} +\input{parts/05_vector-search/02_hash-index.tex} +\input{parts/05_vector-search/03_filtered-vector-search.tex} % \input{parts/05_vector-search/}