A nearest-neighbor picture is a poor place to decide whether retrieval works. The picture has usually thrown away most of the dimensions, and the index has often thrown away most of the candidates. Those are two different losses.
This lab uses real 200-dimensional Word2Vec vectors from TensorFlow’s Embedding Projector. Search runs in all 200 dimensions over a fixed 2,048-row subset. The two-dimensional view is a seeded random projection, deliberately kept separate from the retrieval calculation.
Word2Vec is not a modern instruction-tuned retrieval model. It is useful here because the data is small, inspectable and attributable. The goal is to reason about the index and evaluation contract, not to claim a competitive semantic-search result.
Establish the oracle before tuning the approximation #
Pick a query word. The exact baseline scans every eligible vector under the selected metric and excludes the query itself. The approximate result searches only selected inverted lists.
Recall@10 measures overlap with that exact top ten:
That is an index-quality metric. It says nothing directly about whether a returned document answers a question. An exact index over a poor embedding model can have perfect recall against this oracle and still be useless to users.
Probe all sixteen lists at float32 precision, without post-filtering or a restricted refinement pool. Recall should be 100%. If it is not, stop tuning and investigate the contract: metric, normalization, eligibility rules, self-matches or tie-breaking.
The lab trains sixteen deterministic k-means lists with L2 routing. It is intentionally a small IVF baseline, not HNSW, product quantization or a claim about a production vector database.
There are several places to lose the right neighbor #
Start by lowering the probe count. The first loss is routing: a true neighbor belongs to a list that was never searched.
Then switch candidate precision to four-bit scalar quantization. Each vector is symmetrically quantized using its own scale. Now another loss appears: approximate scores can reorder candidates even within the lists you visited.
Finally, enable exact refinement of the leading 40 or 100 candidates. Refinement can correct score ordering for that candidate pool. It cannot recover a vector from an unprobed list, or one that the approximate ranking already pushed below the refinement cutoff.
These controls are useful precisely because they do not repair the same problem. More expensive reranking is not an answer to poor candidate recall.
The scalar-quantization mode rounds and reconstructs values for scoring; it is not a packed SIMD integer implementation. The memory panel estimates packed payload size, while the browser retains all representations for comparison. Enabling exact refinement also requires retaining or fetching the float32 originals. Leaving those originals out of a memory estimate would overstate the saving.
Filtering changes the retrieval problem #
Enable the synthetic tenant filter. One eighth of IDs belong to the selected tenant. This membership is a controlled test fixture, not metadata claimed by the source corpus.
With pre-filtering, only eligible candidates inside the probed lists compete for the top ten. With post-filtering, the index first takes a global top ten and then removes ineligible results. The latter can return fewer than ten even when the dataset contains plenty of eligible neighbors.
Notice the baseline: it is the exact top ten within the tenant. Comparing a filtered approximate result against an unfiltered oracle would make the recall figure meaningless.
A production system has more options than these two extremes. It can oversample, continue searching until it has enough eligible results, choose a filter-aware graph strategy, or scan an unusually selective subset exactly. Which choice is sensible depends on selectivity, its correlation with the embedding geometry and the cost of evaluating the predicate.
An authorization filter is also a security boundary. Moving it after retrieval must not expose excluded content through logs, caches, explanations or reranker inputs. The synthetic tenant switch models recall and result count only.
Do not read search distance off the scatterplot #
Blue points are true high-dimensional neighbors. Outlines mark retrieved points. Some will look surprisingly far apart in the projection.
The 2D neighbor agreement readout takes the nearest points in the displayed plane and compares their IDs with the real top ten. It is a direct check on how much this particular view loses.
TensorFlow’s Embedding Projector is the source of both the dataset and the useful habit of inspecting representations from several perspectives. The Distill t-SNE explainer, also in the CSV, makes a related warning especially well: local neighborhoods, cluster sizes and inter-cluster distances should not be read as if the visualization were the original metric space. This lab uses a random projection, not t-SNE; the numerical interpretation differs, but the need to name the projection does not.
Normalization is part of the index contract #
For unit vectors,
so cosine similarity, inner product and squared L2 produce equivalent exact rankings, aside from numerical ties. Remove unit normalization and vector magnitude can affect those rankings.
An embedding model may encode information in its norms. Discarding that information might help or hurt the intended task. “Cosine is standard” is not a sufficient reason to normalize an index trained for another scoring objective.
The coarse index here remains L2-trained even when raw inner-product scoring is selected. That is an intentionally visible mismatch, not an inner-product-optimized IVF implementation. Full probing still gives an exact baseline; sparse probing can behave differently.
A curve is more useful than one pleasing query #
Run the fixed twelve-query sweep. It reports mean recall, worst recall within that small set, and the number of vector scores including centroid routing and exact refinement.
Candidate count is a work proxy, not latency. Cache locality, vector dimension, SIMD kernels, memory layout and concurrency change the cost of a score. Nor are twelve common-word queries a representative evaluation set. The sweep makes a tradeoff reproducible; it does not establish an SLO.
For a real retrieval service, I would separate three evaluations: approximation error against exact search, relevance against task judgments, and end-to-end latency under load. I would also segment them by filter selectivity and query type. A single average can hide the queries most likely to produce a bad answer.
Data source: TensorFlow Embedding Projector, revision 6ff71f72, first 2,048 rows of its Word2Vec 10K, 200D demo dataset. Original row order and float32 values are preserved. This subset is frequency-biased and should not be mistaken for a document-retrieval corpus.