turbovec fits a 31 GB vector index into 4 GB of RAM

A quantized vector index trades a little accuracy for a large drop in memory, and turbovec pushes that trade hard. It holds 10 million embeddings in about 4 GB where float32 needs 31 GB. There is no training run first: vectors are searchable the moment you add them.

What a quantized vector index buys you

Memory decides what hardware a retrieval system needs. Ten million embeddings from a common 768-dimension model take about 31 GB as raw float32. Few laptops hold that much.

Compress each coordinate down to 4 bits and the same corpus lands near 4 GB. The turbovec project leads with that claim, and the arithmetic holds up. At half a byte per coordinate, 768 coordinates cost 384 bytes. Ten million of them come to roughly 3.8 GB.

An index that fits in RAM also keeps disk and network off the hot path, so queries come back sooner.

The project’s measured sizes for a 100,000-vector corpus run above the theoretical figure, because they include the index structures around the packed bits.

The project’s compression benchmark results land near 16x at 2-bit and 8x at 4-bit once the embeddings get wide. Compression slips at 200 dimensions, where the per-vector bookkeeping stops being negligible.

You pay for all of this in recall, since compressed vectors are approximations. It is also just the index, so you still need an embedding model and a document store around it.

How TurboQuant compresses without training

turbovec implements TurboQuant, an algorithm from Google Research published at ICLR 2026 by Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni. Credit for the algorithm belongs to them; turbovec is the code that runs it. The quantizer is data-oblivious, so its codebook comes from mathematics alone.

Every vector starts by losing its length. That norm gets stored as a single float, leaving a pure direction on a hypersphere. Each direction is then multiplied by the same random orthogonal matrix. After that rotation every coordinate follows a known distribution, whatever the input data looked like.

The known distribution is asymptotic, though, so at real dimensions the coordinates drift off the canonical shape. A calibration step fits a shift and a scale per coordinate, pulling the empirical quantiles back onto the codebook’s outer centroids. The codebook itself comes from Lloyd’s algorithm , which computes optimal bucket boundaries for the target distribution: 4 of them at 2-bit, 16 at 4-bit.

Finally the small integers get packed tightly into bytes. One extra scalar per vector then corrects the downward bias that scalar quantization puts into inner products.

Consequently there is no training phase. The codebook never depended on the corpus, so the ten millionth vector is encoded exactly like the first one. The index never needs a rebuild as it grows.

One stage does look at data. Calibration fits once after the index has seen 1,000 vectors, re-encodes the rows buffered before that point, then locks. Small incremental adds calibrate as well as one bulk load does.

The paper’s central claim is a proved bound.

Recall and speed against FAISS

The baseline is FAISS IndexPQ with a 256-entry lookup table and 8-bit codes, a harder target than the one in the original paper. FAISS scores against a higher-precision table and trains its codebook with k-means++, so beating it means more than beating a strawman.

On OpenAI-style embeddings at 1536 and 3072 dimensions, TurboQuant leads FAISS by 0.4 to 3.1 points at rank one, across both 2-bit and 4-bit. Both reach perfect recall by rank 8. GloVe vectors at 200 dimensions are the harder regime, since the distribution assumption is loosest at low dimensions. There TurboQuant still leads by 1.4 points at 4-bit and 0.5 points at 2-bit, and the two track each other closely by rank 16.

Every search-speed cell below is 100,000 vectors, 1,000 queries, k of 64, median of five runs, pinned to a single thread.

On ARM turbovec wins every configuration by 19 to 31 percent, measured on a Google Axion instance with 8 vCPUs. On x86 the margin collapses. The 4-bit configurations win by up to about 5 percent, and the 2-bit ones lose by up to about 8 percent single-threaded. FAISS has an AVX-512 VBMI path that suits the short 2-bit accumulate loop better than the hand-written kernel does. That loss is in the project’s own charts, which is a good sign about the rest of the numbers.

On x86 with one thread, bulk insertion runs at about 21,400 vectors per second at 1536 dimensions and 4-bit. That puts a million vectors at roughly 47 seconds. Let all 8 vCPUs work and it drops to about 10 seconds.

The ARM box is faster still, at about 386,000 vectors per second. Warm appends, with calibration already locked, hit roughly 770,000 per second. FAISS keeps one win here too. Its bulk add on x86 with 8 threads runs at about 117,000 vectors per second against turbovec’s 102,000. That figure leaves out the training pass FAISS has to run first. Search and insert both parallelize through Rayon, so the single-threaded table above is a floor.

Filtered search is the practical feature

Most vector libraries disappoint at hybrid retrieval. You want results from one tenant, one time window, or one access-control set, and the index knows nothing about any of those. The usual workaround is to over-fetch far more results than you need, then throw away the disallowed ones. That wastes time and costs recall.

turbovec honours an id allowlist inside the SIMD kernel itself, at a granularity of 32 vectors per block. Blocks with no allowed slots are short-circuited before any lookup or scoring happens, and disallowed slots inside a scored block are dropped at heap insert.

A narrow filter costs less than a wide one. Selective allowlists skip most of the SIMD work outright, while in other libraries the filter width makes no difference to search time.

The result contract is strict too. You get up to k results from the allowed set. When fewer than k vectors are allowed, you get exactly that many, with no padded fallbacks. That makes the two-stage pattern straightforward: let SQL, BM25, or an access-control list narrow the candidates, then rerank densely inside that set.

If you already work inside a framework, the project ships drop-in replacements for the in-tree stores in LangChain , LlamaIndex , Haystack , and Agno . Each keeps the same public surface and persistence semantics, so you swap the import and leave the pipeline alone.

How to build a compressed vector index in Python

The library is MIT licensed and written in Rust, with Python bindings published on PyPI .

Install it

Run pip install turbovec. Rust users can run cargo add turbovec instead and get the same index types.

Create the index

Call TurboQuantIndex(dim=1536, bit_width=4). The dimension has to match your embedding model. Choose 4-bit for higher recall and 2-bit when memory is tighter than accuracy.

Add your vectors

Call index.add(vectors) with a two-dimensional float32 array. Other dtypes are rejected rather than silently converted, so cast first with np.asarray(x, dtype=np.float32).

Search it

Call index.search(query, k=10). It returns scores and indices.

Switch to stable ids if you plan to delete

Use IdMapIndex with add_with_ids(vectors, ids) so removals don’t shift everything else along. Then index.remove(id) is constant time.

Filter at search time

Pass allowlist=allowed to search() once another system has narrowed the candidates. Filtering happens inside the kernel, before scoring.

Persist it

Call index.write("my_index.tv") and reload with TurboQuantIndex.load(...). The write is an fsync plus an atomic rename, so a crash mid-save leaves the old file intact.

Swap it into a framework

Install the extra for LangChain, LlamaIndex, Haystack, or Agno, then replace that framework’s in-memory store with the turbovec one.

Who should reach for it

turbovec suits a team that wants dense retrieval inside an existing service, on hardware it already pays for, with no vector database to run. The ARM result makes it particularly attractive on Graviton or Axion instances, where cheap cores are the whole point.

On x86 at 2-bit it is the weaker call, since FAISS still edges it out on search. If you want replication, sharding, or a query language, you want a database rather than a library like this one.

What the published numbers can’t tell you is how recall holds up on your own corpus. Every figure above comes from the project’s benchmark suite, run on its own datasets. The suite is scriptable and the datasets download with one command. Point it at your own embeddings before you trust the compression ratio with anything important.