Matryoshka representation learning (MRL) trains an embedding model so that the first part of every vector is a usable embedding on its own. Keep the first 256 or 128 of 768 numbers and you still have a working embedding, a little less accurate and several times smaller, so you can store and search short vectors and keep the full ones for a final, precise pass.

It comes from a 2022 paper by Kusupati and colleagues, presented at NeurIPS 2022, and it's now built into many embedding models, including Google's EmbeddingGemma and Gemini embeddings, Qwen3-Embedding and nomic-embed-text-v1.5. Below is how it works, the loss behind it, code to train and use it, which models support it, what you lose at smaller sizes, and how we've used it.

What is Matryoshka Representation Learning (MRL)?

A normal embedding model gives you one vector of a fixed size, say 1,024 numbers, and you can't make it smaller without retraining or losing meaning. MRL trains one model whose vector is nested: the first 64 numbers are a coarse embedding, the first 128 a better one, and so on up to the full size. The paper calls these "coarse-to-fine" representations, and reports that each nested size is at least as accurate as a model trained separately at that size.

It isn't a new architecture. It's a change to how an existing model is trained, and the paper says it adds no cost at inference or deployment. Most embedding models can be trained or fine-tuned this way.

Why is it called Matryoshka?

After Russian nesting dolls, where each smaller doll sits inside a larger one. A 128-dimension Matryoshka embedding isn't a separate vector: it's literally the first 128 values of the 256-dimension one, which is the first 256 values of the 768-dimension one.

Shortlisting and reranking

Nesting makes a two-step search cheap:

Shortlist with a short prefix. Search all your vectors using only the first 64 or 128 dimensions, which is fast and light on memory, and keep the top few hundred.
Rerank that shortlist with the full vectors. Only a few hundred comparisons use the expensive size.

The paper calls this adaptive retrieval and reports up to 14x real-world speed-ups for large-scale retrieval on ImageNet-1K and 4K, and up to 14x smaller embeddings for ImageNet-1K classification at the same accuracy. Those are image benchmarks, so measure the gain on your own text. One rule holds everywhere: a document the shortlist misses can't be rescued by the rerank, so check the shortlist's recall first.

How does MRL work?

During training, a normal model computes one loss on the full embedding. MRL computes the same loss several times, once on each chosen prefix of the embedding, and adds the results up, so the model learns to pack the most important information into the first dimensions.

The prefix sizes are fixed before training. The paper uses at most log(d) of them for a d-dimension embedding, usually halving the size each time. For a 2,048-dimension ResNet50 embedding, it trains on 8, 16, 32 and so on up to 2,048. Sizes in between still work: the paper found the representations interpolate between the trained sizes.

The math behind MRL

Here is the training objective from the paper (equation 1), written out in plain text:

minimize over W(m) and θ:
    (1/N) × sum over training examples i
          × sum over sizes m in M
          of c_m × L( W(m) · F(x_i; θ)[1:m] ; y_i )

F(x_i; θ) is the embedding the model produces for input x_i, with [1:m] keeping its first m dimensions.
M is the set of nested sizes you train on, such as {64, 128, 256, 512, 768}.
L is the usual loss for the task, such as cross-entropy for classification or a contrastive loss for embeddings, and W(m) is a small classifier for size m in the classification setup.
c_m weights each size. The paper sets every c_m to 1, so all sizes count equally. You could weight small sizes more if you mostly search at small sizes.

A simpler way to read it: loss = loss at 64 + loss at 128 + loss at 256 + loss at 512 + loss at 768. The paper also describes a cheaper variant, MRL-E, that shares one set of classifier weights across all sizes, and notes that normalization should be handled separately for each nested size.

Standard way vs Matryoshka way of training

Standard embedding training:

Pass the input through the model to get one full-size embedding, say 768 dimensions.
Compute one loss on that embedding.
Update the weights from that loss.

Matryoshka training:

Pass the input through the model to get one full-size embedding.
Compute the loss on the full embedding and on each prefix: vector[:512], vector[:256], vector[:128], vector[:64].
Add the losses, optionally weighted, into one number.
Update the weights from that combined loss.

In Sentence Transformers, that's a wrapper around the loss you'd use anyway. This is the setup from its Matryoshka training guide:

from sentence_transformers import SentenceTransformer
from sentence_transformers.sentence_transformer.losses import CoSENTLoss, MatryoshkaLoss

# Loading in fp32 is preferred for training if your memory can handle it
model = SentenceTransformer("microsoft/mpnet-base", model_kwargs={"torch_dtype": "float32"})
base_loss = CoSENTLoss(model=model)
loss = MatryoshkaLoss(model=model, loss=base_loss, matryoshka_dims=[768, 512, 256, 128, 64])

Then train as you normally would. The guide trains tomaarsen/mpnet-base-nli-matryoshka the same way, with MatryoshkaLoss wrapped around MultipleNegativesRankingLoss, and compares it with the same model trained without the Matryoshka loss.

Code implementation

To use a Matryoshka model at a smaller size, ask Sentence Transformers to truncate and normalize. This compares a similar and an unrelated sentence at each size:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("tomaarsen/mpnet-base-nli-matryoshka")

sentences = [
    "A person is riding a motorcycle on a road.",
    "A man is on a motorbike on a street.",
    "The cat is sleeping on the kitchen table.",
]

for dim in [768, 512, 256, 128, 64]:
    embeddings = model.encode(sentences, truncate_dim=dim, normalize_embeddings=True)
    scores = model.similarity(embeddings[:1], embeddings[1:])
    print(dim, embeddings.shape, scores.tolist())

truncate_dim keeps the first dim values and normalize_embeddings rescales each vector to length 1 after truncating (API reference). At every size, the motorbike sentence should score well above the cat sentence. If the gap collapses at the smallest size, that size is too small for the task. You can also set truncate_dim once when you load the model.

Which models support Matryoshka embeddings?

Only models trained for it. Slicing a vector from a model that wasn't trained this way throws away information the model spread across all its dimensions. These support shorter vectors, per their docs on October 16, 2026:

Model or API	Full size	Shorter sizes
EmbeddingGemma-300m	768	512, 256 or 128, then re-normalize
Gemini embeddings (gemini-embedding-001 and gemini-embedding-2)	3,072	128 to 3,072 through `output_dimensionality`; 768 and 1,536 recommended
Qwen3-Embedding	1,024, 2,560 or 4,096	Any size from 32
nomic-embed-text-v1.5	768	512, 256, 128 or 64
snowflake-arctic-embed-l-v2.0	1,024	256
pplx-embed-v1	1,024 or 2,560	Supported, per the model card
OpenAI text-embedding-3	1,536 (small) or 3,072 (large)	Any size through the `dimensions` parameter

Google's docs say both Gemini models are trained with MRL, and that gemini-embedding-2 re-normalizes shortened vectors for you. OpenAI's embeddings guide says its text-embedding-3 models were trained with "a technique" that lets you shorten embeddings without losing their concept-representing properties, and its January 2024 launch post names that technique, Matryoshka Representation Learning, in a footnote. For more on picking a model, see the embedding models ranking.

How much quality do you lose at smaller sizes?

Less than the size drop suggests, but not nothing. The clearest published curve is nomic-embed-text-v1.5's, an average over the original MTEB benchmark (model card):

Dimensions	MTEB average	Share of the 768-dimension score
768	62.28	100%
512	61.96	99.5%
256	61.04	98.0%
128	59.34	95.3%
64	56.10	90.1%

The last column is our arithmetic. OpenAI reports a similar effect: on MTEB, text-embedding-3-large shortened to 256 dimensions still beats its older text-embedding-ada-002 at a full 1,536 (OpenAI docs). Averages can hide the hard cases, such as exact identifiers and rare terms, so test the queries you care about.

Calculate the storage part precisely

For one million float32 vectors, 768 dimensions take 3.072 GB of raw values. At 128 dimensions they take 0.512 GB: one-sixth as many values, or about 83.3% less raw vector storage.

That arithmetic excludes index structures, IDs, metadata, replicas, backups and any full vectors you keep for reranking, so it isn't an 83.3% cut in total cost. And if the model still computes the full vector before you truncate it, encoding doesn't get cheaper by the same factor.

MRL also stacks with quantization, which cuts the bits per number instead of the number of dimensions. Snowflake's arctic-embed v2.0 card describes 128 bytes per vector by combining its 256-dimension Matryoshka size with 4-bit quantization (model card).

Follow the model's preprocessing contract

Normalize after you truncate. A prefix of a unit-length vector isn't unit length anymore. Cosine similarity doesn't mind, but dot product does, and many vector indexes assume unit-length vectors. OpenAI's docs say to normalize when you shorten an embedding yourself, and EmbeddingGemma's card says to truncate, then re-normalize. This helper does just that step on a vector you already have:

import math

def normalized_prefix(vector, dimensions):
    """Keep the first `dimensions` values of a Matryoshka embedding and rescale to length 1."""
    if not 1 <= dimensions <= len(vector):
        raise ValueError("dimensions must be between 1 and the vector's length")
    prefix = [float(x) for x in vector[:dimensions]]
    norm = math.sqrt(sum(x * x for x in prefix))
    if norm == 0:
        raise ValueError("cannot normalize an all-zero prefix")
    return [x / norm for x in prefix]

Use the same model version, size and normalization for queries and documents. A 128-dimension query compared against 768-dimension documents isn't a meaningful search. If the API can return a smaller size directly, ask for it rather than slicing, and check whether it comes back normalized.

How we use MRL at supermemory and our learnings

When this guide first ran in October 2025, our team wrote up how the memory engine used Matryoshka embeddings at the time:

Why: full-length embeddings meant big indexes, slower queries and rising storage.
What we did: moved to Matryoshka-style embeddings and kept them in production. Beyond slicing, we L2-normalized every prefix so a 64-, 128- or 256-dimension slice sat in a stable geometry, then applied lightweight quantization after normalizing.
Two views of each vector: we kept the full embedding for reranking and a compact view for fast paths. Storage went up a little overall, but index size and query speed improved where it counted.
What we saw: halving the dimension kept quality nearly intact for our workload, and going down to about a third still preserved most of it. That let us shortlist on small prefixes and save the full vector for the rerank.
The tradeoff: slicing can lose interactions that only exist in the full space. For us it was worth it.

The team also found that normalizing prefixes tightened precision and pulled in items that had been missed before, which fits the paper's note that normalization should be handled for each nested size separately.

You don't manage any of this on the hosted product: embeddings are handled for you. On the self-hosted server, you choose the model and its size. A reduced size is allowed only for OpenAI's text-embedding-3 models and Gemini, local models must use their native size, and the size is locked once vectors are stored.

How to test a smaller size on your data

Compare the full size and one or two smaller sizes under the same candidate budget, with everything else fixed: model version, prompts, preprocessing, index settings, corpus version and labeled relevant chunks. Report recall alongside memory use and latency.

For a two-stage setup, measure the shortlist's recall before you look at final rankings, and record where the full vectors live and whether fetching them adds tail latency. A single model doesn't mean a single index or zero extra storage.

For code search, include exact symbol names, natural-language descriptions, call relationships, version changes and requests for code that doesn't exist, and keep repository and access filters the same across sizes. If shortened vectors start missing exact identifiers, hybrid search adds keyword matching back, and the vector search guide covers the index side. For ranking evaluation, see contextual reranking.

Common questions

Can I truncate any embedding?

No. Only models trained with MRL or a similar method keep their quality when shortened. Supermemory's self-hosted docs, for example, list bge, e5 and MiniLM models as local models that don't support dimension reduction.

Do Matryoshka embeddings make encoding cheaper?

Usually not. The model still computes the full vector, and you keep a prefix. The savings come in storage, memory and search, and the paper notes that MRL adds no cost at inference.

What's the difference between MRL and quantization?

MRL cuts the number of dimensions, and quantization cuts the bits stored per dimension, for example from 32-bit floats to 8-bit integers or single bits. They work together, and some models, such as pplx-embed-v1, return int8 vectors by default.

Matryoshka representation learning and embeddings: how it works and how we use it