🏗️ Building on HF

Tom Aarsen

tomaarsen

2309 737 2159

https://linkedin.com/in/tomaarsen

AI & ML interests

NLP: text embeddings, information retrieval, named entity recognition, few-shot text classification

Recent Activity

posted an update about 13 hours ago

🤗 I've released Sentence Transformers v6.3 and wrote a blogpost about its key new feature: token pooling-aware training, teaching multi-vector models to preserve retrieval quality with fewer document vectors. Details: Multi-vector models keep a vector per token, letting different query tokens match different parts of a document. Great for retrieval, but those vectors add up! Token pooling makes the representations smaller by merging similar document vectors into cluster means. Pooling-aware training teaches the encoder to preserve useful matches after those vectors are merged. With MultiVectorTokenPoolingLoss, you can wrap your standard retrieval loss to train on both full and pooled representations, reusing the same token embeddings without another encoder pass. In short: small overhead. I trained two ModernBERT-base models on the same 120k MS MARCO triplets, changing just the loss. At ~49 vectors per document instead of 193, pooling-aware training raises NanoBEIR@13 mean nDCG@10 from 49.03 to 51.23. That's 97.8% of its own unpooled score with ~4x fewer document vectors, compared with 94.6% retention for ordinary training. v6.3 also adds: - Adaptive token budgets, allocating vectors according to each document's content - Matryoshka training for shorter multi-vector token embeddings - Hugging Face kernels support for MaxSim, Ward clustering & supported transformers layers - Native AWS Neuron inference You can combine pooling-aware and Matryoshka training to learn both fewer vectors and fewer dimensions, then stack these with quantization to reduce storage per coordinate. For kernels, install sentence-transformers[kernels] and set use_kernels=True. The blog walks through the method, training code, checkpoints & evaluation, building on Stefan Josef's Learn to Pool and LightOn's pooling regularization work: ht tps://huggingface.co/blog/token-pooling I'd love to see what you train with this!

upvoted an article about 13 hours ago

Token Pooling-Aware Training for Multi-Vector Embedding Models

published an article 1 day ago

Token Pooling-Aware Training for Multi-Vector Embedding Models

View all activity

Organizations

buckets 8

🤗 I've released Sentence Transformers v6.3 and wrote a blogpost about its key new feature: token pooling-aware training, teaching multi-vector models to preserve retrieval quality with fewer document vectors. Details:

Multi-vector models keep a vector per token, letting different query tokens match different parts of a document. Great for retrieval, but those vectors add up! Token pooling makes the representations smaller by merging similar document vectors into cluster means.

Pooling-aware training teaches the encoder to preserve useful matches after those vectors are merged. With MultiVectorTokenPoolingLoss, you can wrap your standard retrieval loss to train on both full and pooled representations, reusing the same token embeddings without another encoder pass. In short: small overhead.

I trained two ModernBERT-base models on the same 120k MS MARCO triplets, changing just the loss. At ~49 vectors per document instead of 193, pooling-aware training raises NanoBEIR@13 mean nDCG@10 from 49.03 to 51.23. That's 97.8% of its own unpooled score with ~4x fewer document vectors, compared with 94.6% retention for ordinary training.

v6.3 also adds:
- Adaptive token budgets, allocating vectors according to each document's content
- Matryoshka training for shorter multi-vector token embeddings
- Hugging Face kernels support for MaxSim, Ward clustering & supported transformers layers
- Native AWS Neuron inference

You can combine pooling-aware and Matryoshka training to learn both fewer vectors and fewer dimensions, then stack these with quantization to reduce storage per coordinate. For kernels, install sentence-transformers[kernels] and set use_kernels=True.

The blog walks through the method, training code, checkpoints & evaluation, building on Stefan Josef's Learn to Pool and LightOn's pooling regularization work:
ht tps://huggingface.co/blog/token-pooling

I'd love to see what you train with this!