timbmg (Tim)

‼️Sentence Transformers v5.0 is out! The biggest update yet introduces Sparse Embedding models, encode methods improvements, Router module for asymmetric models & much more. Sparse + Dense = 🔥 hybrid search performance! Details:

1️⃣ Sparse Encoder Models
Brand new support for sparse embedding models that generate high-dimensional embeddings (30,000+ dims) where <1% are non-zero:

- Full SPLADE, Inference-free SPLADE, and CSR architecture support
- 4 new modules, 12 new losses, 9 new evaluators
- Integration with @elastic-co , @opensearch-project , @NAVER LABS Europe, @qdrant , @IBM , etc.
- Decode interpretable embeddings to understand token importance
- Hybrid search integration to get the best of both worlds

2️⃣ Enhanced Encode Methods & Multi-Processing
- Introduce encode_query & encode_document automatically use predefined prompts
- No more manual pool management - just pass device list directly to encode()
- Much cleaner and easier to use than the old multi-process approach

3️⃣ Router Module & Advanced Training
- Router module with different processing paths for queries vs documents
- Custom learning rates for different parameter groups
- Composite loss logging - see individual loss components
- Perfect for two-tower architectures

4️⃣ Comprehensive Documentation & Training
- New Training Overview, Loss Overview, API Reference docs
- 6 new training example documentation pages
- Full integration examples with major search engines
- Extensive blogpost on training sparse models

Read the comprehensive blogpost about training sparse embedding models: https://huggingface.co/blog/train-sparse-encoder

See the full release notes here: https://github.com/UKPLab/sentence-transformers/releases/v5.0.0

What's next? We would love to hear from the community! What sparse encoder models would you like to see? And what new capabilities should Sentence Transformers handle - multimodal embeddings, late interaction models, or something else? Your feedback shapes our roadmap!

Tim

AI & ML interests

Recent Activity

Organizations

SciCoQA Discrepancy Detection

SciCoQA Discrepancy Detection

Ocrbench Leaderboard

UKPLab/scicoqa

UKPLab/scicoqa

PeerQA: A Scientific Question Answering Dataset from Peer Reviews

Insights from the ICLR Peer Review and Rebuttal Process

Update README.md

Update README.md

sukannya/lazy_review_plus_emnlp_Qwen2.5-7B-Instruct

Supercharge your OCR Pipelines with Open Models

swe-qa/SWE-QA-Benchmark

The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors

boda/RevUtil_human

The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors

Seems to prefer shorter content? Model config issue?

Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking

Can model be loaded with Cross Encoder?

Tim

AI & ML interests

Recent Activity

Organizations

timbmg's activity

SciCoQA Discrepancy Detection

SciCoQA Discrepancy Detection

Ocrbench Leaderboard

Update README.md

Update README.md

Supercharge your OCR Pipelines with Open Models

Seems to prefer shorter content? Model config issue?

Can model be loaded with Cross Encoder?