November 16, 2026

RAG-Wissen on Qdrant: How We Chose Our Embedding Model and Sharding Setup

A collection design and an embedding model decide whether a RAG system scales across multiple tenants. Our decisions behind RAG-Wissen — and the reasoning.

RAG-Wissen is our internal knowledge base — originally built to make mandate knowledge searchable across client projects, and since extended to serve knowledge queries within individual client mandates themselves. Two architecture decisions determine whether a system like this stays maintainable past the third client, or whether every new client opens a new construction site: the choice of embedding model and the collection design in Qdrant, our vector database. We did not get either one right on the first attempt.

The embedding model choice: bge-m3 instead of a cloud endpoint

Our content is consistently bilingual — German and English, often within the same document. Since the second generation of RAG-Wissen, we have run BAAI/bge-m3 locally via Ollama. The model is based on the XLM-RoBERTa architecture, processes inputs from short sentences up to documents of up to 8192 tokens in a single pass according to the model description on ollama.com/library/bge-m3, and produces 1024-dimensional vectors; the Hugging Face model card (BAAI/bge-m3) states support for over 100 languages under the MIT license. For our use case, the decisive factor was that a single model maps German and English into the same vector space without quality loss — an English-only embedding model would have systematically delivered worse retrieval quality for German technical terms, and a per-language model switch would have meant two vector spaces and doubled infrastructure.

First version: one collection per client — and why it does not scale

Our first Qdrant architecture created a dedicated collection for every new client. At first this looked like the cleanest isolation: no chance that a query would accidentally return another client's data. In practice, the downside only showed up at scale: every new collection needed its own index parameters, its own monitoring, its own backup jobs — and an HNSW index per collection reserves memory independent of the actual data volume. With ten small per-client collections, we paid the memory overhead of ten indexes for a data volume that would barely have registered in a single collection.

The fix: payload-based partitioning instead of one collection per tenant

Qdrant's own multitenancy documentation recommends exactly this fix for this problem: a single collection with payload-based partitioning, where a keyword payload index with the `is_tenant=true` option on a tenant field keeps points belonging to the same tenant stored together and lets queries filter against it efficiently (source: qdrant.tech/documentation/guides/multitenancy). After the switch, tenant separation remained just as reliable — enforced through a payload filter that must be carried on every query, update, and delete, not optionally. That is the price of this model: isolation now lives in application logic, not automatically at the collection boundary. Forget that filter in a single code path, and you have a data leak between tenants — which is why we wrap access exclusively behind an internal query layer that enforces the tenant filter centrally, instead of re-adding it at every call site.

Scaling beyond plain separation: tiered multitenancy

For the largest clients, payload filtering alone eventually stops being enough — their queries should not compete with smaller clients' queries for the same shard resources. Since Qdrant 1.16, this can be solved directly within a single collection: small tenants share a fallback shard, large tenants get their own dedicated shards — configured through custom sharding, which routes queries to a specific shard group (source: qdrant.tech/documentation/production-operations/multitenant-search). For us, that means new, small client projects start on the shared fallback shard with no extra configuration work. When a client grows — say because they feed in their entire internal documentation — we move them onto a dedicated shard without touching the collection structure for everyone else.

What we got wrong the first time

Two assumptions from the first version did not hold up. First, we thought one collection per tenant was the safer way to rule out a data leak between clients — in practice, a centrally enforced query layer with a payload filter is just as safe, at a fraction of the operational cost. Second, we underestimated the cost of switching embedding models: a new model does not just mean new computations, it means fully re-embedding every historical document, because old and new vectors are not comparable in the same space. Since moving to bge-m3 — a model with stable, actively maintained versioning — we plan model switches deliberately as their own project with a re-indexing window, not as a side effect of a dependency update.

Frequently asked questions

Is payload-based partitioning worth it for just two or three tenants? Rarely, for isolation alone — the benefit shows up once the number of collections starts driving operational overhead, in our experience somewhere in the low double digits of tenants. Does every RAG system really need a multilingual embedding model? No — anyone processing exclusively single-language content often gets better retrieval quality from a specialized single-language model than from a multilingual all-rounder like bge-m3. Is Qdrant the right choice at every RAG scale? For small, one-off document sets, the operational overhead of running a dedicated vector database is often not justified — that is where simpler solutions are worth a look before sharding ever becomes a question.

The two decisions that keep RAG-Wissen maintainable across multiple tenants: a multilingual embedding model with stable versioning instead of a cloud endpoint per language, and payload-based partitioning instead of one collection per client, extended with tiered multitenancy for the large cases. At digital opua GmbH, we bring these decisions directly into our RAG advisory work — as architecture questions to settle before the first pilot, not after.

Sources: Ollama Model Library, "bge-m3" (ollama.com/library/bge-m3) — Hugging Face model card, BAAI/bge-m3 (huggingface.co/BAAI/bge-m3) — Qdrant Documentation, "Configure Multitenancy" (qdrant.tech/documentation/guides/multitenancy) — Qdrant Documentation, "Multitenant Search" (qdrant.tech/documentation/production-operations/multitenant-search)

ragqdrantproductionarchitectureembeddings

Auf Deutsch lesen →

Keep reading