Multi-tenant AI platforms: isolation, limits and per-tenant customisation

Where leaks happen

Every shared component is a possible leak:

• A shared vector index without a tenant filter returns another company's documents.
• A shared semantic cache returns tenant A's answer, with their internal pricing, to tenant B.
• Agent memory keyed only by user name mixes two users called Priya in different companies.
• Fine-tuning on pooled data teaches the model one customer's details.
• Traces and eval sets copied into a shared tool expose data to people who shouldn't see it.

Isolation by layer

Layer	Isolation
Vector store	A namespace or collection per tenant, or a mandatory tenant filter applied in code
Keyword index	Same as vector store
Cache	Tenant ID in every cache key
Memory	Keyed by tenant and user
Models	Shared base model, per-tenant prompts or adapters
Tools	Credentials per tenant, never shared
Limits	Rate limits and spend caps per tenant
Logs and evals	Tenant tags, access control on traces

Apply the tenant filter in your code, from the authenticated session, never from anything the model or the user's message says. See metadata filtering and access control.

Shared index or one per tenant

	Shared index with tenant filter	Index per tenant
Isolation	Depends on the filter always being applied	Physical separation
Cost	Lower	Higher with many small tenants
Operations	One index to manage	Many indexes to create, migrate, back up
Good for	Many small tenants	Large tenants, regulated data, contractual requirements

Many platforms use both: a shared index for small tenants, dedicated ones for large or regulated customers.

Per-tenant customisation

• Prompts: a base prompt plus tenant-specific instructions (tone, products, policies).
• Tools: a different set of tools and integrations per tenant.
• Models: a shared base model with per-tenant LoRA adapters, served together. See serving adapters.
• Evals: per-tenant eval sets for customers with specific requirements.

Noisy neighbours

One tenant running a huge batch job shouldn't slow everyone else. Use per-tenant rate limits, separate queues or priorities for batch work, and per-tenant budgets. See cost monitoring.

Cross-tenant leak test

DOCS = [
    {"tenant": "acme", "text": "Acme enterprise discount is 32 percent."},
    {"tenant": "globex", "text": "Globex enterprise discount is 18 percent."},
]


def retrieve(query: str, tenant: str, docs: list[dict]) -> list[str]:
    words = set(query.lower().split())
    hits = [d for d in docs if d["tenant"] == tenant and words & set(d["text"].lower().split())]
    return [d["text"] for d in hits]


def test_no_cross_tenant_leak() -> None:
    for tenant, other in [("acme", "globex"), ("globex", "acme")]:
        results = " ".join(retrieve("what is our enterprise discount", tenant, DOCS))
        assert other.lower() not in results.lower(), f"{other} data leaked to {tenant}"
    print("no cross-tenant leak")


test_no_cross_tenant_leak()

Run the same idea end to end in CI: create two test tenants with distinctive data, ask the same questions as each, and fail the build if anything from one appears for the other, in retrieval, cache, memory or the final answer.

Measuring it

• Cross-tenant leak tests passing on every release.
• Cost and usage per tenant per month.
• Latency per tenant, to spot noisy neighbours.

From the floor

The closest build to this in the cohort is the HR query bot, where every request carried an authenticated user, with OAuth2 and tokens that expire after 30 minutes, and retrieval was limited to what that user could see, which is the same rule a multi-tenant platform applies one level higher, per company.

Related

• Metadata filtering and access control
• Serving adapters
• Design multi-tenant RAG
• Cost monitoring

Sources

• Multitenancy, Qdrant documentation
• Namespaces, Pinecone documentation
• OWASP Top 10 for LLM Applications: sensitive information disclosure