Code lab · World 09 · Multi-tenancy and Security

Per-tenant quota limiter.

A token bucket per tenant sized by plan, costs per operation, a shared cluster bucket on top, and a 429 that says why and when to retry.

Medium~35 min3 example + 11 hidden testsPython · JS · TS+90 XPJump to the code ↓

Saving also stores the Python runtime (~13.5 MB, once for every lab) so this lab runs with no internet.

PractisesW09 · D23.2 Quota Tiers by Plan W09 · D24.1 Token Bucket Algorithm W09 · D29.1 Anti-Patterns

One global limiter can't tell tenants apart: a tenant sending 900 of the cluster's 1,000 requests a second leaves everyone else fighting over the last 100 (Day 2, Chapter 9, Mistake 2). And one limit for every endpoint treats an export like a health check (Mistake 5). This lab builds the gateway limiter the lesson asks for:

A token bucket per tenant, sized by the tenant's plan (section 3.2). A tenant with no plan set is on "free", never unlimited: the lesson's "fail open" warning.
A cost per operation (COSTS in the given code): a search spends 5 tokens, an export 25.
A shared cluster bucket on top, so the tenants together can't exceed what the database can take.
A useful rejection: which limit said no, and when to retry (Mistake 4: no silent throttling).

Implement TenantLimiter(plans, costs, global_rate, global_burst, clock):

Method	Does
`set_plan(tenant, plan)`	moves a tenant to a plan. An unknown plan name raises `ValueError`
`check(tenant, operation)`	returns a `Decision` (given) and spends tokens when allowed

The Decision fields:

Field	Value
`allowed`	`True` or `False`
`reason`	`None` when allowed; `"tenant"` when the tenant's bucket is short; `"global"` when the tenant has enough but the cluster bucket is short; `"plan"` when the cost is bigger than the plan's whole burst
`limit`	the plan's `burst` (feeds `X-RateLimit-Limit`)
`remaining`	the tenant's tokens after this decision, rounded down to an int (feeds `X-RateLimit-Remaining`)
`retry_after`	`0.0` when allowed; seconds until both buckets hold the cost; `None` for `"plan"` (waiting won't help: upgrade)

Rules the tests check:

Both buckets refill lazily, elapsed × rate capped at burst, the same rule as the Week 1 token bucket. A new tenant and the cluster both start full.
All or nothing: a request is allowed only if the tenant bucket and the cluster bucket hold its cost. A rejected request spends nothing from either.
Check the tenant first: if both are short, the reason is "tenant".
An operation that isn't in costs costs costs["default"].
set_plan keeps the tenant's current tokens (refilled up to now first), capped at the new plan's burst. An upgrade doesn't hand out a free burst; a downgrade takes the excess away.
Read time only through clock.now().

In JavaScript and TypeScript it's new TenantLimiter(plans, costs, globalRate, globalBurst, clock) with setPlan(tenant, plan) and check(tenant, operation); PLANS and COSTS are Maps, the Decision field is retryAfter, None is null, and an unknown plan throws a RangeError.

Two buckets, one decision

In production both buckets live in Redis and one Lua script checks and spends them together (section 4.1's distributed limiter). If you spent the tenant's tokens and then found the cluster empty, every rejection would still drain the tenant's quota.

Python 3.14 loads on first run (~6.5 MB once)14

Loading the editor…

⌘/Ctrl + Enternot run yet · 8 s limit

press Run tests

Write your code, then run 3 example tests and 11 hidden tests. Each run is limited to 8 seconds: an infinite loop gets stopped and the runtime restarts, your code stays.