Govern LLM traffic with guardrails: budgets, rate ceilings and model access
Set a monthly team budget, a daily ceiling for a CI key and a model allow-list in Nautili, test them in shadow mode, then enforce without surprise denials.
16 min readLast reviewed October 19, 2026Verified against gateway chart 0.10.0
At the end of this guide your team has one guardrail that caps the month at $2,000 and keeps every member off the frontier models, plus a stricter one that stops a CI service key at $50 a day and 60 requests a minute. You will have watched both in shadow mode on real traffic, so switching the gateway to enforce denies only what you expected it to deny.
Before you begin
- A Nautili team with at least one gateway installed, hosted or self-hosted. See Install if you have none.
- A runtime key for your own testing, from API keys under Access, exported as
$NAUTILI_KEY. - One ownerless service key for CI. A key held by a person cannot be bound directly; it follows that person's guardrail. Issue it on API keys under Advanced, with Owner left at
Service key (no owner). A key issued there arrives already bound to a guardrail of its own, named after the key, so until you repoint it in Step 2 the team default does not govern it. - Managed or BYOK custody for the team. A seat guardrail cannot carry a budget, a rate ceiling or a key binding, so the Limits step does not appear for it.
- Admin rights on the team. Creating a guardrail and marking it the team default changes what every unbound principal may do.
The pick follows a short ladder. The principal's own binding wins. Otherwise the team default applies. Otherwise nothing applies, and nothing means allow: an unconfigured team is not a governed team. Two details matter for the rest of this guide. First, a disabled or expired guardrail denies with policy_disabled or policy_expired; it does not fall through to the default. Second, a key held by a person resolves to its owner's guardrail and shares the owner's spend counter, while a service key gets its own binding and its own counter.
Step 1: Create the team default
Open Guardrails and start the wizard. Fill the five steps with these values.
The Access step offers three modes: allow every model, allow only what I select, or allow everything except what I block. The four raw lists the gate reads (Allowed models, Ignored models, Allowed providers, Ignored providers) sit under Advanced, edit all four rule lists directly on the same step; a blocklist writes Ignored models. The dashboard writes limit_amount 2000 USD, limit_reset monthly, counter_scope team and the two ignored_models patterns. Marking it the team default repoints the single per-team default binding, so from now on every person, and every key whose own binding has been cleared, is governed by this object.

Pick the reset cadence with care. A daily ceiling read against a month of spend looks like a team far over budget when it is not, which is why every surface that shows a budget shows its cadence next to it. Only a monthly ceiling compares directly with a monthly spend figure on Spend. Windows are calendar UTC: daily resets at 00:00 UTC, weekly on Monday 00:00 UTC, monthly on the first. A budget with no reset is a lifetime budget anchored at creation.
Step 2: Add a stricter binding for the CI key
Run the wizard again for the CI service key. This guardrail is not the team default. It binds to one key, and because a key has at most one binding it replaces the team default for that key entirely. The $2,000 pool does not count the CI key's traffic, and the CI key does not inherit the ignored-models list.
Assignments are declarative. Naming a key repoints it from wherever it was bound, and omitting a key unbinds it only if it currently points at this guardrail. The Guardrails table now shows engineering-monthly with the Default badge and ci-release with the key name under Assignments; the key's auto-created guardrail is left with a dash, and a guardrail with a dash under Assignments governs nothing. The Budget column reads the limit and its cadence, $50.00 / daily with 60 rpm beneath it.
Step 3: Confirm which mode the gateway is in
The mode is a gateway setting, not a guardrail setting and not a team setting. It takes one of off, shadow or enforce, carried by the GOVERNANCE_MODE environment variable on the gateway deployment. A hosted gateway runs whatever mode DevZero's control plane is configured with (llm_gateway_governance_mode), which is enforce when unset, as it is in production; DevZero sets it, not the team. The Helm chart renders enforce whenever governance.enabled is true, which is its default, and it cannot render anything else: the chart warns against pinning through config.extraEnv a variable it already renders. Only the bare binary with the variable unset starts in off, where the gate does not run at all. A Docker Compose install reaches shadow by editing GOVERNANCE_MODE in the generated service. If you cannot get a shadow window, treat the gateway as enforcing and introduce rules on a non-default guardrail bound to a test service key before touching the team default.
The command prints the mode the pod booted with. In shadow every rule is evaluated and recorded and the gateway denies nothing: the request is served as usual and the verdict it would have received is written next to it. On a hosted gateway ask DevZero which mode you are in rather than guessing.
Step 4: Send governed traffic
Send a request as the CI key against a model it may not use. In shadow mode the request goes through and the would-deny is recorded; in enforce mode the gateway denies it with a 403.
The response carries X-Dz-Request-Id, which is the id to search for on Traces.
Verify
Open Traces and find the request by its id. Every keyed request writes a decision record whatever the outcome, and the governance part of that record is the triple below. Trimmed to the fields that matter:
decision is allow because the gateway forwarded the request. shadow_would_deny is true and deny_reason names the rule that would have fired, with policy_id pointing at the guardrail that decided. That triple is the whole point of the shadow window: it tells you what enforce mode would do, per request, before it does it.
Repeat for a request from a personal key against anthropic/claude-opus* and expect model_not_allowed under the engineering-monthly id. Then check Guardrails: the Budget column shows each limit with its cadence, and clicking it opens the Limits step, where the burn meter estimates how long the budget lasts at the token ceiling from catalog prices; it does not read actual spend. To read spend against the guardrail's own window, filtered the way enforcement filters, use the management API usage read (GET /api/v1/keys/<KEY_ID>/usage, covered in the Spend guide). Do not sum per-key spend from API keys against the limit; that column is a fixed 30-day read with no policy filter and overstates consumption.
When the shadow window shows only would-denies you can explain, move the gateway to enforce. On a self-hosted gateway set GOVERNANCE_MODE=enforce on the deployment and restart; on a hosted gateway ask DevZero. Re-run Step 4 and the same request now returns a 403 instead of a completion.
Options
Budget
A budget is a USD ceiling per cadence. Unset or zero means no budget, not "no spending allowed"; the wizard's budget field snaps what you type to the nearest 10 USD, so pick a multiple of ten (the API accepts any amount). principal scope gives every bound person and key its own bucket, which is a shared rule, not a shared wallet. team scope pools one bucket across everything the guardrail governs, and the rate ceilings pool into the same bucket. Owned keys always share their owner's bucket; service keys get their own. Spend is counted in integer micro-USD, and the gateway folds the control plane's reconciled figure into its live counter by taking the maximum, so a lost increment cannot reopen an exhausted budget.
Request: the Step 4 call with anthropic/claude-haiku-4.5, once the CI key's day is spent.
Pass: 200 with a completion.
Deny, in enforce mode:
There is no Retry-After, because retrying cannot help before the window rolls over. x-should-retry: false exists because the OpenAI SDKs retry every 429 on status alone; without the header a budget deny would cost three requests. On the Anthropic surface (/v1/messages) the same deny is a 400 with type invalid_request_error, which is that SDK's terminal class.
Rate ceilings
Both are fixed one-minute windows, and 0 means unlimited. Requests per minute is checked before the forward: the request that would be the sixty-first is refused. Tokens per minute is checked after the fact: the request that crosses the ceiling is served, and the next one is denied. Avoid calling an upstream provider's own throttling a rate limit in the same breath; a provider throttle is the upstream refusing you and is reported differently.
Request: the Step 4 call, sent 61 times inside one minute.
Pass: 200 for the first sixty.
Deny, in enforce mode:
Retry-After is the number of seconds to the next minute boundary. A tokens-per-minute deny has the same shape with code tpm_exceeded and a message starting token rate for model.
Model and provider access
Four lists, all wildcard patterns: * matches any run of characters and everything else is byte-exact. An empty allow list means unrestricted, not "permit nothing". A match on an ignore list denies even when the allow list would permit it, so ignore beats allow. A non-empty provider allow list fails closed on a provider the gate cannot recognise, because a request whose provider is unknown cannot be proved to satisfy the restriction. On the keyed connector route the control plane expands provider wildcards into literal slugs from the model catalog before the gateway enforces them.
Request: the Step 4 call with openai/gpt-5.6-luna.
Pass: a request for anthropic/claude-haiku-4.5 returns 200.
Deny, in enforce mode:
A provider rule denies with code provider_not_allowed; from the gate it is a 403 like the model rule, and on the keyed connector route, where the control plane's expanded provider list leaves nothing servable, it is a 400 invalid_request_error with the same code. The message names the model and the policy id so an operator reading a client log can trace it back to the guardrail.
Deny responses at a glance
| Condition | Status (OpenAI surface) | code | Headers | Retry? |
|---|---|---|---|---|
| Budget exhausted | Status (OpenAI surface)429 (400 on /v1/messages) | codeinsufficient_quota | Headersx-should-retry: false, no Retry-After | Retry?No, wait for the window |
| Requests per minute | Status (OpenAI surface)429 | coderate_limited | HeadersRetry-After to the minute boundary | Retry?Yes, after the header |
| Tokens per minute | Status (OpenAI surface)429 | codetpm_exceeded | HeadersRetry-After to the minute boundary | Retry?Yes, after the header |
| Model or provider rule | Status (OpenAI surface)403 | codemodel_not_allowed, provider_not_allowed | Headersnone | Retry?No |
| Guardrail disabled or expired | Status (OpenAI surface)403 | codepolicy_disabled, policy_expired | Headersnone | Retry?No |
Troubleshooting
| Symptom or error | Cause and fix |
|---|---|
A request you expected to deny returned 200, or a 502 with code upstream_auth | The gateway is in shadow or off. A 502 upstream_auth naming the governance decision is a shadow would-deny forwarded without the team credential. Check GOVERNANCE_MODE on the deployment (Step 3). Look for shadow_would_deny: true on the telemetry row before changing the mode. |
model_not_allowed on a model that is in Allowed models | A byte-exact entry does not match the id the gateway sends. Use the author/slug form, for example anthropic/claude-haiku-4.5, or a wildcard. The false deny is cached for a short window after you fix it. |
policy_disabled or policy_expired | The bound guardrail is switched off or past its Expires date. It denies rather than falling through to the team default. Re-enable it or clear the expiry under Lifecycle. |
| The CI key is still counted against the team pool | The key has an owner, so it follows the owner's guardrail and shares the owner's counter. Issue an ownerless service key and assign it on the ci-release guardrail. |
| Assigning a key fails on the wizard | Only ownerless service keys bind directly, and keys cannot bind a seat guardrail. Check the key's owner on API keys and the guardrail's custody. |
insufficient_quota on the first day of the month | The budget window is not monthly. Open the guardrail and read the cadence next to the limit; a daily ceiling compared with a monthly spend figure is the usual cause. |
provider_not_allowed with a provider you did list | The gate could not recognise the provider, and a non-empty allow list fails closed on an unknown one. Match the slug as the catalog spells it, or use a wildcard. |
| Archive refused with assignments still pointing at the guardrail | Unassign first. Clear the team-default flag and the key list on Guardrails, then archive. |
key_policy_not_metered on a keyed request | The key's guardrail is seat custody or absent. Bind a managed or BYOK guardrail to the key (service key) or to its owner (owned key), or set one as the team default. |
Next steps
- Content rules: add PII detectors and prompt-injection scanning to the same guardrail, with their own coverage caveats.
- Keys and credentials: owned versus service keys, custody, and why minting a key is never a way around a person's budget.
- Governance rules: the full deny-reason list, unattributed traffic and how the spend counter stays honest across an outage.
- Spend attribution: read spend by user, team, project and label once the budget is live.
- Self-hosting: the chart values a governed gateway needs, including the Redis the counters ride on.
- Keep secrets and PII out of prompts: the content rules side of the same guardrail.
- Keys: scopes, rotation and custody: issuing the service key this guide binds, and what custody means for it.
- Reading the Spend screen: the monthly figure the budget window is meant to be compared with.