Key Concepts
The vocabulary the product uses -- sources, surfaces, credentials, regimes, savings, guardrails, profiles and evidence.
Key Concepts
Nautir uses a precise vocabulary, and the dashboard, the API and this documentation all use it the same way. This page is the short version; each section links to where the concept is used.
Several of these terms exist specifically to keep a number honest. Where a figure is directional, estimated, or valid only at a particular scope, the product says so in place rather than rounding the caveat away.
Connectivity
| Term | Meaning |
|---|---|
| Source | Anything emitting LLM telemetry for your team. Every source is exactly one kind: a proxy gateway or a direct source. |
| Proxy gateway | A source that routes requests through an installed gateway process. It has an installation record, a provisioned URL and a connectivity status, and it reports liveness by pinging home. |
| Direct source | An SDK or client reporting telemetry directly, with no proxy process in the request path. No installation, no deployment method; its activity comes from request telemetry rather than heartbeats. |
| Installation | The provisioning record for a proxy gateway, with a lifecycle from pending, through provisioning, to ready or failed. Only proxy gateways have one. |
| Connected | DevZero is currently receiving a gateway's heartbeat. It says nothing about traffic -- a gateway can be connected and idle. |
| Receiving data | The source has had at least one request in a fixed recent window of 24 hours, independent of the page's time picker. Otherwise it shows No data. |
| Deployment method | How a gateway says it was put there: hosted, helm, docker or binary. Hosted means DevZero runs the process; every other method means you do. |
Surfaces and credentials
| Term | Meaning |
|---|---|
| Keyed surface | The explicit allowlist of routes that is the AI Gateway product boundary. The published contract mirrors it one-to-one, and nothing outside it is part of the product. |
| Passthrough surface | A gateway listener that forwards a request without reshaping it: outside the keyed allowlist, carried byte-identically in both directions on your own subscription or OAuth credential, with no DevZero header added. |
| Issued key | A DevZero-minted credential (sk-dz-...). One credential type covers inference and management; scopes are a property of the key, set when it is issued. |
| Carrier | The transport slot a gateway control rides in -- header or body. The header is canonical; the body is a compatibility alias. A request presenting both is an error, never a precedence question. |
| Custody | Whose credential pays for a keyed request. Three arms: byok, managed, or empty meaning auto (BYOK where the team holds a credential, managed otherwise). |
| Truthful spec | The property that everything strict in the published contract is something the gateway actually rejects. Body shapes owned by upstream vendors stay loose. When contract and enforcement disagree, the contract is what moves. |
See Product surfaces and Keys and credentials.
Billing and money
| Term | Meaning |
|---|---|
| Billing regime | The classification of a request by credential shape into metered or subscription. It decides which optimizations may apply and how value is denominated. |
| Metered | Per-token API-key traffic. Compression saves real dollars and the full evidence loop applies. |
| Subscription | Prepaid seat traffic on OAuth-shaped credentials. Value is denominated in rate-limit headroom and cache stability, never in measured dollars. |
| Rate-limit headroom | The subscription-regime value unit: tokens not spent, which extend how far a prepaid seat's limits stretch. |
| Contracted rate | Your own negotiated per-model price, supplied by you. The authoritative price input for every billable figure. |
| List-price figure | A dollar figure computed from the public price table rather than a contracted rate. Directional only, labelled as such wherever it appears, and never billable. |
| Billing-grade | The standard a measurement must meet to appear on an invoice: auditable, reconcilable against your own rates, defensible under adversarial review. Stronger than evidence-grade, which is the bar for gating a change internally. |
| Evaluation spend | Money spent to evaluate models rather than to serve your own traffic -- a trial's arms and its judge. Bounded by the per-trial spend cap approved at trial start, deliberately not by your guardrail's budget. |
See AI spend.
Savings and evidence
| Term | Meaning |
|---|---|
| Session | One conversation, identified by the session key the client supplied. The session is the unit of this domain: it spans many turns, and evidence is drawn on it. |
| Holdout | The fraction of traffic deliberately served uncompressed as a control arm, so savings and quality are measured against reality rather than estimated. It runs in both billing regimes. |
| Measured savings | Savings computed against the holdout control arm -- the defensible number. Denominated by regime: dollars on metered, tokens on subscription. |
| Estimated savings | Savings computed without a control arm. Directional, labelled as an estimate, and the only form a subscription dollar figure may take. |
| Effective tokens | A token count re-expressed in fresh-input-token units, each token weighted by the price ratio of the billing stream it came from. The honest denomination for savings, and always shown beside the raw count. |
| Share of effective context | Effective tokens saved over the same window's effective input volume. Genuinely sub-percent, and deliberately not a share of a bill. |
| Recurring saving | The part of effective savings that came from the cache-read stream, where the transform re-fires and removes the same tokens on every later turn of the same conversation. |
| Quality-parity judge | The scorer that compares answer quality between holdout arms and returns a parity verdict. A degraded verdict vetoes promotion. |
| Session-denominated | Counted in sessions rather than rows -- the only valid denomination for an effect that is a property of a conversation. |
| Honest blank | A metric shown as empty or zero with the reason it is empty and the stage that would fill it, never a fabricated number. |
The optimization journey
The five stages a gateway climbs as it earns the right to make measured claims. The dashboard renders it as a strip, and configuration guidance follows it.
| Stage | State |
|---|---|
| 1. Observe | Gateway connected, traffic flowing, nothing altered. |
| 2. Compress | Lossless compression is on. |
| 3. Measure | A holdout is running and the gateway can produce judge evidence, so savings claims become measured instead of estimated. |
| 4. Audition | Content-dropping transforms run in the background -- projected, never served -- accumulating the would-have-saved evidence. |
| 5. Promoted | Evidence has cleared and those transforms serve live traffic, with retrievable originals behind them, per workload type. |
Defaults stay off. The one-click recommended profile advances a gateway through the journey's enabling settings as a correct combination, applied explicitly through the audited configuration path. The profile is consent, not a default.
Configuration
| Term | Meaning |
|---|---|
| Knob | One remotely-manageable gateway setting. Resolution order: operator pin, then the remotely desired value, then the built-in default. Regime differences are built-in policy, never knobs. |
| Pinned | A setting locked by an operator environment variable. Remote management can never override it, and the dashboard shows it read-only with the reason. |
| Effective config | The resolved configuration a gateway is actually running -- the merge of its environment base and the remotely desired values. This is what the gateway reports back and what the dashboard displays. |
| Desired vs applied | Desired is what an admin asked for; applied is what the gateway last reported running. A gap between them is drift. |
| Content uplink | The residency control deciding whether customer content may leave the deployment at all. Environment-only, never a dashboard setting, and it defaults from the deployment method. |
| Prompt retention | Whether bodies that do reach DevZero are stored, and for how long. Team-level, off by default, window of 7, 30 or 90 days. |
| Principled no | An option unavailable by design rather than by risk appetite, because enabling it would violate a stated invariant. It gets no toggle at all -- as distinct from an option that carries real, explained risk and does. |
Governance and routing
| Term | Meaning |
|---|---|
| Guardrail | A named bundle of governance rules -- budget, rate ceilings, model and provider access -- plus content rules. Assignment rides the existing binding ladder. |
| Routing profile | A named bundle of routing behaviour scoped to a team. Profiles merge field by field down the ladder, so one rung can override a single setting and inherit the rest. |
| Routing override | A caller's per-request steer, dz_router, which is the most specific rung of the same ladder. Consent bounds it: it may steer inside a substitution mode you enabled but may not enable one. |
| Model-fit advisory | A benchmark-and-price-based suggestion that a cheaper model would suffice for an observed workload. Advisory only, never auto-applied, and never part of any score. It surfaces a candidate; it does not constitute a savings claim. |
See Routing and Guardrails.
Evaluation
| Term | Meaning |
|---|---|
| Trial | A long-lived, team-owned comparison of an incumbent model against a candidate slate over your own traffic, within a scope, with stop conditions and a hard spend cap. |
| Experiment | The same machinery run over a stored dataset instead of production traffic. Nobody ever ran those inputs, so both arms are dispatched and it costs twice per item. |
| Dataset | A team-owned corpus an experiment runs over: an input, an optional expected output, and filterable metadata per item. |
| Watch | A post-promotion sentinel with the roles flipped: the adopted model is the served side and the model it replaced is the sole candidate, sampled at a low rate to detect drift. It alerts only, and never writes routing. |
| Compare | An ad-hoc fan-out of one prompt to several models, returning per-model output, latency, cost and score. Ephemeral by construction -- nothing is persisted and no result can reach a verdict. |
| Promotion | Making a candidate the scope's model, and it is always an explicit human act. No trial mutates routing on its own. |
See Evals.
Efficiency
| Term | Meaning |
|---|---|
| Efficiency | The diagnostic lens on waste still on the table -- retries, redundant prompts, poor model fit, missed cache wins -- attributed to people, teams and services. It answers "where is spend inefficient and who needs help", where savings answers "what did the platform already save". |
| Efficiency score | The per-person waste diagnostic, rolled up per team, computed from exactly three directly-measured pillars: retry waste, cache economics and prompt repetition. Heuristic and quality-judged signals are never folded in. |
| Waste share | The fraction of a person's or team's tokens that were waste over a trailing window. Token-denominated, so it is valid in both billing regimes, and absolute rather than relative to peers. |
| Recoverable waste | Absolute wasted tokens for a person or team -- the prioritization metric. A low score on tiny volume is noise; recoverable waste is what orders the list. |
| Coverage statement | The honest treatment for a partially-measured metric: state since when a pillar has been measured and what fraction of traffic is attributed, rather than backfilling from a different methodology. |
| Ratings visibility boundary | The line between measured telemetry and judged content. Measured per-person usage and waste is operational data and openly visible; anything a judge produced about a person is a rating, self-only by default and enforced server-side. |
See Efficiency.