Gateway Overview
What the AI Gateway is, the four ways to deploy one, the listeners it exposes, and how it reports itself back to DevZero.
Gateway Overview
The AI Gateway is a lightweight HTTP proxy that sits between your applications and the LLM provider APIs. It requires no change to your application code beyond pointing at a new base URL.
In the dashboard's own words: route requests through DevZero so every LLM call is attributed, traced, cached, and billable to the right person or service.
What one gateway gives you
| Capability | Detail |
|---|---|
| Attribution | Every call tagged with the person, team, project, environment and service responsible for it. |
| Governance | Budgets, rate and token ceilings, and model and provider access, enforced before a request reaches a model. |
| Routing | Which model and which provider serves a request, configured per team and steerable per request. |
| Caching | An exact-match cache, and a semantic cache where content may leave the deployment. |
| Optimization | Prompt transformation, measured against a holdout control arm rather than estimated. |
| Telemetry | Per-request token counts, cost, latency, cache outcome and failure classification, streamed asynchronously to DevZero. |
Two kinds of traffic on one gateway
A single gateway deployment carries both, and the split is decided by the credential a request presents -- never by a broader property of the request.
Keyed product traffic presents a DevZero issued key (sk-dz-...) or a
DevZero access token with the inference scope. It reaches the
product surfaces, is governed, routed, optimized and
metered, and carries DevZero's own response headers.
Passthrough traffic presents your own credential -- a provider key, a Claude seat token, a ChatGPT JWT. It is forwarded byte for byte in both directions, on your credential, with no DevZero header added. This is where coding tools land, and your provider subscription keeps paying.
The split is enforced at the route predicate, which is why both can share a
listener, a method and a path. A Claude Code seat on POST /v1/messages keeps
the passthrough with byte-identical bytes even when a connector is registered
on the same listener for keyed traffic.
The four deployment methods
A gateway reports how it was deployed, and that fact decides residency.
| Method | Who runs it | Notes |
|---|---|---|
| Helm / Kubernetes | You | The production default. Prompts and retrievable originals stay in your own network. |
| DevZero-hosted | DevZero | The fastest way to evaluate. Point any client at a provisioned URL. |
| Docker | You | Run anywhere with docker run. |
| Docker Compose | You | Declarative YAML for local machines, VMs or simple server deploys. |
Hosted keeps content uplink on. Every self-hosted method is content-free by default -- see content uplink.
A hosted install is not a Helm release. DevZero applies it directly through the Kubernetes API, one namespace per team, so nothing decided in the chart reaches it. If you need to control the pod topology, self-host.
Listeners and ports
One binary exposes several listeners. The OpenAI-compatible surface and the native surfaces are separate listeners because they publish the same paths to different upstreams.
| Port | Listener | Purpose |
|---|---|---|
8080 | OpenAI-compatible surface and the main proxy | Chat Completions, Responses, and the management API under /api/v1/. |
8081 | Native Anthropic surface | POST /v1/messages, the shape Anthropic SDKs and Claude Code speak. |
8082 | Native Gemini surface | Gemini-shaped passthrough. |
8083 | Native OpenAI / Codex surface | Carries ChatGPT-JWT Codex traffic to OpenAI's own backend. |
8090 | Probe and metrics listener | /health, /ready and the Prometheus exporter at /metrics. Deliberately not published as a service port. |
The native OpenAI listener on 8083 declares no keyed product surface.
It exists to carry subscription Codex traffic, so a key-authenticated request
aimed there takes the passthrough and is never governed. Send keyed OpenAI
traffic to the OpenAI-compatible listener on 8080.
Two optional listeners exist and are off by default: a Cursor CONNECT forward proxy and a Cursor metrics tap. Cursor cannot be repointed by base URL, so the tap intercepts Cursor's own hosts and requires the client to trust its certificate authority.
How a gateway reports itself
Each gateway has an installation -- the provisioning record DevZero files its health, its dashboard-managed configuration, its operator state, its version pin and every telemetry row under.
Configuration: where a setting comes from
A gateway setting is resolved in one order: an operator pin (an environment variable you set on the deployment), then the remotely desired value from the dashboard, then the built-in default.
- Every dashboard-managed field is optional. Absent means unmanaged, and the gateway's own environment base or built-in default stays in effect.
- A pinned setting is shown read-only in the dashboard, with the reason.
- The gateway reports back what it is actually running, alongside which fields are pinned and whether a restart is pending. A gap between what an admin asked for and what the gateway reports is drift, and it is visible.
Two things are deliberately reported and never set from the dashboard. The gateway's static provider table, because one entry supplies both an upstream's URL and the name of the environment variable holding that provider's credential -- a remotely writable URL would let the control plane name a destination and have the gateway send your own key to it. And content uplink, which is environment-only so a content-free deployment stays content-free whatever the control plane asks for.
Where to go next
- Quickstart -- the four-step install flow.
- Self-hosting -- chart values, ports, ingress and the state store.
- Connect coding tools -- Claude Code, Codex, Cursor, Continue.dev.
- Keys and credentials -- issued keys, custody, BYOK.
- Product surfaces -- the routes an issued key reaches.