No Gateway Yet? How to Give Your AI Agents One Door

The evidence check runs against the traces you already have. A lot of teams get stuck one step earlier: agents run on laptops, API keys are pasted into a dozen configs, and there is no single place their actions pass through. So there is nothing to check. This guide is how to fix that, with free open-source parts, in days.

What an AI gateway is

One door with two sides. On one side, your agents' tool calls: every read and write to Slack, GitHub, your database, your CRM goes through one box (an MCP gateway). On the other side, their model calls: the conversations with Claude, GPT, or whatever your team runs. When both streams pass through one door, both halves of the record of what your agents actually do, intent and actions, pass through one place for the first time. Before that, they don't meet anywhere. Keeping and joining them is a separate job, and the honest part below is about exactly that.

You don't need both sides on day one. There is a right order.

Step 1: the tool side first. Enforcement is key custody, not policy

The common failure mode is starting with a mandate: “all AI traffic must go through the proxy.” Nobody complies, and you end up policing configs and fighting a privacy argument about personal ChatGPT use.

The tool side needs none of that. Move the credentials, the Slack token, the GitHub app, the database password, out of people's dotfiles and into the gateway's vault. Agents get a URL instead of keys. Now internal tools work only through the door, because the keys live there. No traffic interception, no honor system, nothing to enforce. An agent that doesn't go through the door doesn't reach the data.

And it stays selective by construction: the gateway serves the tools you put behind it. Personal AI use on personal accounts is untouched.

Access is per person, too: each person connects with their own gateway key. Who can reach what stops being a guess and becomes a config line: deny a tool to someone, and it doesn't get refused, it simply disappears from their catalog. The agent never sees it.

Step 2: the model side, at the strictness you actually need

Model traffic joins second, and you can pick how hard to turn the screw. Three modes, in increasing order:

1. Personal stays personal. Do nothing. Only tool calls route through the door.
2. Per work repo. Both major coding harnesses read their model endpoint from repo-level config (Claude Code: .mcp.json and ANTHROPIC_BASE_URL; Codex: .codex/config.toml). One committed file routes every work session through the gateway, onboards the whole team including future hires, and separates work from personal by folder, with no policy document at all. This is the underrated middle and the right default for most teams.
3. Machine-wide via MDM. Managed settings (Claude Code's managed-settings.json, Codex's requirements.toml) pushed to company devices, not overridable. Only needed for the strict regime, and if you are heading toward SOC 2 you are acquiring MDM anyway.

One thing that lowers the bar for both routed modes: this works with plain claude.ai subscriptions. The gateway routes the traffic and logs the metadata (model, token counts) without an Anthropic API contract, and the subscription keeps paying for the calls.

Picking the door: buy durability, not features

The gateway itself is commodity open source. What kills a choice within a year is not missing features, it's two other things: the owner (a startup gets acquired or narrows its license, and your door changes terms under you) and protocol churn (the MCP spec has had three revisions in eighteen months; a door that trails the spec breaks your clients). So choose on those axes:

• foundation-owned, not single-vendor;
• a permissive license with no seat ceiling;
• tracks the current MCP spec closely;
• handles both planes, tool calls and model calls, in one process.

Our default pick is agentgateway: Linux Foundation-owned, Apache 2.0, speaks MCP, A2A, and LLM traffic in one process, and ships both as a single binary for a small VM and as a Kubernetes deployment. If you already run Envoy or Istio, Envoy AI Gateway is the natural pick instead. And one property matters more than the choice itself: every serious door speaks the same protocol, and your clients' configs hold only a URL. If you ever swap doors, it is days of migration, not a rebuild.

Where the keys live: the fast build and the own build

The door is the same either way; what differs is who holds the credentials for your SaaS tools.

Fast build. The gateway as a single binary, plus a cloud credential provider such as Composio for the SaaS connections. Live in days. The trade: OAuth tokens for your tools sit with a third party.

Own build. The gateway on your own VM or Kubernetes, plus self-hosted Nango for SaaS credentials (hundreds of APIs, tokens stored in your own Postgres), plus Vault for the remaining secrets. Everything inside your perimeter.

Start fast, upgrade later: moving from the fast build to the own build swaps the credential layer and never touches the door, so your agents' configs don't change.

The honest part: a gateway alone does not give you the record

This is the claim most teams get wrong in the optimistic direction. “We installed a gateway, so we have traces” is false on both sides of the door. Model-side gateways see the prompt but are built not to keep it: Anthropic's own gateway, for instance, stores usage metrics, not prompt or completion content. Tool-side gateways log the call but never had the prompt: the MCP protocol carries no field for the user's original request. Neither side, on its own, can tell you why an agent did what it did.

The door is still the right first move, because it is the only place both streams can be joined: intent from the model side, actions from the tool side, assembled into one record by construction rather than by after-the-fact correlation. Once the record starts accumulating, run the evidence check against it. Six questions, no install, nothing sent anywhere. It will tell you exactly which questions your record can answer and where the gaps are, and closing those gaps is where the real work is.

Do it yourself, or hand it to us

Everything above is free, open-source, and documented: the door, the credential layer, the harness configs, the evidence check. A capable platform engineer can stand up the fast build in days.

If you'd rather not spend those days, or you want the whole thing done end to end, gateway installed, keys moved, harnesses connected, and the record checked, that is work we do. Write to [email protected] or book a call, and tell us how your agents run today.