Your AI users shouldn't need API keys

How we put identity-aware access in front of a model gateway so developers sign in with their directory account instead of holding API keys.

We did not want our internal users establishing individual relationships with external model providers. Personal accounts and provider-issued credentials would allow AI traffic to bypass our organization's access controls, approved-model policies, and compliance requirements.

Instead, we introduced an open source model gateway as the shared entry point for AI traffic. It gave us centralized model access, routing, and provider credential management — but created another problem: authenticating our users.

The gateways we evaluated generally offered directory integration and single sign-on only in their enterprise editions — with pricing beyond what our budget could justify for this specific requirement. We needed something narrower: authenticate users with our existing directory and authorize models based on their groups or roles.

GateBroker closes that gap. It adds identity-aware access control in front of an existing model gateway while preserving a simple workflow for developers:

gabro login
gabro run claude

Users sign in with their organizational identity and receive short-lived access to approved models. They do not need personal provider accounts or static gateway keys, while AI traffic remains within the organization's governance boundary.

How it works

GateBroker sits between AI clients and an existing model gateway.

For each request, it:

1. validates the identity provider's signed token;
2. maps the user's groups or roles to an access policy;
3. checks whether that policy permits the requested model; and
4. forwards the request using a server-side gateway credential.

AI client
    │ short-lived identity token
    ▼
GateBroker
    │ validate identity and model entitlement
    │ inject gateway credential
    ▼
Model gateway
    ▼
Model provider

The model gateway continues to handle provider integration, routing, and provider credentials. GateBroker adds identity-aware authorization without exposing gateway credentials to users.

Access follows organizational identity

A policy associates directory groups with permitted models and a reference to the appropriate gateway credential:

{
  "policies": [
    {
      "id": "engineering",
      "group_ids": ["8f2c1e4a-..."],
      "allowed_models": ["gpt-4o-mini", "gpt-4o"],
      "key_ref": "ENGINEERING_GATEWAY_KEY",
      "priority": 10
    }
  ]
}

The policy contains a credential name, not its value. The deployment resolves key_ref from an environment variable, mounted secret, or secret-manager integration. This keeps authorization rules reviewable in Git while credentials remain in the deployment environment.

Access changes through normal directory administration. Adding a developer to a group grants the corresponding model entitlement. Removing a user — or disabling their organizational account — removes access without distributing or rotating a user-specific gateway key.

Policy selection fails closed. A request is denied when:

• its token is invalid or expired;
• no policy matches the user;
• equally ranked policies make the result ambiguous; or
• the selected policy does not permit the requested model.

Designed for existing AI tools

Server-side authorization is useful only if users can use it with their preferred clients.

The gabro CLI uses device-code authentication, stores renewal state in the operating system's credential store, and supplies a short-lived token only to the child process it launches. It also configures the endpoint variables expected by supported AI tools.

Users do not need to understand the gateway's authentication model or maintain secrets in shell profiles and configuration files. They sign in, start their tool, and receive access to the models allowed by their organizational role.

Launcher profiles can provide similarly short commands for other agents and clients. Because a profile determines which executable receives the token, profiles are authenticated using a key in the operating system's credential store.

Distribution-specific identity-provider and gateway settings are compiled into the CLI. Local environment variables therefore cannot redirect a newly issued token to an untrusted endpoint.

Request enforcement and attribution

GateBroker does not trust identity or routing information supplied by the client.

It removes client-provided authorization, API-key, and routing headers before forwarding a request. It also replaces fields such as OpenAI's user or Anthropic's metadata.user_id with identity derived from the validated token.

This gives the gateway reliable attribution: it records who made a request based on verified identity rather than a string supplied by the client.

Each request produces an audit event containing operational metadata such as:

• the selected policy and requested model;
• the route and response status; and
• the request duration.

The audit log excludes tokens, prompts, request bodies, subject identifiers, and IP addresses. Request and response sizes, JSON nesting, and streaming traffic are also bounded to protect the broker itself.

Operational boundaries

GateBroker is an authorization layer, not a complete AI governance platform. It does not provide spend analytics, automated directory provisioning, compliance reporting, or enterprise support.

It also cannot prevent users from bypassing it when the underlying model gateway remains directly reachable. The deployment must restrict gateway traffic so that requests can arrive only through GateBroker.

Other current constraints include:

• policies are loaded at startup and require a restart to change;
• the bundled rate limiter is local to each replica; and
• identity-provider group-overage responses are denied rather than resolved through an additional directory lookup.

These limits should be considered before using the component in production.

When it fits

GateBroker is intended for teams that already operate a model gateway and an identity provider but do not want users managing static gateway credentials.

Its main value is straightforward:

• users sign in with a familiar organizational identity;
• AI tools receive short-lived access without persistent local secrets;
• model entitlements follow directory groups;
• platform teams can review authorization policy in Git; and
• gateway usage can be attributed to verified identities.

The gateway still uses API keys where required. The difference is that AI users no longer need to possess or manage them.

GateBroker is available under the Apache 2.0 license. The repository includes a local demonstration of the complete identity-provider, broker, gateway, and model request path: github.com/kayahk/gatebroker.