Unifying Workers AI and AI Gateway: Architecture, Routing, and Integration

Unifying Workers AI and AI Gateway Architecture

Historically, Cloudflare offered two separate entrypoints for building AI applications: Workers AI, an inference-as-a-service platform hosting models on Cloudflare’s managed GPU infrastructure, and AI Gateway, a proxy designed to route requests to external model providers while delivering observability, access controls, logging, and security. While these products were built with different underlying architectures, developer usage converged on a shared goal: connecting to models through a flexible control plane.

To address this convergence, Cloudflare is unifying Workers AI and AI Gateway into a single control plane. This unification allows developers to manage both internal Workers AI inference and third-party provider calls (such as OpenAI and Anthropic) under one management interface. By integrating these systems, developers obtain cross-provider logging, unified pre-paid billing, token usage metrics, automatic caching, and dynamic routing capabilities without maintaining separate API pathways.

Unified Entrypoints: Workers Binding and REST API

The unification merges access patterns into shared entrypoints across both the Cloudflare Workers binding interface and the REST API. Developers no longer need to choose between separate SDKs or client constructs.

The Unified Workers Binding

In Cloudflare Workers, inference calls are executed through the single env.AI binding. Rather than instantiating a distinct gateway client, developers pass gateway configuration options directly into the env.AI.run() execution options.

Standard execution direct to Workers AI previously looked like this:

const response = await env.AI.run('@cf/zai-org/glm-5.2', {
  messages: [{ role: 'user', content: 'Hello!' }]
});

With the unified binding, developers pass a third parameter specifying the gateway configuration:

export default {
  async fetch(request, env) {
    const response = await env.AI.run(
      '@cf/zai-org/glm-5.2',
      {
        messages: [
          { role: 'user', content: 'What is the capital of France?' },
        ]
      },
      {
        gateway: {
          id: 'default', // Route through the default gateway
        },
      }
    );
    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

The Unified REST API

For external HTTP execution outside of Cloudflare Workers, a single /ai/ endpoint handles request execution and automatically routes traffic through AI Gateway. Specifying the target gateway is managed via the HTTP header cf-aig-gateway-id.

curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/zai-org/glm-5.2" 
  -H "Authorization: Bearer {api_token}" 
  -H "Content-Type: application/json" 
  -H "cf-aig-gateway-id: default" 
  -d '{
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

Zero-Setup Observability via Default Gateways

A central feature of the unified control plane is the default gateway. Previously, telemetry and request logging required manually provisioning an AI Gateway instance inside the Cloudflare dashboard before dispatching requests.

With default gateway support, explicitly creating a gateway in advance is optional. Passing default as the gateway ID in the Worker binding or HTTP header automatically instantiates the default gateway on the first authenticated request. This provides instantaneous access to observability features:

• Full Payload Logging: Every request and response payload is logged for auditing and prompt debugging.
• Per-Model Token Metrics: Input and output token counts are tracked on a per-model basis.
• Cost Attribution: Spend is tracked automatically without manual dashboard setup.
• Performance Telemetry: Latency breakdowns and error rate visualizations are surfaced directly in the AI Gateway dashboard.

When an application’s requirements expand—such as needing application-specific cache rules or segregated project metrics—developers can create custom named gateways in the dashboard and update the id parameter in their code base without altering request payloads.

Unified Billing and Elevated Rate Limits

Previously, billing systems for AI Gateway and Workers AI operated independently. AI Gateway credits could only be applied against calls to third-party model providers like Anthropic or OpenAI, while Workers AI usage was billed separately.

The unified control plane introduces shared pre-paid wallet billing across all providers. Developers can fund a single credit balance and draw from it across any supported endpoint, whether calling Workers AI’s managed GPUs or external provider APIs.

To incentivize adoption of this unified pathway, Cloudflare offers elevated rate limits for Workers AI models when calls are routed through AI Gateway unified billing. Developers can consult Cloudflare’s technical documentation to verify specific limit thresholds or request customized rate increases.

Model-First Routing and Cross-Provider Resiliency

Traditional multi-provider architectures require provider-first routing, where the application code explicitly dictates the host provider (e.g., dispatching directly to OpenAI or Moonshot). If the host provider experiences downtime or hits rate limits, application-level fallback logic or manual retry mechanisms are required.

The unified control plane is transitioning toward model-first routing. Under this paradigm, application code specifies the requested model weights rather than the hosting infrastructure.

curl -X POST "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions" 
  -H "Authorization: Bearer {api_token}" 
  -H "cf-aig-gateway-id: my-gateway" 
  -H "Content-Type: application/json" 
  -d '{
    "model": "kimi-k2.7-code",
    "messages": [{"role": "user", "content": "Review this function"}]
  }'

When a request is submitted (such as requesting kimi-k2.7-code):

• The gateway evaluates available execution targets across Workers AI infrastructure, primary developer APIs (like Moonshot), or alternative host providers running the same model weights.
• If Workers AI has available GPU capacity, the request is served on Cloudflare’s managed infrastructure.
• If Workers AI is at capacity or experiencing degradation, the gateway transparently load balances traffic to an alternate vetted provider serving identical model weights.
• All selected providers conform to compliance rules, including Zero Data Retention (ZDR) guarantees where required.

Smart Routing with Automated Prompt Classification

Beyond explicit model request routing, the control plane is introducing smart routing to automate model selection based on request intent.

Instead of hardcoding a specific model identifier in application code, requests can defer execution choices to the gateway’s routing engine:

1. Prompt Classification: A classifier running on Workers AI inspects incoming prompts to determine task type (e.g., general Q&A, code review, research, or summarization), evaluating task complexity and necessary context depth.
2. Heuristic Scoring: A heuristic engine maps the classified prompt characteristics to a curated pool of underlying models.
3. Dynamic Execution: The engine selects the optimal model balancing latency, cost, and output quality without requiring custom client routing logic.

Smart routing offers zero-configuration optimization for generalized traffic, while allowing teams to retain strict model selection by specifying explicit model parameters when necessary.

Key Practical Limitations and Operational Details

While unified bindings, default gateways, and shared wallet billing are available today, developers should note current deployment statuses and limitations outlined in the platform roadmap:

• Model-First Routing Status: Model-first cross-provider routing is scheduled for pilot deployment in upcoming releases for AI Gateway and Workers AI users.
• Smart Routing Status: Automated prompt classification and heuristic scoring are currently undergoing internal testing ahead of public release.
• Rate Limits: Higher rate limits are linked specifically to unified billing via AI Gateway; non-gateway pathways remain bound to baseline limits.
• Custom Gateways: While the default gateway auto-provisions, customized caching policies, isolated traffic segregation, and per-project routing rules require configuring named gateways within the dashboard.

Primary reference: Review the original announcement for exact release details. This article is an independent explanation and does not reproduce the source text.