## Article: “Whatâs new with AI Gateway and Workers AI”
Cloudflare has been evolving its AI offerings to make deploying and managing large language models easier, more secure, and observable. Recently, AI Gateway and Workers AI were developed as distinct products. Over time, users have started converging towards a single interface for managing their AI requests. With AI Gateway, users can proxy requests to any model provider while benefiting from built-in observability, logging, access, and security. On the other hand, Workers AI hosts models on Cloudflare’s managed GPU infrastructure, exposing an API endpoint for inference-as-a-service.
The architecture of these products may look different, but they achieve the same goal: connecting users to models with a sophisticated control plane. Today, Cloudflare is announcing the convergence of these products into one unified path. This means users can now connect to any model provider (including Workers AI) while managing observability, billing, security, and logging from a single control plane. This is a significant step toward more extensive plans for model routing.
### Merging the Binding and API
Cloudflare has been hinting at the convergence of these products through entry points: the Workers binding and the REST API. There is now a single AI binding that can be used to call both AI Gateway and Workers AI, eliminating the need for separate bindings. A “default” gateway can be used for automatic observability and logging, but users also have the option to specify their own gateway to split applications into multiple projects.
The binding call now looks like this when calling Workers AI via AI Gateway:
“`javascript
export default {
async fetch(request, env) {
const response = await env.AI.run(
‘@cf/zai-org/glm-5.2’,
{
messages: [{ role: ‘user’, content: ‘What is the capital of France?’ }],
},
{
gateway: {
id: ‘default’, // Use ‘default’ for the built-in gateway
},
}
);
return new Response(JSON.stringify(response), {
headers: { ‘Content-Type’: ‘application/json’ },
});
},
};
“`
A unified REST API has also been announced—the `/ai/` endpoint—which allows similar calls to Workers AI via AI Gateway:
“`bash
curl
-H “Authorization: Bearer {api_token}”
-H “Content-Type: application/json”
-H “cf-aig-gateway-id: default”
-d ‘{
“messages”: [{“role”: “user”, “content”: “What is the capital of France?”}]
}’
“`
This unification allows users to choose either product without having to decide between them—everything now comes with built-in functionality.
### Automatic Observability and Control for All Workers AI Users
One of the most immediate benefits of this convergence is automatic observability. Users no longer need to explicitly create an AI Gateway to gain visibility into their inference traffic. Simply passing `default` as the gateway ID in binding or REST API calls will automatically create a gateway on the first authenticated request.
With this setup:
– Every request is logged with full request and response payloads.
– Token counts are tracked per model.
– Cost attribution is provided without any dashboard setup.
If users outgrow the default gateway, they can create a named gateway and point requests at it with a single parameter change.
Here’s how it looks in the binding. Previously, Workers AI was called directly:
“`javascript
const response = await env.AI.run(‘@cf/zai-org/glm-5.2’, {
messages: [{ role: ‘user’, content: ‘Hello!’ }],
});
“`
Now, adding a third argument routes requests through AI Gateway, providing full observability:
“`javascript
const response = await env.AI.run(
‘@cf/zai-org/glm-5.2’,
{ messages: [{ role: ‘user’, content: ‘Hello!’ }] },
{ gateway: { id: ‘default’ } } // Auto-creates the gateway on first use
);
“`
The Cloudflare AI Gateway dashboard will log every request, showing latency breakdowns, token usage, error rates, and the exact prompts and responses.
### New: Using AI Gateway Credits for Workers AI
As part of this update, AI Gateway credits can now be used for Workers AI. Previously, credits could only be applied to external model providers like OpenAI or Anthropic. Now, unified billing allows credits to be spent across OpenAI, Anthropic, Workers AI, or any other supported provider.
Additionally, elevated rate limits are available for Workers AI models when using AI Gateway unified billing. Users are encouraged to refer to the developer documentation for current rate limit information and how to request higher limits.
### Coming Soon: Model-First Routing
With all inference traffic flowing through a single control plane, smarter routing decisions are now possible. The next evolution is model-first routing, where users specify the model they need, and the control plane handles provider selection, failover, and load balancing.
Currently, users must know which provider hosts a model. If that provider is down or rate-limiting, the application may break. Model-first routing flips this approach—users specify the model (e.g., “Kimi K2.7 Code”), and the gateway transparently selects the best provider. If Workers AI is at capacity, traffic shifts to another provider without complex fallback logic.
This also improves resiliency. The gateway treats model availability as a routing problem, ensuring consistent application performance.
### Next: Smart Routing
The next step is intelligent routing that understands the task at hand without requiring user configuration. A classifier running on Workers AI will analyze prompts to determine:
– The type of task (coding, research, summarization, Q&A)
– Complexity
– Context requirements
A heuristic scorer will then map these attributes to the best model from a curated pool. Users who want control can still specify exact models, while others can benefit from zero-config routing for better economics and performance. This feature is currently being piloted internally and will be tested in the coming weeks.
### Get Started Today
– If you’re already using Workers AI, start routing calls through a default gateway to immediately gain request logging, token tracking, and cost attribution.
– If you’re already using AI Gateway, adding Workers AI is as simple as calling a Workers AI model. Load your AI Gateway wallet to get unified billing across all providers, plus elevated rate limits on Workers AI models.
Set up your first gateway, browse the Workers AI model catalog, and get started building today.
—
### FAQ
**Q1: What is AI Gateway?**
AI Gateway allows users to proxy requests to any model provider while offering built-in observability, logging, access, and security. It centralizes management of AI requests into a single control plane.
**Q2: What is Workers AI?**
Workers AI hosts models on Cloudflare’s managed GPU infrastructure, providing an API endpoint for inference-as-a-service without managing underlying infrastructure.
**Q3: How are AI Gateway and Workers AI converging?**
They are merging into a unified path, allowing users to connect to any model provider (including Workers AI) while managing observability, billing, security, and logging from one control plane.
**Q4: How do I get started with the unified gateway?**
Use the `default` gateway ID in your binding or REST API calls to automatically create and use a gateway on the first authenticated request.
**Q5: Can I still use a custom gateway?**
Yes, you can specify your own gateway ID to split applications into multiple projects or apply custom configurations.
**Q6: What are the benefits of model-first routing?**
Model-first routing lets users specify the model they need while the control plane handles provider selection, failover, and load balancing for better resiliency and flexibility.
**Q7: How can I use AI Gateway credits for Workers AI?**
Unified billing now supports applying AI Gateway credits to Workers AI usage, enabling flexible spending across all supported providers.
**Q8: When will smart routing be available?**
Smart routing is currently being piloted internally and will be tested in the coming weeks before release.
—
### Conclusion
Cloudflare’s unification of AI Gateway and Workers AI represents a significant step forward in simplifying AI infrastructure management. By converging these products into a single control plane, users gain automatic observability, unified billing, and flexible routing options without sacrificing performance or control. Whether you’re already using Workers AI or AI Gateway, now is the ideal time to adopt the unified approach and prepare for upcoming smart routing capabilities.
