Sandboxing AI brokers, 100x quicker

Final September we launched Code Mode, the concept that brokers ought to carry out duties not by making software calls, however as an alternative by writing code that calls APIs. We have proven that merely changing an MCP server right into a TypeScript API can lower token utilization by 81%. We demonstrated that Code Mode may function behind an MCP server as an alternative of in entrance of it, creating the brand new Cloudflare MCP server that exposes your entire Cloudflare API with simply two instruments and beneath 1,000 tokens.

But when an agent (or an MCP server) goes to execute code generated on-the-fly by AI to carry out duties, that code must run someplace, and that someplace must be safe. You possibly can’t simply eval() AI-generated code straight in your app: a malicious consumer might trivially immediate the AI to inject vulnerabilities.

You want a sandbox: a spot to execute code that’s remoted out of your utility and from the remainder of the world, apart from the particular capabilities the code is supposed to entry.

Sandboxing is a scorching subject within the AI trade. For this job, most individuals are reaching for containers. Utilizing a Linux-based container, you can begin up any type of code execution atmosphere you need. Cloudflare even affords our container runtime and our Sandbox SDK for this objective.

However containers are costly and sluggish to start out, taking lots of of milliseconds in addition and lots of of megabytes of reminiscence to run. You most likely have to hold them heat to keep away from delays, and you could be tempted to reuse current containers for a number of duties, compromising the safety.

If we need to assist consumer-scale brokers, the place each finish consumer has an agent (or many!) and each agent writes code, containers will not be sufficient. We want one thing lighter.

And we have now it.

Tucked into our Code Mode submit in September was the announcement of a brand new, experimental function: the Dynamic Employee Loader API. This API permits a Cloudflare Employee to instantiate a brand new Employee, in its personal sandbox, with code specified at runtime, all on the fly.

Dynamic Employee Loader is now in open beta, out there to all paid Employees customers.

Learn the docs for full particulars, however here is what it seems to be like:

// Have your LLM generate code like this.
let agentCode: string = `
  export default {
    async myAgent(param, env, ctx) {
      // ...
    }
  }
`;

// Get RPC stubs representing APIs the agent ought to give you the option
// to entry. (This may be any Employees RPC API you outline.)
let chatRoomRpcStub = ...;

// Load a employee to run the code, utilizing the employee loader
// binding.
let employee = env.LOADER.load({
  // Specify the code.
  compatibilityDate: "2026-03-01",
  mainModule: "agent.js",
  modules: { "agent.js": agentCode },

  // Give agent entry to the chat room API.
  env: { CHAT_ROOM: chatRoomRpcStub },

  // Block web entry. (You can too intercept it.)
  globalOutbound: null,
});

// Name RPC strategies exported by the agent code.
await employee.getEntrypoint().myAgent(param);

That is it.

Dynamic Employees use the identical underlying sandboxing mechanism that your entire Cloudflare Employees platform has been constructed on since its launch, eight years in the past: isolates. An isolate is an occasion of the V8 JavaScript execution engine, the identical engine utilized by Google Chrome. They’re how Employees work.

An isolate takes just a few milliseconds to start out and makes use of just a few megabytes of reminiscence. That is round 100x quicker and 10x-100x extra reminiscence environment friendly than a typical container.

That implies that if you wish to begin a brand new isolate for each consumer request, on-demand, to run one snippet of code, then throw it away, you possibly can.

Many container-based sandbox suppliers impose limits on international concurrent sandboxes and fee of sandbox creation. Dynamic Employee Loader has no such limits. It does not have to, as a result of it’s merely an API to the identical know-how that has powered our platform all alongside, which has at all times allowed Employees to seamlessly scale to thousands and thousands of requests per second.

Wish to deal with one million requests per second, the place each single request masses a separate Dynamic Employee sandbox, all working concurrently? No drawback!

One-off Dynamic Employees normally run on the identical machine — the identical thread, even — because the Employee that created them. No want to speak around the globe to discover a heat sandbox. Isolates are so light-weight that we are able to simply run them wherever the request landed. Dynamic Employees are supported in each one in every of Cloudflare’s lots of of places around the globe.

The one catch, vs. containers, is that your agent wants to write down JavaScript.

Technically, Employees (together with dynamic ones) can use Python and WebAssembly, however for small snippets of code — like that written on-demand by an agent — JavaScript will load and run a lot quicker.

We people are likely to have sturdy preferences on programming languages, and whereas many love JavaScript, others may desire Python, Rust, or numerous others.

However we aren’t speaking about people right here. We’re speaking about AI. AI will write any language you need it to. LLMs are specialists in each main language. Their coaching information in JavaScript is immense.

JavaScript, by its nature on the net, is designed to be sandboxed. It’s the appropriate language for the job.

If we wish our agent to have the ability to do something helpful, it wants to speak to exterior APIs. How will we inform it in regards to the APIs it has entry to?

MCP defines schemas for flat software calls, however not programming APIs. OpenAPI affords a solution to categorical REST APIs, however it’s verbose, each within the schema itself and the code you’d have to write down to name it.

For APIs uncovered to JavaScript, there’s a single, apparent reply: TypeScript.

Brokers know TypeScript. TypeScript is designed to be concise. With only a few tokens, you can provide your agent a exact understanding of your API.

// Interface to work together with a chat room.
interface ChatRoom {
  // Get the final `restrict` messages of the chat log.
  getHistory(restrict: quantity): Promise;

  // Subscribe to new messages. Dispose the returned object
  // to unsubscribe.
  subscribe(callback: (msg: Message) => void): Promise;

  // Put up a message to talk.
  submit(textual content: string): Promise;
}

sort Message = {
  creator: string;
  time: Date;
  textual content: string;
}

Evaluate this with the equal OpenAPI spec (which is so lengthy you need to scroll to see all of it):

openapi: 3.1.0
data:
  title: ChatRoom API
  description: >
    Interface to work together with a chat room.
  model: 1.0.0

paths:
  /messages:
    get:
      operationId: getHistory
      abstract: Get latest chat historical past
      description: Returns the final `restrict` messages from the chat log, latest first.
      parameters:
        - identify: restrict
          in: question
          required: true
          schema:
            sort: integer
            minimal: 1
      responses:
        "200":
          description: A listing of messages.
          content material:
            utility/json:
              schema:
                sort: array
                gadgets:
                  $ref: "#/components/schemas/Message"

    submit:
      operationId: postMessage
      abstract: Put up a message to the chat room
      requestBody:
        required: true
        content material:
          utility/json:
            schema:
              sort: object
              required:
                - textual content
              properties:
                textual content:
                  sort: string
      responses:
        "204":
          description: Message posted efficiently.

  /messages/stream:
    get:
      operationId: subscribeMessages
      abstract: Subscribe to new messages through SSE
      description: >
        Opens a Server-Despatched Occasions stream. Every occasion carries a JSON-encoded
        Message object. The consumer unsubscribes by closing the connection.
      responses:
        "200":
          description: An SSE stream of recent messages.
          content material:
            textual content/event-stream:
              schema:
                description: >
                  Every SSE `information` subject incorporates a JSON-encoded Message object.
                $ref: "#/components/schemas/Message"

parts:
  schemas:
    Message:
      sort: object
      required:
        - creator
        - time
        - textual content
      properties:
        creator:
          sort: string
        time:
          sort: string
          format: date-time
        textual content:
          sort: string

We predict the TypeScript API is best. It is fewer tokens and far simpler to know (for each brokers and people).

Dynamic Employee Loader makes it straightforward to implement a TypeScript API like this in your individual Employee after which cross it in to the Dynamic Employee both as a way parameter or within the env object. The Employees Runtime will routinely arrange a Cap’n Internet RPC bridge between the sandbox and your harness code, in order that the agent can invoke your API throughout the safety boundary with out ever realizing that it’s not utilizing an area library.

Which means your agent can write code like this:

// Pondering: The consumer requested me to summarize latest chat messages from Alice.
// I'll filter the latest message historical past in code in order that I solely need to
// learn the related messages.
let historical past = await env.CHAT_ROOM.getHistory(1000);
return historical past.filter(msg => msg.creator == "alice");

In the event you desire to offer your brokers HTTP APIs, that is absolutely supported. Utilizing the globalOutbound choice to the employee loader API, you possibly can register a callback to be invoked on each HTTP request, in which you’ll be able to examine the request, rewrite it, inject auth keys, reply to it straight, block it, or the rest you may like.

For instance, you should utilize this to implement credential injection (token injection): When the agent makes an HTTP request to a service that requires authorization, you add credentials to the request on the way in which out. This fashion, the agent itself by no means is aware of the key credentials, and subsequently can’t leak them.

Utilizing a plain HTTP interface could also be fascinating when an agent is speaking to a well known API that’s in its coaching set, or while you need your agent to make use of a library that’s constructed on a REST API (the library can run contained in the agent’s sandbox).

With that stated, within the absence of a compatibility requirement, TypeScript RPC interfaces are higher than HTTP:

• As proven above, a TypeScript interface requires far fewer tokens to explain than an HTTP interface.

• The agent can write code to name TypeScript interfaces utilizing far fewer tokens than equal HTTP.

• With TypeScript interfaces, since you might be defining your individual wrapper interface anyway, it’s simpler to slender the interface to reveal precisely the capabilities that you just need to present to your agent, each for simplicity and safety. With HTTP, you might be extra doubtless implementing filtering of requests made in opposition to some current API. That is laborious, as a result of your proxy should absolutely interpret the that means of each API name with a view to correctly resolve whether or not to permit it, and HTTP requests are sophisticated, with many headers and different parameters that would all be significant. It finally ends up being simpler to only write a TypeScript wrapper that solely implements the capabilities you need to enable.

Hardening an isolate-based sandbox is hard, as it’s a extra sophisticated assault floor than {hardware} digital machines. Though all sandboxing mechanisms have bugs, safety bugs in V8 are extra frequent than safety bugs in typical hypervisors. When utilizing isolates to sandbox possibly-malicious code, it is essential to have further layers of defense-in-depth. Google Chrome, for instance, applied strict course of isolation because of this, however it isn’t the one doable answer.

We now have almost a decade of expertise securing our isolate-based platform. Our methods routinely deploy V8 safety patches to manufacturing inside hours — quicker than Chrome itself. Our safety structure encompasses a customized second-layer sandbox with dynamic cordoning of tenants primarily based on danger assessments. We have prolonged the V8 sandbox itself to leverage {hardware} options like MPK. We have teamed up with (and employed) main researchers to develop novel defenses in opposition to Spectre. We even have methods that scan code for malicious patterns and routinely block them or apply further layers of sandboxing. And way more.

Once you use Dynamic Employees on Cloudflare, you get all of this routinely.

We have constructed plenty of libraries that you just may discover helpful when working with Dynamic Employees:

@cloudflare/codemode simplifies working model-generated code in opposition to AI instruments utilizing Dynamic Employees. At its core is DynamicWorkerExecutor(), which constructs a purpose-built sandbox with code normalisation to deal with frequent formatting errors, and direct entry to a globalOutbound fetcher for controlling fetch() behaviour contained in the sandbox — set it to null for full isolation, or cross a Fetcher binding to route, intercept or enrich outbound requests from the sandbox.

const executor = new DynamicWorkerExecutor({
  loader: env.LOADER,
  globalOutbound: null, // absolutely remoted 
});

const codemode = createCodeTool({
  instruments: myTools,
  executor,
});

return generateText({
  mannequin,
  messages,
  instruments: { codemode },
});

The Code Mode SDK additionally supplies two server-side utility capabilities. codeMcpServer({ server, executor }) wraps an current MCP Server, changing its software floor with a single code() software. openApiMcpServer({ spec, executor, request }) goes additional: given an OpenAPI spec and an executor, it builds an entire MCP Server with search() and execute() instruments as utilized by the Cloudflare MCP Server, and higher suited to bigger APIs.

In each circumstances, the code generated by the mannequin runs inside Dynamic Employees, with calls to exterior companies revamped RPC bindings handed to the executor.

Be taught extra in regards to the library and how you can use it.

Dynamic Employees anticipate pre-bundled modules. @cloudflare/worker-bundler handles that for you: give it supply information and a package deal.json, and it resolves npm dependencies from the registry, bundles every thing with esbuild, and returns the module map the Employee Loader expects.

import { createWorker } from "@cloudflare/worker-bundler";

const employee = env.LOADER.get("my-worker", async () => {
  const { mainModule, modules } = await createWorker({
    information: {
      "src/index.ts": `
        import { Hono } from 'hono';
        import { cors } from 'hono/cors';

        const app = new Hono();
        app.use('*', cors());
        app.get('/', (c) => c.textual content('Good day from Hono!'));
        app.get('/json', (c) => c.json({ message: 'It really works!' }));

        export default app;
      `,
      "package.json": JSON.stringify({
        dependencies: { hono: "^4.0.0" }
      })
    }
  });

  return { mainModule, modules, compatibilityDate: "2026-01-01" };
});

await employee.getEntrypoint().fetch(request);

It additionally helps full-stack apps through createApp — bundle a server Employee, client-side JavaScript, and static belongings collectively, with built-in asset serving that handles content material sorts, ETags, and SPA routing.

Be taught extra in regards to the library and how you can use it.

@cloudflare/shell offers your agent a digital filesystem inside a Dynamic Employee. Agent code calls typed strategies on a state object — learn, write, search, substitute, diff, glob, JSON question/replace, archive — with structured inputs and outputs as an alternative of string parsing.

Storage is backed by a sturdy Workspace (SQLite + R2), so information persist throughout executions. Coarse operations like searchFiles, replaceInFiles, and planEdits decrease RPC round-trips — the agent points one name as an alternative of looping over particular person information. Batch writes are transactional by default: if any write fails, earlier writes roll again routinely.

import { Workspace } from "@cloudflare/shell";
import { stateTools } from "@cloudflare/shell/workers";
import { DynamicWorkerExecutor, resolveProvider } from "@cloudflare/codemode";

const workspace = new Workspace({
  sql: this.ctx.storage.sql, // Works with any DO's SqlStorage, D1, or customized SQL backend
  r2: this.env.MY_BUCKET, // massive information spill to R2 routinely
  identify: () => this.identify   // lazy — resolved when wanted, not at development
});

// Code runs in an remoted Employee sandbox with no community entry
const executor = new DynamicWorkerExecutor({ loader: env.LOADER });

// The LLM writes this code; `state.*` calls dispatch again to the host through RPC
const outcome = await executor.execute(
  `async () => {
    // Search throughout all TypeScript information for a sample
    const hits = await state.searchFiles("src/**/*.ts", "answer");
    // Plan a number of edits as a single transaction
    const plan = await state.planEdits([
      { kind: "replace", path: "/src/app.ts",
        search: "42", replacement: "43" },
      { kind: "writeJson", path: "/src/config.json",
        value: { version: 2 } }
    ]);
    // Apply atomically — rolls again on failure
    return await state.applyEditPlan(plan);
  }`,
  [resolveProvider(stateTools(workspace))]
);

The package deal additionally ships prebuilt TypeScript sort declarations and a system immediate template, so you possibly can drop the total state API into your LLM context in a handful of tokens.

Be taught extra in regards to the library and how you can use it.

Builders need their brokers to write down and execute code in opposition to software APIs, fairly than making sequential software calls separately. With Dynamic Employees, the LLM generates a single TypeScript perform that chains a number of API calls collectively, runs it in a Dynamic Employee, and returns the ultimate outcome again to the agent. Because of this, solely the output, and never each intermediate step, results in the context window. This cuts each latency and token utilization, and produces higher outcomes, particularly when the software floor is massive.

Our personal Cloudflare MCP server is constructed precisely this manner: it exposes your entire Cloudflare API by way of simply two instruments — search and execute — in beneath 1,000 tokens, as a result of the agent writes code in opposition to a typed API as an alternative of navigating lots of of particular person software definitions.

Builders are utilizing Dynamic Employees to let brokers construct customized automations on the fly. Zite, for instance, is constructing an app platform the place customers work together by way of a chat interface — the LLM writes TypeScript behind the scenes to construct CRUD apps, hook up with companies like Stripe, Airtable, and Google Calendar, and run backend logic, all with out the consumer ever seeing a line of code. Each automation runs in its personal Dynamic Employee, with entry to solely the particular companies and libraries that the endpoint wants.

“To enable server-side code for Zite’s LLM-generated apps, we needed an execution layer that was instant, isolated, and secure. Cloudflare’s Dynamic Workers hit the mark on all three, and out-performed all of the other platforms we benchmarked for speed and library support. The NodeJS compatible runtime supported all of Zite’s workflows, allowing hundreds of third party integrations, without sacrificing on startup time. Zite now services millions of execution requests daily thanks to Dynamic Workers.”

— Antony Toron, CTO and Co-Founder, Zite 

Builders are constructing platforms that generate full purposes from AI — both for his or her clients or for inner groups constructing prototypes. With Dynamic Employees, every app may be spun up on demand, then put again into chilly storage till it is invoked once more. Quick startup occasions make it straightforward to preview adjustments throughout lively growth. Platforms may block or intercept any community requests the generated code makes, conserving AI-generated apps secure to run.

Dynamically-loaded Employees are priced at $0.002 per distinctive Employee loaded per day (as of this submit’s publication), along with the standard CPU time and invocation pricing of normal Employees.

For AI-generated “code mode” use circumstances, the place each Employee is a singular one-off, this implies the value is $0.002 per Employee loaded (plus CPU and invocations). This value is often negligible in comparison with the inference prices to generate the code.

Through the beta interval, the $0.002 cost is waived. As pricing is topic to alter, please at all times examine our Dynamic Employees pricing for probably the most present info. 

In the event you’re on the Employees Paid plan, you can begin utilizing Dynamic Employees immediately. 

Use this “hello world” starter to get a Employee deployed that may load and execute Dynamic Employees. 

You can too deploy the Dynamic Employees Playground, the place you’ll be capable to write or import code, bundle it at runtime with @cloudflare/worker-bundler, execute it by way of a Dynamic Employee, see real-time responses and execution logs.

Dynamic Employees are quick, scalable, and light-weight. Discover us on Discord if in case you have any questions. We’d like to see what you construct!