How to run thousands of AI agents in parallel

The jump from one agent to ten thousand

You got one agent working. Now product wants it running across every customer, every PR, every document — all at once. Going from one to ten thousand isn't a bigger for loop; it's an infrastructure problem.

Here's what actually breaks at scale, and how to design around it.

1. Isolation is non-negotiable

At ten agents you can get away with shared processes. At ten thousand — many of them running untrusted, model-generated code — a single bad actor takes down the rest. Each agent needs its own environment: its own filesystem, its own network, its own blast radius.

Containers help, but they share a kernel. For true isolation under hostile workloads, hardware-isolated microVMs give you VM-grade boundaries at near-container speed.

2. Cold starts decide your ceiling

If a fresh environment takes ten seconds to boot, you can't burst. Spiky workloads — a thousand PRs landing at 9am — demand sub-second starts. Anything slower and you're either over-provisioning (paying for idle) or queueing (shipping slow).

Aim for cold starts under a few hundred milliseconds. That's the line between "scale on demand" and "pre-warm a fleet and pray."

3. State is the silent killer

A surprising amount of agent work is stateful: a cloned repo, an installed toolchain, a warm cache, a half-finished task. Rebuild that from scratch on every run and you'll burn most of your compute on setup instead of work.

The pattern that scales: prepare a base environment once, snapshot it, then fork. Each of your thousand agents starts from the snapshot in milliseconds, already warm.

const base = await am.create({ model: "claude-opus-4.8" })
await base.exec("git clone … && npm install")
const snapshot = await base.snapshot()

// fan out 1,000 ready-to-go agents
const fleet = await am.fork(snapshot, { count: 1000 })

4. Pay for work, not for waiting

The naive way to run a fleet is to keep it hot. The expensive way, too. Most agents spend most of their life idle — waiting on a model, a human, an API. Bill per second and hibernate the idle ones to near-zero, and your cost tracks actual work instead of wall-clock time.

Putting it together

Thousands of agents in parallel comes down to four properties: hard isolation, sub-second starts, cheap state via snapshots, and per-second billing with hibernation. Get those right and "scale to ten thousand" stops being a project and becomes a single API call.

That's exactly what agentmachines is built for. Start for free and spin up your first fleet.