Skip to content
toolnexus

Fan out research across parallel sub-agents

Someone asks: “Should we migrate our billing off Stripe?” That is not one question. It is five — fee structure, migration cost, feature parity, what our own code depends on, what the competitors did — and each one is a research thread with its own sources, its own dead ends, and its own several thousand tokens of raw material.

One agent answering all five in sequence is slow (five times the latency) and worse (by thread four its context is packed with thread one’s discarded sources). A coordinator with a team is faster and cleaner: it splits the question, dispatches the threads concurrently, and each worker returns one paragraph instead of a pile of pages.

The shape

Section titled “The shape”
coordinator                       one model, one plan, one synthesis
├── market-analyst    ─┐
├── cost-analyst       │          all four dispatched in ONE turn
├── code-auditor       │          → they run concurrently
└── competitor-scout  ─┘

The coordinator’s turn emits four task tool calls at once. Parallel tool calls in a single turn already run concurrently in the shipped client loop — and task is just a tool. Concurrency is free: there is no parallel() API to learn, no executor to configure. The model asking for four things in one turn is the fan-out.

Each child gets a fresh transcript, spends its own window, and returns exactly one tool message to the parent. The coordinator’s context grows by four paragraphs, not by four research sessions.

1. Designing the team — does is the routing signal

Section titled “1. Designing the team — does is the routing signal”

This is where fan-out quality is actually decided, and it is prose, not configuration.

The task tool’s description is composed from the caller’s team — sorted by name, each entry built from that agent’s does (the same progressive-disclosure pattern as the skill tool). That composed description is the entire basis on which the coordinator picks a worker and writes its prompt. A team of four agents all described as “researches things” will get random routing and overlapping work.

Weak does Strong does
“researches the market” “Sizes a market and names the incumbents. Give it a product category and a geography.”
“looks at costs” “Estimates one-time and recurring cost for a stated migration. Needs current volumes to be useful.”
“checks the code” “Answers what OUR codebase depends on for a named vendor. Read-only; cites file:line.”

Three rules that hold up in production:

  1. State the input shape. “Give it a product category and a geography” makes the coordinator write a better prompt, which is most of the win.
  2. State the boundary. “Read-only”, “does not have web access”, “US only” — the coordinator routes around limits it can see and hallucinates around limits it cannot.
  3. Make the names disjoint. Overlapping remits produce duplicated tokens, and duplicated tokens are the whole cost of fan-out.

Scoping backs the prose up structurally: uses is the worker’s toolkit view, and the registry is the transitive closure of the entry agent’s team graph — an agent you did not wire in is not reachable, and a task call naming an out-of-team agent is refused with the team’s names listed.

2. Bounding the fan-out — maxConcurrent

Section titled “2. Bounding the fan-out — maxConcurrent”

An unbounded fan-out is a rate-limit incident. Two gates bound it, at different scopes:

Gate Where What it bounds
budget.maxConcurrent per parent agent Running children under that parent. Admission is atomic with wake; over the cap, wakes queue FIFO and a completed sibling transfers its slot. Default 8.
maxConcurrentTurns runtime-wide (RuntimeOptions) Concurrent LLM HTTP calls across the entire tree. Wraps only the HTTP call — never a whole run, which would deadlock a parent against its own children — and releases on acquirer death. Default 8.
budget.maxChildren per parent agent How many children may be spawned at all (a total, not a rate).

Use maxConcurrent to shape the work (how wide is a sensible fan-out for this question) and maxConcurrentTurns to respect your provider (how many requests may this process have in flight). They compose: eight workers with maxConcurrentTurns: 3 still run eight threads, but only three are ever talking to the model at once.

3. Seeing the true cost — usage roll-up

Section titled “3. Seeing the true cost — usage roll-up”

Fan-out spends money in places you did not directly call. A child’s tokens, turns, and tool calls roll up into its parent’s usage, so the root result’s totalTokens is the ledger for the whole tree — coordinator plus every worker plus anything they delegated to.

That single number is what you meter, alert on, and compare against the single-agent baseline. For a per-handle breakdown, the runtime’s read-only list() returns one row per handle with its id, state, and rolled-up tokens:

const r = await coordinator.run(question, { llm })


console.log(r.totalTokens)          // the WHOLE tree — this is your bill
for (const h of r.runtime.list()) { // per-handle breakdown, tree order
  console.log(h.id, h.state, h.tokens)
  // root/coordinator.1            idle  184203
  // root/coordinator.1/cost-analyst.2   idle   41120
}

4. A failed worker must not kill the run

Section titled “4. A failed worker must not kill the run”

The most common real failure in a fan-out is one worker dying — a flaky source, a bad prompt, a timeout. If that propagated as an exception, one bad thread would discard four good ones.

It does not. A failed child Run crosses the handle boundary as a uniform isError tool result, never an exception. The coordinator’s model sees the failure in its transcript alongside the successes and decides what to do: re-prompt the worker, route the question to a different one, or write the report with a stated gap. Mechanical retry and backoff already ran at the level where the failure happened; what reaches the parent is the considered outcome. Only the root throws to the host.

That is why the coordinator’s soul should say what to do with a failure. Prose is the policy:

If a worker returns an error or an empty result, do not abandon the report.
Retry it ONCE with a narrower prompt. If it fails again, route the question to
the closest other worker, and if nothing covers it, state the gap explicitly in
the final report under "Not covered". Never present a gap as a finding.

The same rule covers budget stops: a worker that hits its ceiling returns status: "incomplete" with partial work preserved — a partial answer the coordinator can still use, not a crash.

5. When a worker needs a human — durable resume

Section titled “5. When a worker needs a human — durable resume”

A worker can hit something only a person can clear: a paywalled source needing login, an approval, a clarifying question. It suspends, and the suspension escalates one hop at a time to the nearest waitFor interpreter — worker → coordinator → root. Give the coordinator a waitFor and the pause never leaves the process. Give it none and the root run returns status: "pending" carrying the Request, with data.path naming the suspended handle.

Parked levels burn zero tokens. A tree can sit suspended for days at no cost.

On resume, the Answer routes to the deepest suspended handle, which continues from its checkpoint (turns and usage grow, they never reset). The cascade re-runs each parent — and this is the part that matters for a fan-out — a re-invoked task reattaches to the existing child by task key (agent + prompt): settled ⇒ its recorded result, suspended ⇒ its pending, running ⇒ await. The other three workers are not re-run. Reattachment, not transcript inspection, is the required idempotency mechanism in every port.

const r = await coordinator.run(question, { llm })
if (r.status === "pending") {
  const req = r.pending!                     // { id, kind, prompt, url?, data: { path } }
  await db.save(jobId, req)                  // plain serializable data
  await slack.post(channel, req.prompt)      // ask the human, out of band
  // ... hours later, in the same process holding the runtime:
  await r.runtime.resume({ id: req.id, ok: true, data: { credential } })
}

The full orchestrator

Section titled “The full orchestrator”

Four workers, disjoint remits, a bounded fan-out, and a coordinator whose soul tells it to dispatch in parallel and how to handle a failure.

import { createToolkit, agents } from "toolnexus"


const llm = { baseUrl: "https://openrouter.ai/api/v1", style: "openai" as const, model: "anthropic/claude-sonnet-4" }


// Workers get a read-only view: fetch + search, never write or shell.
const research = await createToolkit({
  builtins: { tools: { bash: false, write: false, edit: false, apply_patch: false } },
})
const tools = research.tools()


const worker = (name: string, does: string, soul: string) =>
  agents.agent(name, { does, soul, uses: { tools }, budget: { maxTurns: 15, maxTokens: 80_000 } })


const team = [
  worker("cost-analyst",
    "Estimates one-time and recurring cost of a stated migration. Needs current volumes to be useful.",
    "Produce a cost range with the assumptions named. Say 'unknown' rather than inventing a number."),
  worker("competitor-scout",
    "Finds what comparable companies did about a named decision, and what happened after.",
    "Three concrete examples with sources. No generic industry commentary."),
  worker("code-auditor",
    "Answers what OUR codebase depends on for a named vendor. Read-only; cites file:line.",
    "Search the repo. Cite file:line for every dependency you report. Never guess."),
  worker("risk-officer",
    "Argues the strongest case AGAINST a proposed change. Adversarial by design.",
    "Your job is to find what breaks. Be specific and concrete, never rhetorical."),
]


const coordinator = agents.agent("coordinator", {
  does: "Answers a strategic question by delegating research and synthesizing the results",
  soul: [
    "Split the question into INDEPENDENT parts, one per worker, and dispatch them ALL in a",
    "SINGLE turn so they run in parallel. Give each worker a self-contained prompt — they",
    "cannot see this conversation.",
    "",
    "If a worker errors or returns nothing: retry ONCE with a narrower prompt, then route to",
    "the closest other worker. If nothing covers it, state the gap under 'Not covered'.",
    "Never present a gap as a finding.",
    "",
    "Finish with a recommendation, the evidence for it, and what would change your mind.",
  ].join("\n"),
  team,
  budget: { maxTurns: 25, maxTokens: 600_000, maxConcurrent: 4, maxChildren: 8, maxDepth: 2 },
})


const r = await coordinator.run(
  "Should we migrate billing off Stripe? We do ~40k charges/month across US and EU.",
  { llm, maxConcurrentTurns: 3 },   // provider-friendly: 4 workers, 3 requests in flight
)


console.log(r.status, r.totalTokens)          // whole-tree ledger
console.log(r.text)                            // the brief
for (const h of r.runtime.list()) console.log(h.id, h.state, h.tokens)

When fan-out beats one big agent — and when it does not

Section titled “When fan-out beats one big agent — and when it does not”

Fan-out is not free. Be honest about the trade before you reach for it.

It wins when:

Condition Why
The parts are independent Nothing serializes; wall-clock collapses toward the slowest single worker.
Each part generates a lot of raw material This is the real win. Five workers reading forty sources each return five paragraphs, not two hundred pages. The coordinator’s context stays small and its synthesis stays sharp.
The parts need different tool scopes The code auditor gets the repo, the market scout gets the web, and neither can reach the other’s surface. One agent with the union of both is a larger attack surface.
You want an adversarial step A risk-officer on a fresh transcript genuinely argues the other side. The same model that just wrote the recommendation will not.
You want cheap parts Per-agent model lets a summarizer run on a small model while the coordinator runs on a large one.

It loses when:

Condition Why
The parts are dependent Worker 2 needs worker 1’s answer, so nothing runs in parallel and you paid the overhead anyway.
The task is small Every task call costs a spawn, a fresh system prompt, and a synthesis turn. Below roughly a few thousand tokens of real research per part, one agent is cheaper and faster.
The parts overlap Two workers researching the same thing bill you twice for one answer. This is a does-design failure.
You need the full transcript The parent gets the child’s final text and metadata — nothing else. If you need the child’s intermediate reasoning, delegation is the wrong tool.
Total tokens are the binding constraint Fan-out trades tokens for latency and quality. Each worker pays its own system prompt and its own framing of the question. Expect a fan-out to cost more tokens than one sequential agent — you are buying speed and context hygiene, not savings.

See also

Section titled “See also”