Backends & configuration

Read this before you set a threshold

A decision is advisory. Calibration is not correctness — a decision can be confidently wrong. Authority is enforced in code: numeric limits, permission checks and allowlists stay in your if statements, never in a probability. A classifier interprets; it never authorises. Nothing on this page is a security control.

One option chooses where a decision comes from. Everything else — the questions, the state, the answers you read back — is identical across all four.

style	what it is
systemone (default)	one POST {baseUrl}/systemone with the canonical body. Calibrated.
llm	the same questions as one structured-output call on any Client. A compatibility exit for a host with no System One credential. Reports calibrated: false.
custom	your own function — a fine-tuned encoder, a rules engine, or a cache in front of either.
static	recorded decisions. This is what CI runs: no network, no credential.

static is not a convenience

The live backend is non-deterministic. The same bytes, twelve times, move by σ ≈ 0.015 — so static is the only backend a test may assert a number against.

That is why every test in this repo, in all seven ports, runs on static: no network, no credential, no cost, and a numeric assertion that means something. A live call, if it is run at all, asserts shape only. It is also why the runnable examples work for you with nothing configured — they replay one recorded decision unless TYPESAFE_API_KEY or OPENROUTER_API_KEY is present in the environment, in which case they go live.

A recorded decision is a (state, questions, response) triple, matched on the canonical request bytes. Record it once from a live run; it stays honest because the questions are part of the key — change the encoding and the recording no longer matches.

Two ways to reach systemone: TypeSafe directly, or a gateway

style: "systemone" says what a decision is, not whose endpoint answers it. Two endpoints serve the same wire, and the library ships configured for the first:

	TypeSafe’s own API (the default)	OpenRouter
baseUrl	https://api.typesafe.ai/v1	https://openrouter.ai/api/v1
model	jev-latest (answers as jev-1.13.0)	typesafe/jev-1.13
apiKeyEnv	TYPESAFE_API_KEY	OPENROUTER_API_KEY
credential	a first-party TypeSafe key	a gateway key you may already hold
in the path	nobody	a gateway
usage.cost	not returned	returned
p50 / p95	339 ms / 449 ms	351 ms / 400 ms

// TypeSafe's own API — these three values are the library defaults, spelled out
const judge = createClassifier({
  baseUrl: "https://api.typesafe.ai/v1",
  model: "jev-latest",
  apiKeyEnv: "TYPESAFE_API_KEY", // the NAME of an env var, never the value
})

Neither one is the fast one. Those latencies are a warm, interleaved measurement of both, and they are a tie: TypeSafe wins p50 by 12 ms and loses p95 by 49 ms, which is noise at this sample size. Do not choose between them on speed. The real difference is a gateway dependency versus a first-party key — one fewer party between you and the model, against one more key to obtain when you already have an OpenRouter account.

A cost budget only works through the gateway

usage.cost is a gateway field. TypeSafe’s own API returns model, answers and usage (input and output tokens) and no cost at all — and it does not return OpenRouter’s id or provider either. Every port reports an absent cost as absent, never as 0, precisely so a budget cannot read “free” off a backend that simply does not say. If you meter spend per decision from usage.cost, run through OpenRouter, or meter on tokens instead.

llm is an exit, not an equal

The llm style sends the same typed questions as one structured-output call on any Client, so a host with no System One credential is not blocked. It is not a substitute.

Measured on one routing job, the llm backend was 3.3–4.4× slower and 2.5–3.4× costlier, returned no distribution, emitted round self-reported confidence (0.95, 1.00), and on one fixture disagreed outright with the calibrated backend.

A threshold tuned on one backend does not transfer to another

Read calibrated at the point where you compare a number to a threshold. The systemone style reports true; llm reports false unless it derived its probabilities from provider token probabilities. If calibrated is false, either re-tune for that backend or decline to threshold at all.

const d = await judge.evaluate(state, questions)
if (!d.calibrated) throw new Error("thresholds were tuned on a calibrated backend")
if (d.score("risk").score >= 2.5) return deny()

Leave clearance, too: a band boundary within ~0.1 of a typical value will flip between runs on the same backend.

Configuration

ClassifierOptions mirrors ClientOptions field-for-field wherever a field makes sense, so a host that has configured one has configured the other: style, baseUrl, model, apiKeyEnv, headers, the injectable transport, timeout, retries/onError, requestParams/bodyTransform, onMetric, client (for llm), evaluate (for custom), decisions (for static), and retryableStatuses.

Two deliberate differences:

• apiKeyEnv takes the NAME of an environment variable, never a value. It is read at call time and never logged. headers values expand ${ENV_VAR} at call time on the same terms. No credential and no expanded header value appears in any log, metric, error message or returned value — an authentication failure names the status and the endpoint and nothing else.
• timeout bounds one request, not a run. A classifier has no loop to bound.

There is no second retry policy: onError and the Retry-After rule are the client’s ones verbatim, and there is no suspend tier here — there is nothing to suspend into. The default retryable set is 408, 429, 500, 502, 503, 504, 529 plus transport failures. It is an exhaustive list, not shorthand for “5xx”: 501, 505 and the Cloudflare 520–527 family are terminal unless you say otherwise. 529 Overloaded is in the list because TypeSafe documents it as retry-with-backoff.

Anything else is opt-in, declaratively, through retryableStatuses:

const judge = createClassifier({
  // A Cloudflare-fronted origin answers 520-527 when the tunnel wobbles. These are ADDED to the
  // defaults, never substituted for them — 429 keeps retrying, and keeps honouring Retry-After.
  retryableStatuses: [520, 521, 522, 523, 524, 525, 526, 527],
})

It decides the default classification only. onError still runs on every failed attempt and has the final say, so onError returning "fail" overrides a status you listed yourself. The option exists on ClientOptions and ClassifierOptions alike, with the same meaning.

The canonical request

For identical questions and model, every port emits byte-identical questions + model bytes. That is what makes a recording made in one language replayable in another, and a fixture shared across seven test suites:

• object keys sorted recursively in ASCII order — not culture-sensitive order;
• arrays are never reordered — a score rubric’s order is its level numbering, so a “sort everything” canonicaliser would silently renumber the rubric;
• compact separators, no spaces;
• <, >, &, ', quotation marks and non-ASCII characters transmitted raw, never escaped;
• absent and empty are different: an absent criteria is absent, not null and not {}.

state is transmitted verbatim as you supplied it, and is explicitly outside that claim — numbers do not canonicalise identically across seven languages, so the library does not pretend they do.