toolnexus.client/create-client

Clojure (JVM) + cljgo · package net.clojars.muthuishere/toolnexus · SPEC §8 · clojure/src/toolnexus/client.cljc

(create-client {:base-url      "…"
                :model         "…"
                :retries       3        ; extra attempts after the first (default 0)
                :retry-base-ms 500      ; backoff base; delay = base * 2^attempt + jitter
                :timeout-ms    30000    ; per HTTP call, passed to the transport
                :retryable-statuses [520 521 522]  ; ADDED to the default set, never replacing it
                :on-error      (fn [info] :retry)})  ; or :fail

;; :on-error receives, per failed attempt:
;; {:error      "the response body, or a koine error keyword"
;;  :status     429            ; nil on a transport failure
;;  :attempt    0              ; zero-based
;;  :retryable? true}          ; the default verdict, for you to accept or override

The default policy retries 429, 500, 502, 503, 504 and 529 plus any transport failure, and treats everything else as terminal. That is the whole list, not “429 plus any 5xx” — a 501, a 507 and the Cloudflare 520–527 family are terminal until you say otherwise. Backoff is exponential from :retry-base-ms — 500ms, 1s, 2s, each plus 0–99ms of jitter — and a numeric Retry-After header, in seconds, overrides it. The HTTP-date form of Retry-After is deliberately not parsed: date parsing is not portable across these two hosts without reaching past koine, and a server that sends it gets our backoff instead of a wrong answer.

:retryable-statuses says otherwise. It is a collection of statuses added to the default set and never replacing it, so it can only make more statuses retryable, never fewer — a host cannot accidentally drop 429 and lose Retry-After handling with it. The case it exists for is a Cloudflare-fronted origin:

(create-client {:base-url base-url
                :model model
                :retries 3
                :retryable-statuses [520 521 522 523 524 525 526 527]})

It sets the default verdict only. :on-error still runs on every failed attempt and has the last word, so an :on-error returning :fail overrides a status the host itself listed. Retry-After handling is unchanged either way.

:retries counts extra attempts, so the default of 0 means one try. A budget of 3 means up to four calls.

:on-error overrides the default verdict in either direction, but :retries still bounds it. A classifier that could loop unbounded would be a denial-of-service on your own bill, so a forced :retry on a terminal status still stops when the budget runs out.

The verdicts are :retry and :fail, and nothing else. An LLM failure never becomes a §10 suspension: suspension is a user-action pause, and a 500 from the provider is not one. When the budget is exhausted the call throws an ex-info carrying :status or :error — a provider failure is not something the model can be shown and asked to retry, unlike a tool error.

:timeout-ms depends on the host

The client passes :timeout-ms to koine.http/request, which classifies a timeout as data — {:status nil :error :timeout} — so the retry loop treats it as one more retryable failure rather than a host-specific exception. Measured 2026-08-01 against koine 0.9.0: the JVM bounds the call; cljgo accepts the budget and ignores it. The port’s own suite reports that gap rather than asserting past it. Passing the budget through is all this port can do until koine enforces it on both hosts.

When to use it

• Any production run — 429 and 503 are normal traffic from every provider, and a bare client makes one attempt and gives up.
• When your gateway has its own error vocabulary — an :on-error classifier is how a 402 top-up-required or a proxy’s 418 gets the treatment it deserves.
• When a run must not hang — :timeout-ms bounds each HTTP call, and :max-turns bounds how many there can be.

Why this and not the alternative

Wrap the transport instead when the policy is not about the LLM

:http-client on create-client hands you the whole (url headers body) call. If you already have a resilience library, a circuit breaker or a proxy with its own retry rules, put them there and leave :retries at 0 — two retry layers stacked on each other multiply attempts rather than adding them. Use :on-error when the decision genuinely depends on the provider’s status code.

Note the scope: this policy covers the LLM path only. MCP transports are their own seam, and a tool that fails returns an error ToolResult the model can react to — retrying it is the model’s job, not the client’s.

Examples

Surviving rate limits

(require '[toolnexus.client :as client])

(def llm
  (client/create-client
    {:base-url      "https://api.anthropic.com"
     :style         "anthropic"
     :model         "claude-sonnet-4-5"
     :retries       3        ; up to four attempts per LLM call
     :retry-base-ms 500      ; 500ms, 1s, 2s (+jitter) — unless Retry-After says otherwise
     :timeout-ms    60000}))

(client/run llm "Summarise the incident" {:toolkit toolkit})

Two 429s followed by a 200 costs three calls and returns normally. A 400 costs one call and throws immediately — no budget is spent on an error that will not change.

A classifier for your gateway

(def llm
  (client/create-client
    {:base-url "https://gateway.internal/v1"
     :model    "gpt-4.1"
     :retries  4
     :on-error (fn [{:keys [status attempt error retryable?]}]
                 (println "llm attempt" attempt "failed:" status)
                 (cond
                   ;; the gateway rate-limits with 409, which is not in the default set
                   (= 409 status) :retry
                   ;; do not burn the budget on a bad key
                   (= 401 status) :fail
                   ;; anything else: keep the default verdict
                   retryable?     :retry
                   :else          :fail))}))

Returning anything other than :retry means :fail, so a classifier that falls through a cond fails safe rather than looping.

Bounding the whole run, not just one call

Resilience options bound a single HTTP call; :max-turns bounds how many the loop may make. Both matter — a model that keeps calling tools will otherwise spend a long time succeeding.

(def llm
  (client/create-client
    {:base-url   "https://api.openai.com/v1"
     :model      "gpt-4.1"
     :timeout-ms 20000       ; per call
     :max-turns  5           ; per run
     :retries    2
     :on-metric  (fn [m] (when (= "run" (:event m))
                           (println "run took" (:ms m) "ms over" (:turns m) "turns")))}))

(let [r (try (client/run llm "Reindex everything" {:toolkit toolkit})
             (catch Throwable e
               ;; an LLM failure that outlived the retry budget arrives here
               {:status "failed" :text (ex-message e) :ex-data (ex-data e)}))]
  (case (:status r)
    "done"       (println (:text r))
    "incomplete" (println "turn limit:" (:limit r))
    "failed"     (println "provider error:" (:text r))
    (println (:status r))))

Only the LLM call throws. A tool that throws is converted to an error ToolResult and the run continues, which is why the try here is narrow rather than a blanket around everything.

Options

Option	Default	What it does
:retries	0	Extra attempts after the first. Bounds :on-error too.
:retry-base-ms	500	Backoff base; the delay for attempt n is base * 2^n plus 0–99ms of jitter.
:timeout-ms	none	Per HTTP call, passed to koine.http/request. See the host caveat above.
:retryable-statuses	none	Extra statuses added to the default retryable set. Widens it only — never shrinks it — and :on-error still has the last word.
:on-error	none	(fn [info] :retry-or-:fail). Overrides the default verdict in either direction.

The default verdict

Condition	Default
Transport failure (no status)	:retry — :retryable? is true, :status is nil
429, 500, 502, 503, 504, 529	:retry
Anything in :retryable-statuses	:retry
Any other non-2xx	:fail

The classifier has the same backoff base

Classifier carries the same knob under the same name, the same default of 500, and the same base * 2^attempt shape — minus the jitter, which the classifier path does not add. See create-classifier.

See also

• toolnexus.client/create-client — The unified client: system prompt, skills injection, parallel and chained tool calls, retries, memory.
• toolnexus.client/run — Send a prompt, let the loop call tools until the model stops, get a RunResult.
• toolnexus.client/create-client — Emit one metric event per model call and per tool call: tokens, latency, outcome.
• toolnexus.client/in-memory-store — Swap in-memory history for your own store so a conversation survives a process restart.