Toolnexus.Classifier.Noul

Elixir · package toolnexus · SPEC §8B · elixir/lib/toolnexus/classifier.ex

%Toolnexus.Classifier.Noul{instructions: String.t(), criteria: map() | nil}
# criteria: %{"true" => description, "false" => description} — or nil

%Toolnexus.Classifier.Choice{instructions: String.t(), criteria: %{String.t() => String.t()}}
# criteria: one description per option id, 1..255 options

%Toolnexus.Classifier.Score{instructions: String.t(), criteria: [String.t()]}
# criteria: 2..10 ordered levels — the LIST ORDER is the level numbering

Toolnexus.Classifier.choice_over(instructions :: String.t(), items :: map()) :: Choice.t()

The three question types, the criteria each one needs, and the limits enforced client-side before the request. You pass them as a map from caller-chosen keys to question structs. The keys are addressing, not content: they are never transmitted, so a key may be a tool, skill or agent name verbatim, and two evaluations differing only in their keys send identical bytes.

Questions are independent. One answer is never context for another — which is also why many questions in one call cost one state ingest instead of several.

When to use it

type	you get back	reach for it when
%Noul{}	one number in 0..1 — no confidence; the number is the answer	is this true, is this text from an untrusted source
%Choice{}	one of your named options, a probability for every option, and a confidence	routing to a skill, a tool, an agent, a move
%Score{}	a number against an ordered rubric of 2–10 levels (1.21 is a real answer), with a probability per level	risk, urgency, severity

• %Noul{} — :criteria labels the true and the false case, and is optional. nil and %{"true" => "", "false" => ""} are different values: nil omits the field from the wire entirely, an empty map sends empty strings. Both survive intact.
• %Choice{} — the answer is always one of the options you offered, and the probability map names exactly those options. choice_over/2 builds one from any (name, description) pairs — an atom key travels as its plain name, so :billing sends billing, never ":billing", and an atom roster and its string equivalent are the same request.
• %Score{} — the list order is the numbering; level 0 is the first string you passed. Nothing sorts it, and no canonicaliser in the library reorders an array for exactly this reason.

Why this and not the alternative

criteria[id] is the whole ball game on a choice

For a %Choice{}, the description is the only thing that differentiates one option from another to the model. The instructions describe the question and the state describes the situation; neither tells the model what picking billing rather than technical would mean. Passing the id itself, an empty string, or one value repeated is schema-valid, returns HTTP 200 and a well-formed distribution — and ranks at chance. Measured: options described by consequence scored 17, 17, 17 apples; the same state with each description replaced by its own id scored 0, 1, 0, on the floor of a shuffle control. The full argument, with the three rules that follow from it, is in The encoding obligation.

Limits, enforced before the request

evaluate/3 validates every question client-side, walking the keys in sorted order so the same malformed set always names the same key first. On a violation you get {:error, reason}, the reason names the offending question key and the limit, and no request is sent — faster and more legible than the backend’s own 400 "Too many choices. Must have at most 255 choices.", which is still surfaced intact if it ever arrives.

rule	limit	failure
a %Choice{}’s options	1..255	classifier: question "k": a choice needs 1..255 options, got N
a %Score{}’s levels	2..10	classifier: question "k": a score needs 2..10 ordered levels, got N
a %Noul{}’s criteria	none	absent (nil) and empty are both valid, and different
the questions map	non-empty	classifier: no questions to evaluate

Separately, degenerate %Choice{} criteria — every description empty, or every description equal to its own id, or every description identical — emit one "classifier.warning" event per question key per classifier, carried in the event’s :warning field and never :error. The request still goes out byte-unchanged: this is detection, not repair, because repairing would invent descriptions you did not write. A single-option choice is never reported.

Examples

1. The smallest useful call — all three types in one evaluation

alias Toolnexus.Classifier
alias Toolnexus.Classifier.{Choice, Decision, Noul, Score}

state = "Ticket 4021: charged twice for the annual plan; not blocked, but I want the money back."

questions = %{
  "wants_money_back" => %Noul{instructions: "Is the customer asking for money to be returned?"},
  "department" => %Choice{
    instructions: "Which desk should own this ticket?",
    criteria: %{
      "billing" => "own it here when the problem is money that moved: a duplicate charge, a refund owed",
      "shipping" => "own it here when the problem is a physical parcel: a late or damaged delivery",
      "technical" => "own it here when the problem is the product itself: a login that fails"
    }
  },
  "urgency" => %Score{
    instructions: "How fast does this ticket need a human?",
    criteria: [
      "the customer is working normally and is waiting on an answer",
      "the customer is inconvenienced and will chase if nobody replies today",
      "the customer is blocked from working right now and every hour costs them"
    ]
  }
}

recorded = %{
  "model" => "typesafe/jev-1.13-20260917",
  "answers" => %{
    "wants_money_back" => %{"type" => "noul", "noul" => 0.99},
    "department" => %{
      "type" => "choice",
      "choice" => "billing",
      "probabilities" => %{"billing" => 1, "shipping" => 0, "technical" => 0},
      "confidence" => 1
    },
    "urgency" => %{
      "type" => "score",
      "score" => 0.49,
      "legend" => %{"0" => "waiting", "1" => "inconvenienced", "2" => "blocked"},
      "probabilities" => %{"0" => 0.52, "1" => 0.48, "2" => 0},
      "confidence" => 0.27
    }
  },
  "usage" => %{"input_tokens" => 516, "output_tokens" => 72}
}

{:ok, judge} =
  Classifier.create(
    style: "static",
    model: "jev-1.13.0",
    decisions: [%{state: state, questions: questions, response: recorded}]
  )

{:ok, decision} = Classifier.evaluate(judge, state, questions)

{:ok, noul} = Decision.noul(decision, "wants_money_back")
{:ok, choice} = Decision.choice(decision, "department")
{:ok, score} = Decision.score(decision, "urgency")

true = noul.noul == 0.99
true = choice.choice == "billing"
# A score MAY fall between levels — 0.49 is a genuine split between level 0 and level 1.
true = score.score == 0.49

IO.puts("ok: refund=#{noul.noul} desk=#{choice.choice} urgency=#{score.score}")

2. The realistic case — the limits fire before anything leaves the process

No credential is configured here and none is needed: validation runs first, so an over-long rubric or an empty option set never reaches the wire.

alias Toolnexus.Classifier
alias Toolnexus.Classifier.{Choice, Noul, Score}

# style: "systemone" with NO network reachable — the point is that we never get there.
{:ok, judge} = Classifier.create(base_url: "http://127.0.0.1:1/never-called")

{:error, too_many_levels} =
  Classifier.evaluate(judge, "some state", %{
    "urgency" => %Score{
      instructions: "How urgent?",
      criteria: Enum.map(1..11, fn i -> "level #{i}" end)
    }
  })

true = String.contains?(too_many_levels, ~s("urgency"))
true = String.contains?(too_many_levels, "2..10")

{:error, too_many_options} =
  Classifier.evaluate(judge, "some state", %{
    "route" => %Choice{
      instructions: "Where to?",
      criteria: Map.new(1..256, fn i -> {"opt#{i}", "description #{i}"} end)
    }
  })

true = String.contains?(too_many_options, ~s("route"))
true = String.contains?(too_many_options, "1..255")

{:error, empty} = Classifier.evaluate(judge, "some state", %{})
true = String.contains?(empty, "no questions")

# A noul needs no criteria at all: nil is valid, and is NOT the same value as an empty map.
absent = Classifier.canonical_request("m", %{"q" => %Noul{instructions: "true?"}})
empty_crit = Classifier.canonical_request("m", %{"q" => %Noul{instructions: "true?", criteria: %{}}})

true = absent != empty_crit
false = String.contains?(absent, "criteria")
true = String.contains?(empty_crit, "criteria")

IO.puts("ok: limits named the key before any request — #{too_many_levels}")

3. The full surface — degenerate criteria are reported, once, and change nothing

alias Toolnexus.Classifier

{:ok, warnings} = Agent.start_link(fn -> [] end)

state = "the user typed: open the billing page"

# Every description is just its own option id — schema-valid, HTTP 200, ranks at chance.
bad = %{
  "route" =>
    Classifier.choice_over("Where should this go?", %{
      "billing" => "billing",
      "shipping" => "shipping"
    })
}

recorded = %{
  "model" => "typesafe/jev-1.13-20260917",
  "answers" => %{
    "route" => %{
      "type" => "choice",
      "choice" => "billing",
      "probabilities" => %{"billing" => 0.51, "shipping" => 0.49},
      "confidence" => 0.51
    }
  },
  "usage" => %{"input_tokens" => 40, "output_tokens" => 8}
}

{:ok, judge} =
  Classifier.create(
    style: "static",
    model: "jev-1.13.0",
    decisions: [%{state: state, questions: bad, response: recorded}],
    on_metric: fn
      %{event: "classifier.warning"} = ev -> Agent.update(warnings, &[ev | &1])
      _ -> :ok
    end
  )

{:ok, _} = Classifier.evaluate(judge, state, bad)
{:ok, _} = Classifier.evaluate(judge, state, bad)

# ONE warning for the key, however many times it is asked — a per-turn judge must not flood.
[w] = Agent.get(warnings, & &1)
true = w.question == "route"
true = String.contains?(w.warning, "degenerate criteria")
# It travels in :warning, never :error. A consumer filtering the sink on "has an error"
# must not count this as a failure.
false = Map.has_key?(w, :error)

IO.puts("ok: one warning for #{inspect(w.question)}, request unchanged")

See also

• Toolnexus.Classifier.create — A sibling of the client: pre-declared typed questions in, calibrated answers out — no messages, no tool calling, no loop.
• Toolnexus.Classifier.Decision — One answer per question under the caller’s own keys, read through typed accessors that fail loudly rather than hand back a zero.
• The encoding obligation — the measurements behind the caution above, and the three rules that follow.
• Typed decisions — where a judgment sits next to an if and a frontier-model turn.
• Cookbook: a typed decision — the same three questions, end to end.