Compaction.compactor

Java · package io.github.muthuishere:toolnexus · SPEC §7F · Compaction.java

public static Function<LlmClient.BeforeLLMEvent, LlmClient.LLMOverride> compactor(Compaction.Options opts)

public static int estimateTokens(List<Object> messages)

Returns a beforeLLM hook (§8) that summarizes the older part of a growing transcript and keeps a recent tail, so a long-lived agent never overflows the model’s context window. It rides the existing hook seam — no new loop behavior — by replacing the working messages array, which then flows into RunResult.messages like any other beforeLLM rewrite. Below Options.maxTokens it returns null (no-op, byte-identical to no compactor at all). Above it, the compacted transcript is [leading system prompt (verbatim), summary system message, (flush reminder?), …tail], holding two invariants: the tail always starts at a user turn (so no tool message is ever orphaned from the assistant carrying its tool_call_id), and a leading system message is preserved verbatim.

When to use it

A persona or long-running agent whose transcript keeps growing turn after turn — a support agent staying live across a whole shift, a coordinator delegating dozens of subtasks — will eventually exceed the model’s window. Hand compactor(opts) to LlmClient.Options.hooks.beforeLLM (or an agent’s own hooks, per §7D) and the loop keeps the transcript under budget automatically, summarizing only when it actually needs to.

Why this and not the alternative

compactor for a live transcript, the memory tool for facts across sessions

compactor operates on the live in-flight transcript of a single run — it never touches disk and adds no state a fresh process could read. When a fact needs to survive past the current session (a stated preference, an onboarding note), that’s the §7E Agents.memoryTool — set Options.flushToMemory to have the compactor remind the model to persist anything worth keeping just before the older turns are summarized away.

Examples

1. The smallest useful call — under budget is a byte-identical no-op

import io.github.muthuishere.toolnexus.Compaction;
import io.github.muthuishere.toolnexus.LlmClient;
import java.util.List;
import java.util.Map;
import java.util.function.Function;

public class Example {
    public static void main(String[] args) {
        Function<List<Object>, String> summarize = older -> "summarized " + older.size() + " messages";

        var hook = Compaction.compactor(new Compaction.Options()
                .maxTokens(100_000) // far above what three short messages need
                .summarize(summarize));

        List<Object> messages = List.of(
                Map.of("role", "system", "content", "Be terse."),
                Map.of("role", "user", "content", "hi"),
                Map.of("role", "assistant", "content", "hello"));

        LlmClient.LLMOverride out = hook.apply(new LlmClient.BeforeLLMEvent(messages, List.of(), "m", 0));

        if (out != null) throw new AssertionError("under budget must be a no-op, got: " + out);

        System.out.println("ok: no-op below budget");
    }
}

2. The realistic case — above budget: summarize the head, keep a safe tail

import io.github.muthuishere.toolnexus.Compaction;
import io.github.muthuishere.toolnexus.LlmClient;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;

public class Example {
    private static Map<String, Object> msg(String role, String content) {
        Map<String, Object> m = new LinkedHashMap<>();
        m.put("role", role);
        m.put("content", content);
        return m;
    }

    public static void main(String[] args) {
        // A transcript of many user/assistant pairs, padded so it overflows a small budget.
        List<Object> messages = new ArrayList<>();
        messages.add(msg("system", "You are a support agent."));
        for (int i = 0; i < 20; i++) {
            messages.add(msg("user", "question " + i + " " + "pad ".repeat(20)));
            messages.add(msg("assistant", "answer " + i + " " + "pad ".repeat(20)));
        }

        Function<List<Object>, String> summarize = older -> "summarized " + older.size() + " messages";
        var hook = Compaction.compactor(new Compaction.Options()
                .maxTokens(200)
                .keepTail(80)
                .summarize(summarize));

        LlmClient.LLMOverride out = hook.apply(new LlmClient.BeforeLLMEvent(messages, List.of(), "m", 0));

        if (out == null) throw new AssertionError("expected compaction above budget");
        List<Object> compacted = out.messages();

        // [system, summary, ...tail] — the leading system prompt survives verbatim.
        if (!"system".equals(role(compacted.get(0)))) throw new AssertionError("system prompt not first");
        Object systemContent = ((Map<?, ?>) compacted.get(0)).get("content");
        if (!"You are a support agent.".equals(systemContent)) {
            throw new AssertionError("system prompt content changed: " + systemContent);
        }
        String summaryContent = String.valueOf(((Map<?, ?>) compacted.get(1)).get("content"));
        if (!summaryContent.startsWith("[Summary of earlier conversation]")) {
            throw new AssertionError("expected a summary system message: " + summaryContent);
        }
        if (compacted.size() >= messages.size()) throw new AssertionError("compaction did not shrink the transcript");
        // the retained tail always starts at a user turn (tool-pair safety, §7F)
        if (!"user".equals(role(compacted.get(2)))) throw new AssertionError("tail must start at a user turn");

        System.out.println("ok: " + messages.size() + " -> " + compacted.size() + " messages");
    }

    @SuppressWarnings("unchecked")
    private static String role(Object m) {
        return (String) ((Map<String, Object>) m).get("role");
    }
}

3. The full surface — wired into a real client run through hooks.beforeLLM

import com.sun.net.httpserver.HttpServer;
import io.github.muthuishere.toolnexus.*;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;

public class Example {
    public static void main(String[] args) throws Exception {
        AtomicInteger hits = new AtomicInteger(0);
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
        server.createContext("/", ex -> {
            int n = hits.incrementAndGet();
            String text = n == 1 ? "First." : "Second, with a compacted history.";
            byte[] body = ("{\"choices\":[{\"message\":{\"content\":\"" + text + "\"}}]}")
                    .getBytes(StandardCharsets.UTF_8);
            ex.getResponseHeaders().add("Content-Type", "application/json");
            ex.sendResponseHeaders(200, body.length);
            try (OutputStream os = ex.getResponseBody()) { os.write(body); }
        });
        server.start();

        try (Toolkit tk = Toolkit.create(new Toolkit.Options())) {
            Function<List<Object>, String> summarize = older -> "summarized " + older.size() + " messages";
            var compactor = Compaction.compactor(new Compaction.Options()
                    .maxTokens(20) // deliberately tiny — the second turn's history overflows it
                    .keepTail(10)
                    .summarize(summarize));

            LlmClient client = LlmClient.create(new LlmClient.Options()
                    .baseUrl("http://127.0.0.1:" + server.getAddress().getPort())
                    .style("openai").model("test-model").apiKey("test-key")
                    .hooks(new LlmClient.Hooks().beforeLLM(compactor)));

            LlmClient.RunResult first = client.run("What's 2+2?", tk);
            if (!first.text.equals("First.")) throw new AssertionError(first.text);

            // Continue the SAME transcript — large enough now that the compactor engages.
            LlmClient.RunResult second = client.run("And double it?", tk, first.messages);
            if (!second.text.equals("Second, with a compacted history.")) throw new AssertionError(second.text);
            if (hits.get() != 2) throw new AssertionError(hits.get());

            System.out.println("ok: " + first.text + " | " + second.text);
        } finally {
            server.stop(0);
        }
    }
}

Options

Option	Type	What it is
maxTokens	int	Compact only when the estimate exceeds this; at/below ⇒ no-op.
keepTail	Integer	Keep at least this many tokens of the most recent tail. Default maxTokens/2.
summarize	Function<List<Object>, String>	Required. Produces the summary of the older messages; MAY call an LLM.
countTokens	Function<List<Object>, Integer>	Token estimator; default Compaction::estimateTokens (ceil(chars/4) per message, summed).
flushToMemory	boolean	When set, injects a pre-compact reminder to persist durable facts via the §7E memory tool. Off by default.

See also

• LlmClient.Hooks — The lifecycle middleware compactor plugs into via beforeLLM.
• Agents.memoryTool — Where flushToMemory tells the model to persist facts before they’re summarized away.
• LlmClient.run — The loop whose transcript (RunResult.messages) a compactor keeps bounded across turns.