LlmClient.ConversationStore

Java · package io.github.muthuishere:toolnexus · SPEC §8 · LlmClient.java

public interface ConversationStore {
    List<Object> get(String id);
    void save(String id, List<Object> messages);
}

public static final class InMemoryConversationStore implements ConversationStore { /* default */ }

// LlmClient.Options
public Options store(ConversationStore v)
// LlmClient
public ConversationStore conversationStore()

The pluggable persistence seam behind ask(prompt, toolkit, id) and stream(prompt, toolkit, onEvent, id): a two-method interface — get(id) loads a transcript, save(id, messages) persists it. The client ships InMemoryConversationStore as the default (process-lifetime only); implement the interface yourself to back it with a file, a database, or Redis.

When to use it

Whenever a conversation must survive past this process — a chat app that resumes after a deploy, a worker that picks up a paused session on a different machine. Implement ConversationStore, pass it as Options.store(...), and every ask/stream call with the same id now reads and writes through your backing store instead of an in-memory map.

Why this and not the alternative

Reach for Conversation when the process IS the lifetime

LlmClient.Conversation keeps its transcript in a plain Java field — simplest option when the conversation only needs to live as long as the object does. ConversationStore is the seam to reach for the moment “as long as this process runs” stops being good enough.

client.conversationStore() (§8 Gap 4) returns the EXACT store instance the client is using — your custom one, or the default in-memory one it built — so a caller can read/write it directly without a shadow copy.

Examples

1. The smallest useful call — a custom store, wired through ask

import com.sun.net.httpserver.HttpServer;
import io.github.muthuishere.toolnexus.*;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

public class Example {
    static final class LoggingStore implements LlmClient.ConversationStore {
        final Map<String, List<Object>> backing = new HashMap<>();
        int gets, saves;

        @Override public List<Object> get(String id) {
            gets++;
            return backing.get(id);
        }

        @Override public void save(String id, List<Object> messages) {
            saves++;
            backing.put(id, new ArrayList<>(messages));
        }
    }

    public static void main(String[] args) throws Exception {
        AtomicInteger hits = new AtomicInteger(0);
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
        server.createContext("/", ex -> {
            int n = hits.incrementAndGet();
            String text = n == 1 ? "Sure, starting a list." : "Added it to the list.";
            byte[] b = ("{\"choices\":[{\"message\":{\"content\":\"" + text + "\"}}]}").getBytes(StandardCharsets.UTF_8);
            ex.getResponseHeaders().add("Content-Type", "application/json");
            ex.sendResponseHeaders(200, b.length);
            try (OutputStream os = ex.getResponseBody()) { os.write(b); }
        });
        server.start();
        int port = server.getAddress().getPort();

        LoggingStore store = new LoggingStore();
        try (Toolkit tk = Toolkit.create(new Toolkit.Options())) {
            LlmClient client = LlmClient.create(new LlmClient.Options()
                    .baseUrl("http://127.0.0.1:" + port)
                    .style("openai")
                    .model("test-model")
                    .apiKey("test-key")
                    .store(store));

            client.ask("start a todo list", tk, "user-42");
            client.ask("add milk", tk, "user-42");

            if (store.gets != 2) throw new AssertionError("expected 2 loads, got " + store.gets);
            if (store.saves != 2) throw new AssertionError("expected 2 saves, got " + store.saves);
            if (store.backing.get("user-42").size() < 4) throw new AssertionError("expected an accumulated transcript");

            System.out.println("ok: " + store.gets + " gets, " + store.saves + " saves");
        } finally {
            server.stop(0);
        }
    }
}

2. The realistic case — surviving a “process restart”

A second LlmClient instance, pointed at the SAME backing store, picks the conversation right up — the continuity lives in the store, not in the client.

import com.sun.net.httpserver.HttpServer;
import io.github.muthuishere.toolnexus.*;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicInteger;

public class Example {
    public static void main(String[] args) throws Exception {
        AtomicInteger hits = new AtomicInteger(0);
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
        server.createContext("/", ex -> {
            int n = hits.incrementAndGet();
            String text = n == 1 ? "Got it, I'll remember 7." : "You said 7.";
            byte[] b = ("{\"choices\":[{\"message\":{\"content\":\"" + text + "\"}}]}").getBytes(StandardCharsets.UTF_8);
            ex.getResponseHeaders().add("Content-Type", "application/json");
            ex.sendResponseHeaders(200, b.length);
            try (OutputStream os = ex.getResponseBody()) { os.write(b); }
        });
        server.start();
        int port = server.getAddress().getPort();

        LlmClient.ConversationStore shared = new LlmClient.InMemoryConversationStore();

        try (Toolkit tk = Toolkit.create(new Toolkit.Options())) {
            LlmClient clientA = LlmClient.create(new LlmClient.Options()
                    .baseUrl("http://127.0.0.1:" + port)
                    .style("openai")
                    .model("test-model")
                    .apiKey("test-key")
                    .store(shared));
            LlmClient.RunResult first = clientA.ask("remember the number 7", tk, "sess");

            // A brand-new client instance ("after a restart") — same backing store.
            LlmClient clientB = LlmClient.create(new LlmClient.Options()
                    .baseUrl("http://127.0.0.1:" + port)
                    .style("openai")
                    .model("test-model")
                    .apiKey("test-key")
                    .store(shared));
            LlmClient.RunResult second = clientB.ask("what number did I say?", tk, "sess");

            if (!second.text.equals("You said 7.")) throw new AssertionError(second.text);
            if (second.messages.size() <= first.messages.size()) throw new AssertionError("history should grow");

            System.out.println("ok: " + first.text + " | " + second.text);
        } finally {
            server.stop(0);
        }
    }
}

3. The full surface — reading and seeding conversationStore() directly

import com.sun.net.httpserver.HttpServer;
import io.github.muthuishere.toolnexus.*;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class Example {
    public static void main(String[] args) throws Exception {
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
        server.createContext("/", ex -> {
            byte[] b = "{\"choices\":[{\"message\":{\"content\":\"Continuing tersely.\"}}]}".getBytes(StandardCharsets.UTF_8);
            ex.getResponseHeaders().add("Content-Type", "application/json");
            ex.sendResponseHeaders(200, b.length);
            try (OutputStream os = ex.getResponseBody()) { os.write(b); }
        });
        server.start();
        int port = server.getAddress().getPort();

        try (Toolkit tk = Toolkit.create(new Toolkit.Options())) {
            // No store option => the client's own default in-memory store.
            LlmClient client = LlmClient.create(new LlmClient.Options()
                    .baseUrl("http://127.0.0.1:" + port)
                    .style("openai")
                    .model("test-model")
                    .apiKey("test-key"));

            LlmClient.ConversationStore store = client.conversationStore();
            if (store.get("preloaded") != null) throw new AssertionError("expected nothing stored yet");

            // Seed a transcript directly, bypassing ask() — e.g. importing history from elsewhere.
            List<Object> seeded = new ArrayList<>();
            seeded.add(Map.of("role", "system", "content", "You are terse."));
            seeded.add(Map.of("role", "user", "content", "hi"));
            seeded.add(Map.of("role", "assistant", "content", "hi."));
            store.save("preloaded", seeded);

            LlmClient.RunResult res = client.ask("continue", tk, "preloaded");

            if (res.messages.size() <= seeded.size()) throw new AssertionError("ask should extend the seeded transcript");
            if (!store.get("preloaded").equals(res.messages)) throw new AssertionError("store should hold what ask() wrote back");

            System.out.println("ok: seeded " + seeded.size() + " -> " + res.messages.size() + " messages");
        } finally {
            server.stop(0);
        }
    }
}

Fields

Member	Type	What it is
get(id)	List<Object>	Return the stored transcript for id, or null if none.
save(id, messages)	void	Persist the (updated) transcript for id.
InMemoryConversationStore	ConversationStore	The shipped default — process-lifetime only, ConcurrentHashMap-backed.
Options.store(v)	—	Wire a custom store into the client; null ⇒ the default in-memory store.
client.conversationStore()	ConversationStore	The exact instance in use — yours, or the default the client created. Read/write it directly to share state with ask/stream.

See also

• LlmClient.create — The unified client: system prompt, skills injection, parallel and chained tool calls, retries, memory.
• LlmClient.run — Send a prompt, let the loop call tools until the model stops, get a RunResult.
• LlmClient.stream — The streaming loop: text deltas, tool-call events, and suspension events as they happen.
• LlmClient.Hooks — Intercept before/after model calls and tool calls: audit, redact, veto, or rewrite.