browser-llm-nexus

One interface. The browser is the messy part.

Running a model in a tab means absorbing four kinds of variation at once: where the model comes from, which backend the browser has, which quantization actually works, and which call syntax the model family speaks. Most projects handle one of them and leave the other three in your code.

This library’s job is to absorb all four, so the code you write does not change when any of them does.

npm install browser-llm-nexus

import { NexusChat } from 'browser-llm-nexus';

const chat = await NexusChat.loadForTools({ hub: 'onnx-community/Qwen3-0.6B-ONNX' });

chat.tool('get_weather', 'Get current weather for a city',
  { city: 'string' },
  async ({ city }) => (await fetch(`/api/weather?c=${city}`)).json());

chat.on('token', t => render(t));
console.log(await chat.chat('What is the weather in Chennai?'));

Any source

The Hub, a folder you serve, a zip URL, or a file the user picked off a USB stick. Four ways to say where a model is; one way to load it. Nothing is guessed — there is no default host and no assumed path. Model sources

Any backend

WebGPU when the browser exposes a usable adapter, WASM/CPU when it doesn’t, with no code change. GPU is an accelerator here, never a requirement — which matters on the locked-down laptop where “this must not leave the page” is the whole point. GPU or CPU

Any model family

Qwen/Hermes <tool_call>, Mistral [TOOL_CALLS], Llama bare JSON, fenced JSON and nested OpenAI-style objects all parse to the same ToolCall. Register a plain function; the dialect is not your problem. Tool calling

Ships offline in a zip

Export a model, a vector index, or a whole knowledge base to a portable archive. Import it on an air-gapped machine — nothing re-embedded, nothing downloaded. Knowledge & bundles

The same four lines, whatever you point it at

await NexusChat.load({ hub: 'onnx-community/Qwen3-0.6B-ONNX' });  // Hugging Face
await NexusChat.load({ base: '/models/', id: 'Qwen/Qwen3-0.6B' }); // your own host
await NexusChat.load({ archive: 'https://host/model.zip' });       // a portable zip
await NexusChat.load({ archive: fileTheUserPicked });              // a file on disk

No server of ours sits behind any of these. No weights are bundled. Transformers.js is an injectable peer dependency, so you can pin it, patch it, or bundle it offline.

Verified, not assumed

A common interface is only worth having if what comes out the other side actually works.

Which quantization a model can call tools at is model-specific and inverted between models — Qwen2.5-0.5B works at q4 and is useless at q8; Qwen3-0.6B is the exact reverse. No ordering wins everywhere, so loadForTools() stops guessing: it loads a candidate, hands it a throwaway tool returning an unguessable token, and keeps the first one that both calls the tool and answers from the result.

const chat = await NexusChat.loadForTools(source);   // proven, or it throws
const check = await chat.selfCheck();                // ask any loaded model directly

If nothing works it throws and names every quantization it tried and why. It will not hand you a chat object that silently cannot do the job.

We publish what we measured rather than a curated list of favourites — including the combinations that fail, because a table with no failures in it is a marketing page. Which models actually work

Getting a model into the layout

Most models on the Hub ship PyTorch weights, not ONNX. hf2browser is an open-source converter that produces exactly the layout above and hands you the result as a single zip this library loads directly. It’s a sister project, not a dependency. Converting and serving

The sister library

toolnexus gives any LLM tools. browser-llm-nexus gives you the LLM. One runs on your server in six languages, one runs in the tab with no server at all — and both emit the same OpenAI function schema. See how they fit together.